Friday, March 6, 2009

Do you trust me?

For most of us, the System Administrators, a.k.a Sysadmins, are life-savers. They reset our passwords when we forget them, recover our files when we delete them and sometimes give us a hard time about it. For corporate management, however, this kind of power can be frightening. An administrator would usually have access to every bit of information in the company, including every employees employment and HR data, personal email, and usually customer data as well. This kind of power, if abused, can cause irreparable damage to a company, but despite that, most companies interview and screen their sysadmin just like any other employee. If, for some reason, this employee becomes bitter or estranged, there's no telling what could happen, and there have been documented cases where entire companies have been complete destroyed intentionally by their admins.


Can this happen to your company too? Possibly. CEOs and CIOs have been looking for ways to counteract this sort of threat for a while now. There is a logical problem here - if you don't trust your admin, and appoint someone to watch over him, then how do you trust that someone to not break bad? After all, even CEOs have been known to go astray and stick their hands into inappropriate pockets. Who shaves the barber?


There is no simple answer here, but generally, the answer has two parts. The logical solution is separation of powers. You appoint at least 2 or 3 administrators, and try to make sure they don't become too friendly with each other so there's less chance of collusion. One way to go about this is appointing people who are a world apart - big age difference, for example. Then, add to that job or responsibility rotation. For example, one can be appointed to manage the finance department servers, while the other owns the engineering servers, and then rotate those roles every 3-6 months. This way, if one used abuses these resources, it will most likely be revealed upon the next rotation. Another good practice is the force the administrators to go on vacation on a regular basis (and YES, it's totally worth to give them an extra few annual vacation days just for that). When the admin goes on vacation, someone else has to take over, and that would usually expose any foul play.


The 2nd part is technological - Use some system to track and log activity. This serves two purposes - people tend to mess around a lot less when they know they are being watched, and that will affect not only administrators, but also regular users. Secondly, if someone does go to the dark side, at least there will be a way to check what's been going on, and have evidence in case a law suit or criminal charges need to be filed. One such software solution is Intellinx, and another is InFlight. These solutions can record user activity directly from the network, including keystrokes and screen output from every station in the company.

Is any of that foolproof? Of course not. A smart crook can always find some way to scam his way around, and the only answer to this is to carefully build a security policy that tries to address each and every possible threat - external or internal. Another important lesson to be learned here is that the system administrator is a very sensitive position, and should be screened appropriately. The screening process should include not only technical evaluation, but also personality and psychological testing, and it wouldn't hurt to have this monitored on a regular basis too, especially if a big change has happened in the company. If you had your sysadmin fire half his technicians because the company is tight on money, you can bet he's preparing for the possibility of him being next on the chopping board, and his preparation might include stashing sensitive data or implanting backdoors into servers. Also, keep in mind that even a small-time technician that you are hiring today to haul some printers around might end up being the sysadmin in 10 years. That means that those guys should also be chosen carefully, and reviewed once again upon getting promoted. And speaking of Admins, a fun thing to read is the old classic BOFM, which tells some tails of a particularly nasty sysadmin.