What to Do When Your Security is Breached 177
ancientribe writes "When you've got a full-blown security breach on your hands, what do you do? If you've been smart, you'll already have a computer security incident response team — and a plan — in place. But many companies are too resource-strapped to have a full-blown, fully-tested incident response strategy. DarkReading has some tips on what to do — and what not to do."
The problem is (Score:5, Insightful)
A plan may not apply (Score:4, Insightful)
I'd note that even if your company has a response plan, you may find it either completely useless or so general that it doesn't provide any help. Look at the article's point #1: it's almost nothing but "If $X, you may need $Y.". And it's far from complete. That's going to be a flaw in any security response plan: it's likely to not address the actual problem you face. Problems that you've thought of tend to get caught earlier before they turn into full-blown incidents, it's the ones nobody thought of that are most likely to bite you badly and it's exactly those that a plan won't cover. About the only part of the plan that'll be guaranteed to be useful is the part explaining what parts of the system are responsible for what and how to lock them down to preserve the evidence while you figure out where the breach is and what you need to do next. Beyond that you're into a twisty maze of little possibilities, all almost but not quite completely unlike each other, and what you need most isn't a plan but someone with enough Clue to analyze the situation and formulate a plan to fit it on the fly.
Re:The problem is (Score:4, Insightful)
I would further add, that they chose Microsoft because Microsoft promises lower TCO through lowered administrative (geek) needs.
I suppose that most Microsoft shops wouldn't even know if they were breached, because most breaches don't actually desctroy data, they just steal it.
Outsource (Score:3, Insightful)
Disconnect and reinstall... (Score:4, Insightful)
It's been a long time (thankfully) since I've had to deal with this. But I'd echo the article about disconnecting from the net to eliminate further attacks. Then I'd remove the drive and save it for forensics -- replacements are cheap (I'm assuming a small business doesn't have expensive RAID setups). Assume that everything has been compromised and restore from a backup prior to the intrusion (hopefully you can tell when that was).
Oh, and keep your clocks synchronized. This will help if you need to trace intrusions across systems.
Don't panic! (Score:5, Insightful)
First, find out the extent of the breach. Analyze your log files. Find out what time it happened. Find out who was logged in at the time, and find out any log messages from any system services that can help you figure out what the problem was. If you can't figure out what the scope of the breach was with a high level of confidence, then you have to assume the worst: the entire network is compromised.
Second, salvage what you can. Again, be very careful about doing this. Hopefully you have a backup somewhere which would allow you to avoid or shorten this step as much as possible. In essence, do what you have to do to the compromised machine to avoid losing work, but always be concious of the fact that the machine is compromised, and may be transmitting or recording keylogs or other sensitive information. If possible, disconnect the compromised machines from the Internet and isolate it from the rest of your LAN.
Third, plan for the future. How would this breach be avoided in the future? Was it an OS problem? If so, then maybe you need to install OpenBSD instead. Was it a problem with a particular package you were using? Choose a different package. Can you configure your firewall or server to prevent or limit the abuse that caused the problem in the first place (e.g. fail2ban to deal with SSH phishing attacks) or install monitoring software to alert you of a problem (e.g. an IDS like Snort)? Do your users need further training? Does your password policy allow weak passwords? Etc.
Finally, take a deep breath. Unless you've been totally negligent in your job, there wasn't much you could do to prevent it. Don't worry about the fact that you don't have enough to go to the police; most Network Administrators don't have the hardware, training or certification to present evidence in a courtroom anyway. If you can go to the cops, then bully for you! Make that black-hat asshole pay!
not necessarily (Score:4, Insightful)
er, don't do that (Score:1, Insightful)
From personal experience (unfortunately the personal experience came before the Red Cross training), running cold water over a burn causes excruciating pain about 30 seconds after the cold source is removed. My theory is that the cold constricts blood flow, and after you remove the cold source, the blood starts coming back through the damaged tissue area and oh my god does it hurt.
Huh? Reinstall ofcourse! (Score:2, Insightful)
Re:I love these content-free articles (Score:5, Insightful)
1) You apparently shouldn't rely on what you 'figured out on your own'.
2) In addition to getting a plan for a security breach you should also look at getting some help with your first aid plan too.
Re:Anyone ever followup with law enforcement agenc (Score:5, Insightful)
You need to get the subpoena to identify the person behind the attack. That assumes that your evidence actually points to a specific suspect. Unless your attacker was a complete moron, or your network logs are incredibly voluminous, that's not very likely. Once the subpoena is served and you've got your suspect and laid charges, you need to present evidence. That requires an expert witness. If you're lucky, YOU are the expert witness, but there's training and certification involved in that process. Otherwise, you get to hire an expert witness, and that won't be cheap. Your opponent will probably hire an opposing expert, just to confuse everybody.
Overall, I'd say that chances of success are incredibly low. Legal fees will be very high, and you have to turn over a fair chunk of your network assets to Law Enforcement. Basically, if you aren't really, really sure that you've got your man, it's really not worth the time and effort to find out who it was. That effort is much better spent allowing you to sleep at night knowing that people aren't getting in, IMO.
A good start, but... (Score:4, Insightful)
Re:The problem is (Score:2, Insightful)
Seems to me the problem was an incompetent system administrator and not the OS.
Re:I dunno, it's sorta... news to me (Score:5, Insightful)
Depending on what you want to accomplish, pulling the plug or the network cable isn't something you want to do. If you want to catch the people who did it, instead of just minimize the damage, you need to approach this from a forensics POV. If you power-off the system, you lose everything that is stored in memory, which may be the only location where an important email, webpage or IP address is stored. Without this information it may not be possible to track-down the attacker. Yes, if they are communicating directly with the machine, you can get this info from a router or even the ISP but, if they are using some sort of anonymizer, you can't. Also, the rootkit (or whatever) may have a self-destruct built-in; can't communicate for 3 minutes, delete and overwrite everything. This would mean pulling the network cable will destroy any important information on your system. You might have backups for your data, but you don't for the attacker's information.
Another important consideration is that powering down the system may prevent any information that's gathered from being admissible in court (U.S. jurisdiction). For example, can you guarantee that the email address on the disk is the attackers email, or is it from an email sent or received, or something else. Since you didn't shutdown properly, you may not be able to claim that the address is really attacker124@gmail.com, but might be attacker123, or attacker224, etc. - meaning no warrant and no charges. There are devices out there that you can plug into a USB port that will attempt to copy everything from RAM just so you have a complete record - then you can pull the plug, since that will prevent the hard drive from being written to. This preserves the information and it can be used as evidence. Whatever you do, don't do a normal shutdown.
So, a reason you might want to wait for your lawyers and HR people is to determine if you need to worry about prosecution, or just make the problem go away. If they compromised an old desktop, or the web server in your DMZ, you might decide that it isn't worth it to pursue a conviction - lawyer's call - they know how expensive/difficult it will be. If the system holds personal information, the HR guy may need to help make the call. Ex. - Do you have to report a breach to all of your customers? Just employees? No reporting required, it isn't the info designated under the laws and/or regulations. Now, if it is a development server, you might want to leave it live if you suspect corporate espionage. You can bring in the feds and let them assess the situation. You might also want to buy time to work with you ISP to trace the attack. You actions should be done based on what the server contains and its value - which is why you have the CIO or CEO in the room.
Now, a lot of this may not apply to your situation, but it isn't a black and white issue. There are a lot of things to consider. If you want some good information, I would recommend any of Brian Carrier's work - papers and his book. I have read a couple of his papers and they were really good and, while I haven't read his book, it has been recommended to me by others.
easy... (Score:2, Insightful)
#1 advise (Score:5, Insightful)
My advise to sysadmins who notice a breach is this:
Take your hands off the damn keyboard. Don't do anything unless you are 150% certain that you can see all possible consequences of your action. Call the IRT if you have one.
If there's nobody to call and you have to act right now, pull the power plug on the machine, then call the experts. Don't power the machine up again under any circumstances. If you want to look at the harddrive, make a copy first and mount it read-only in a different machine.
Why? Because back in the days when I was, err... looking around inside machines not my own, one of the things everyone I knew did was put in some scripts, tools, something, that'd wipe the logs or even the machine if my shell gets killed or the machine shut down or rebooted.
TFA assumes that you learn of the incident long after it has happened. Many incidents in real life are being noticed while they are going on, no matter if it's a remote access or your machine running an FTP server that wasn't there last month. That FTP server is almost certainly patched, and one of the things it might do is destroy evidence if you kill it. There might be an invisible process watching it to wipe evidence if you kill -9 it. Heck,