Cleaning Up the Mess After a Major Hack Attack 100
Hugh Pickens writes "Kevin Mandia has spent his entire career cleaning up problems much like the recent breach at Stratfor where Anonymous defaced Stratfor's Web site, published over 50,000 of its customers' credit card numbers online and have threatened to release a trove of 3.3 million e-mails, putting Stratfor is in the position of trying to recover from a potentially devastating attack without knowing whether the worst is over. Mandia, who has responded to breaches, extortion attacks and economic espionage campaigns at 22 companies in the Fortune 100 in the last two years and has told Congress that if an advanced attacker targets your company then a breach is inevitable (PDF), calls the first hour he spends with companies 'upchuck hour' as he asks for firewall logs, web logs, and emails to quickly determine the 'fingerprint' of the intrusion and its scope. The first thing a forensics team will do is try to get the hackers off the company's network, which entails simultaneously plugging any security holes, removing any back doors into the company's network that the intruders might have installed, and changing all the company's passwords. 'This is something most people fail at. It's like removing cancer. You have to remove it all at once. If you only remove the cancer in your leg, but you have it in your arm, you might as well have not had the operation on your leg.' In the case of Stratfor, hackers have taken to Twitter to announce that they plan to release more Stratfor data over the next several days, offering a ray of hope — experts say the most dangerous breaches are the quiet ones that leave no trace."
Clean up? Start fresh (Score:3, Insightful)
It's not a huge deal here anyway - because this lot have a high profile everyone forgets how small they really are. Your local newspaper probably has a bigger operation and a hell of a lot more subscribers.
And, as usual... (Score:5, Insightful)
Re:Clean up? Start fresh (Score:5, Insightful)
Not a problem here. we simply re store the workstation boot image from the creation CD and run all the updates on it.
Thumb drives, not a problem, thumb drives dont work here.
as for switches, I can update ios on every switch in 60 seconds. not a hard thing to do.
as for the "backups" problem. I have yet to see a hacker that can infect a machine using an odf file, I'm not backing up ANY executables.
Honestly I can do a complete wipe and restore in under 5 days for a company that has 1000 employees and 20 servers. IF the IT department was set up and run by competent people.
If it's a typical cluster-turd... far far longer.
Re:And, as usual... (Score:5, Insightful)
Which is why I send the email 10 times with a receipt request. Boss is too stupid to turn off that feature, and I also get a reply from him saying, "PLEASE STOP EMAILING ME THIS!"
Never EVER trust your boss. he will burn you to save his own butt.
Re:Clean up? Start fresh (Score:4, Insightful)
Clean installs on everything, new passwords, and don't trust anything executable that has been on the compromised machines anywhere near the time it was hacked. It's not a huge deal here anyway - because this lot have a high profile everyone forgets how small they really are. Your local newspaper probably has a bigger operation and a hell of a lot more subscribers.
I seriously doubt your local newspaper has more money involved - or any local newspaper. Maybe some of the national broadsheets - but that's a moot point.
Cleanups aren't complicated - but fixes are - they just sound simple. And most commonly people seem to believe they are the same thing - I contend that they're not.
In my experience these things happen again and again to the same companies (though the majority put a lot of effort into keeping it secret). Not the same dog each time, but definitely the same leg action.
I've done a bit of due diligence on companies, listened in on workers at lunch, chatted to ex-staff, and hired investigators - and I've found few that are as clean as presented - it's like buying a pub where the bartenders or staff don't dip into the till, or regulars (and staff) have never dealt in drugs (rare as hen's teeth).
I'm not talking about defending against attackers - and I don't dispute that a determined, well resourced, intelligent attack will always succeed if time permits (it's like robbing armoured cash vans really - or so I've heard). I'm talking about the things that make it easy for attackers - I believe that if you raise the bar enough - all the hurdlers don't get better - just a few of them (and when you're robbed you're robbed, so number of occurrences is important)
What interests me is why there's always talk of plugging gaps and fixing procedures - but never any mention of fixing the primary problem. The primary problem being institutional psychology. Like storing your beer on the nature strip it having it stolen (surprise - people want your beer). Then "cleaning up" by making sure all liquor is secured inside the premises, and "fixing" the problem by telling people to store their beer in the fridge and lecturing them on physical security. It overlooks the possibility that only an untrustworthy idiot would put beer on the nature strip in the first place - and even if they don't put it on the nature strip again they will probably lose a house key, or leave a window open.
*1 I don't believe lazy, stupid staff change if you send them to motivation and inspiration seminars either, certainly I've seen no evidence to support it.
I'm working on a theory that dumb travels downward - I call it "The Argument from Moron Motion"
Re:Clean up? Start fresh (Score:4, Insightful)
> Imagine that you have 1000 employees. Every workstation, every server, every switch, every usb-stick, every external drive could hold the seed to restoring hacker control on your network. You'd have to wipe all of them before allowing them to reconnect to the network.
I wish people would remember this when they claim company's estimates of damage from a cracked system are excessive. You can bring an entire company to a standstill for an extended period of time by requiring (unless as a customer you're just fine with them taking risks with your data?) multiple critical systems to be isolated and rebuilt from scratch at the same time, even if there's no clear damage done, because you have no other way of verifying they're clear.
In a high security environment, destroying the physical machines to be sure (tampered firmware, stuff hidden in bad blocks on the hard drive, or who knows what else) is probably a sensible move.
Re:More effort into tracking and capturing hackers (Score:4, Insightful)
Uhm no, mere vandals need to be cherished and promoted; those who work for the Chinese govt won't tell you something is amiss.
It is the companies' fault for not following basic security practices, especially if what they take taxpayers' money for is "intelligence".
Re:Government warnings?? (Score:2, Insightful)
You miss the point.
If someone has access to a corporate network, and is smart, they're not going to blow it by using that companies internet facing machines to start running portscans on DoD machines, well, not unless they are script-kiddy stupid.
So, the target network is breached surreptitiously and information is quietly pilfered....al la corporate espionage........how's the DoD ever going to know ?
They should say "The government tells 90% of the small subset that do something stupid like launch DDoS attacks on DoD systems straight from the compromised machines.......the rest, no-one probably knows about".
Re:And, as usual... (Score:5, Insightful)
Any job that requires a CYA email archive is not worth having.
Re:Clean up? Start fresh (Score:5, Insightful)
Honestly I can do a complete wipe and restore in under 5 days for a company that has 1000 employees and 20 servers.
That's not too bad. But of course any machine that's not been wiped and restored can not be allowed on the network. And for the employees that means up to five days of not being able to do much. That's a long time to wait.
Re:Clean up? Start fresh (Score:5, Insightful)
The trick is to recognize that you cannot make these mistakes. Period. No matter who you have to run over, who you have to piss off, who you have to overrule, who you have to upset, no matter what. You have to be, and yes I am, an arrogant bastard. Because the moment you compromise, you're doomed. We've seen it over and over and over and over again, we're seeing it again today, we'll see it again tomorrow. Every single data breach incident I've ever read about included at least one of those six mistakes, and most of them included several. Yet incompetent, weak-willed IT people insist on making them because "we've always done it this way" or "that can't work!" or "but it would break..." or for a thousand other reasons...none of which matter. (What good is having a spiffy computing environment if it's not secure?)
The problem isn't that we don't know what to do. We do. The problem is lack of will to do it.