Forgot your password?
typodupeerror
Government Security The Almighty Buck United States IT Hardware

Got Malware? Get a Hammer! 254

Posted by timothy
from the sorry-but-it's-the-only-way dept.
FuzzNugget writes "After the Economic Development Administration (EDA) was alerted by the DHS to a possible malware infection, they took extraordinary measures. Fearing a targeted attack by a nation-state, they shut down their entire IT operations, isolating their network from the outside world, disabling their email services and leaving their regional offices high and dry, unable to access the centrally-stored databases. A security contractor ultimately declared the systems largely clean, finding only six computers infected with untargeted, garden-variety malware and easily repaired by reimaging. But that wasn't enough for the EDA: taking gross incompetence to a whole new level, they proceeded to physically destroy $170,500 worth of equipment (PDF), including uninfected systems, printers, cameras, keyboards and mice. After the destruction was halted — only because they ran out of money to continue smashing up perfectly good hardware — they had racked up a total of $2.3 million in service costs, temporary infrastructure acquisitions and equipment destruction."
This discussion has been archived. No new comments can be posted.

Got Malware? Get a Hammer!

Comments Filter:
  • Sounds more like Fucking Retards Money Wasting Administration to me.

    • Re: (Score:3, Insightful)

      by ArcadeMan (2766669)

      And why the hell would there be $2.3 million in service costs to destroy $170,500 worth of equipment?

      • by gl4ss (559668)

        And why the hell would there be $2.3 million in service costs to destroy $170,500 worth of equipment?

        best buddy system.

        that's why.

        • by egamma (572162) <egamma@gm a i l . com> on Tuesday July 09, 2013 @11:05AM (#44225729)

          And why the hell would there be $2.3 million in service costs to destroy $170,500 worth of equipment?

          RTFS.

          service costs, temporary infrastructure acquisitions and equipment destruction

          Or, RTFA for the details:

          The total cost to the taxpayer of this incident was $2.7 million: $823,000 went to the security contractor for its investigation and advice, $1,061,000 for the acquisition of temporary infrastructure (requisitioned from the Census Bureau), $4,300 to destroy $170,500 in IT equipment, and $688,000 paid to contractors to assist in development a long-term response. Full recovery took close to a year.

          Still outrageously stupid, but I think $4,300 to destroy $170,500 is a reasonable cost. I think the other costs--the ones with 6 or 7 figures--are the ones you should focus on.

          But really, isn't giving US companies #2.3 million what the Economic Development Administration is supposed to be doing anyways? Better than spending it on the salaries for these government employees.

          • by gmuslera (3436)

            $823,000 for the security contractor that adviced them to do that destruction? I know that for police not having high IQ is a requirement [go.com], but seems that the standards are even lower in other places.

            • by icebike (68054) on Tuesday July 09, 2013 @12:54PM (#44227111)

              $823,000 for the security contractor that adviced them to do that destruction?

              Read the story, or at least read the summary.
              The contractor did not tell them to do that. The contractor found exactly 6 machines, which they recommended by re-imaged.

              This stupidity was not the contractors fault.

              • by Darinbob (1142669)

                The stupidity appears to be with the CIO firmly convinced that there is persistent malware present that can't be removed with reimaging. Ie, assumed that the worst case scenario exists with malware from a nation state that remains hidden and undetectable. There was also some confusion that advice to "rebuild your network" meant destroying the existing infrastructure.

                Sort of reminds me of the scene in Zoolander when the two idiots say "she said the files are IN the computer!" and then proceed to literally

        • by parkinglot777 (2563877) on Tuesday July 09, 2013 @01:59PM (#44227977)

          best buddy system.

          that's why.

          That could be true; however, have you read the audit paper written by OIG in PDF (http://www.oig.doc.gov/OIGPublications/OIG-13-027-A.pdf)? It is very interesting and contains what the auditor (OIG) thinks where to blame (although those who are at fault simply brush the responsibility to others). Everything seems to be from miscommunication between DOC CIRT and EDA, and both did not know about this miscommunication until too late (the end of 2012, about a year after the incident).

          What happened (from the audit paper) was that the incident handlers from DOC CIRT sent out 2 notifications to EDA regarding the US CERT notification. The first notification simply listed all 146 components, and EDA thought all of them were infected. Then the incident handlers from DOC CIRT sent the 2nd notification with accurate analysis of only 2 infected commponents, but the notification did not clarify or mention that the 1st notification was inaccurate (wrong). As a result EDA thought all 146 components were still infected.

          Then the EDA selected and submitted 2 components to the DOC CIRT as a process to verify whether they were infected. Apparently, the EDA submitted the 2 components mentioned in the 2nd notification, and the result came back positive. As a result, the EDA thought that all 146 components were infected.

          It got worse when EDA already knew that their IT system is outdated and needed a lot of updates/patches (since 2006 from NSA and OIG system reviewed) but they never fixed the issues. They believed this incident was an attack from nation-state actors (hackers), so their system could be extremely vulnerable to the attack. As a result, their system could open a hole to other systems' access. Therefore, the system was isolated.

          Keep in mind, the Chief Information Officer (CIO) believed that this incident is from hackers. Then the EDA hired an external security company (contractor) to come in and assess the situation/system. The contractor found no actual malware infections. However, the CIO of EDA asked for a guarantee that there is non-existing of infection at all in the system [CIO is trying to safe his behind because of his belief]. The contractor could not give a guarantee due to the different between "could not exist" and "did not exist" of infections. That let to destroying the hardware part.

          During the wait for recovery, the EDA entered into an agreement with Census to use their resources (e-mail, Internet, laptops, etc).

          This is not done yet (and not included in the summary of this topic). The EDA did not listen to the recommendation from NSA or DHS about recovery plan -- quickly & fully recovery IT system. The EDA wanted a whole new system. This would cost $26 millions in total and won't be finished until the end of FY2014.

          In summary, the miscommunication and other factors escalate the issue to be worse and worse. 1.DOC CIRT incorrectly handled the notification
          1.DOC CIRT did not admit that their 1st notification was wrong to EDA
          2.EDA did not verify the 2nd notification against the 1st with DOC CIRT
          3.EDA did not submit random components (from 146) for verification
          4.EDA IT system is outdated and has never been fixed/patched
          5.CIO of EDA wanted to cover his behind by asking for a guarantee which is unrealistic
          6.EDA wanted a whole new IT system which cost $26 millions

          What do these people learn from the incident? No punishment but simply recommendations Deputy Assistant Secretary and the CIO of EDA (page 17 of the report/page 22 of the PDF file)! This situation is very similar to a big corporation making a mistake, and as a result, tax payers paid the price and nobody who were involved in the incident was punished.

      • by omnichad (1198475)

        The actual destruction costs were only: $4,300 (still too much). The rest of that price tag is the total cost of doing the destruction - temporary infrastructure and so on. Not sure why a temporary replacement would cost 10x what was being replaced, though. Still plenty of government waste in the story.

        • by icebike (68054) on Tuesday July 09, 2013 @01:01PM (#44227223)

          The actual destruction costs were only: $4,300 (still too much). The rest of that price tag is the total cost of doing the destruction - temporary infrastructure and so on. Not sure why a temporary replacement would cost 10x what was being replaced, though. Still plenty of government waste in the story.

          Well except for the mice. You know how mice breed. Destroying those infected mice can take forever, because you find them breeding in closets, junk drawers, sometimes in their original boxes if bought at a TwoFer sale. And the wireless ones can be found a long way away from their nest, under desks, leaving their dongles everywhere.

          They were lucky they managed to nip the infestation in the bud. It could have gotten totally out of hand had they owned any traveling laptops with mice. Entire countries might need quarantine. One mouse on a plane, and its game over.

      • by Tridus (79566) on Tuesday July 09, 2013 @10:31AM (#44225291) Homepage

        Because, RTFA?

        "The total cost to the taxpayer of this incident was $2.7 million: $823,000 went to the security contractor for its investigation and advice, $1,061,000 for the acquisition of temporary infrastructure (requisitioned from the Census Bureau), $4,300 to destroy $170,500 in IT equipment, and $688,000 paid to contractors to assist in development a long-term response. Full recovery took close to a year."

      • by JDG1980 (2438906)

        And why the hell would there be $2.3 million in service costs to destroy $170,500 worth of equipment?

        From the original article:

        The total cost to the taxpayer of this incident was $2.7 million: $823,000 went to the security contractor for its investigation and advice, $1,061,000 for the acquisition of temporary infrastructure (requisitioned from the Census Bureau), $4,300 to destroy $170,500 in IT equipment, and $688,000 paid to contractors to assist in development a long-term response. Full recovery took

        • And they wasted an additional $1.5 million paying various "contractors" who apparently didn't know what they were doing.

          Or maybe they did, if you get my drift.

          • Re: (Score:3, Insightful)

            Yeah baby, it's a great way to stimulate the economy. We know whst gets done is less important than things get done, and money gets pushed from person to person.

            Buying computers to destroy employs people, as does destroying them. Hell, what we should do is just increase taxes and hire the tens of millions of unemployed to dig ditches and then fill them back in over and over.

          • obligatory Zorg [youtube.com] speech

      • Welcome to the magical world of government contract accounting and a little accounting term we like to call overhead.

    • by amiga3D (567632)

      The feds are pretty much incompetent at almost everything. The only thing that works is Department of Defense who's primary purpose is to blow things apart. Even they can't seem to buy new hardware without 3000% cost over runs although I think that's actually more of a corruption thing. All we need now is to completely federalize health care which should do wonders for ending the danger of overpopulation.

      • Re: (Score:2, Informative)

        by Anonymous Coward

        Most cost overruns are due to scope creep. Customer solicits bids, contractors bid, one wins, shortly after contract is awarded the customer changes requirements.

        General cycle is:
        Customer asks if they can change a requirement
        Contractor says it'll cost $$$ (usually a pretty big number, because many requirements are difficult to change after you've architected your system to the original requirements)
        Customer says "sure"
        Costs skyrocket.

        As an example, with the last presidential helicopter.
        Government requested

        • by Entropius (188861)

          A bit OT, but:

          It seems to be a symptom of some underlying pathology in a democracy when so much effort is put into protecting the head of government. At least in the ideal it doesn't matter who is president; they're ultimately a representative of the popular will and, to first order, one will do just as well as the next. There is even ideological continuity, since the vice president is selected by the president (you couldn't shoot Bush to end the Iraq war, since then you'd get Cheney). Historically assassin

          • by The Rizz (1319) on Tuesday July 09, 2013 @11:35AM (#44226103)

            It seems to be a symptom of some underlying pathology in a democracy when so much effort is put into protecting the head of government. At least in the ideal it doesn't matter who is president;

            You're completely missing the point of protecting the Head of State - it's not because an assassination would cause a change in policies, but to keep extremists from using threat of assassination to to blackmail a Head of State into changing those policies.

            In other words, if the POTUS has to fear for his life as a result of every decision he makes, he is going to be pressured to cater to the most radical and violent groups.

            • That and candidates are going to be pretty interesting personality types if security isn't visibly high. Sure though it could be interesting to have a president whose security consists of keeping a well practiced Colt at his side and his back always to the wall.

      • by Tridus (79566)

        The feds are over reliant on contractors for everything. Contractors are there to just milk as much money as they can out of the system. They do a pretty good job.

        • by mellon (7048) on Tuesday July 09, 2013 @10:40AM (#44225409) Homepage

          Yup. Likely what happened here is that the million-dollar security contractors gave the advice to do this bug hunt in the first place, and then provided the temporary replacement infrastructure, and walked away from the whole fiasco with a tidy profit. The reason this happens is because the government isn't generally allowed to hire people to do work like this, because "private industry is better." Of course, this sort of private industry is just a mechanism for siphoning off tax dollars, and the people who believe that hiring government employees to do government work is wasteful are actually responsible for fiascos like this, which are depressingly common.

          Even when the contractors aren't crooked, the cost of employing them instead of federal employees is typically several times higher. But "corporations good, government wasteful." If we keep repeating that long enough maybe it will come true.

      • by Anonymous Coward on Tuesday July 09, 2013 @10:36AM (#44225361)

        Devil's advocate:

        I've worked at private companies, for education institutions, in the public sector, and in the Federal government. None are perfect, none are completely horri-bad.

        All places have had those people who I had zero clue what their function was, but they always had a nice office.

        It is easy to pick on government, but go to almost any work environment, and you will find the same thing.

        • by ArsonSmith (13997)

          the main difference when things go bad either people get fired or businesses go under, in government when things go bad those people with no real job get raises.

          • by Chickan (1070300) on Tuesday July 09, 2013 @11:07AM (#44225753)
            Not always true. I've seen many incompetent people continue to get promoted in industry. The government ones just get more press.
          • the main difference when things go bad either people get fired or businesses go under, in government when things go bad those people with no real job get raises.

            Not true. Just look at all the Motorola execs who drove the company to the ground, all playing golf and going "caaachiiiiiin!". On a more plebeian note, people do not get fired for chronic incompetence in general. They get shuffled somewhere else.

            More to the point, in general very few people are actually utterly incompetent. There are occasional or at worst chronic incompetent people who by sheer brute force gets by. Sometimes their incompetence gets contained by giving them narrow tasks, like ant soldier

        • by Solandri (704621)
          Difference is when a private company pulls a stunt like taking down its entire IT system [slashdot.org], customers start to abandon it and head to a competitor. If they screw up badly enough, they go bankrupt and everyone who worked there is out of a job. That creates a huge incentive to do things in a manner least disruptive to their customers.

          When a government agency pulls the same stunt, they tell the customers "f- you, wait in line like a good citizen while we get everything worked out, because we're the governme
    • by timeOday (582209) on Tuesday July 09, 2013 @10:32AM (#44225309)
      Either that or the hardware was outdated and/or soon-to-be replaced anyways (like the CRT photo in the accompanying story), so they just went with the upgrade instead of spending money to verify old stuff.

      Any IT upgrade could be spun exactly like this story, if you wanted... "why did you get a new mouse with that new system, the old one was working perfectly fine and now it's going in the trash!"

    • economic development spurred by almost two and a half million dollars, and a few hammers... we'll have the complete story live at 10.

    • by synapse7 (1075571)
      Possibly, or were they trying to hide something substantial?
    • by ArsonSmith (13997)

      Hopefully they'll be the ones in charge of healthcare.

  • Wow! (Score:5, Funny)

    by Enry (630) <enry.wayga@net> on Tuesday July 09, 2013 @10:10AM (#44225003) Journal

    You mean I get to release my pent-up anger by destroying physical systems *and* get paid a boatload of money to do it? Where do I sign up?

    • by chill (34294)

      ... *and* get paid a boatload of money to do it?

      You sound like you actually read the report. Of the $2.74 million spent, close to $1.5 million was on contractors.

    • WTF?

    • by jittles (1613415)

      You mean I get to release my pent-up anger by destroying physical systems *and* get paid a boatload of money to do it? Where do I sign up?

      I used to work for the CA Attorney General's office and I got to destroy equipment all the time. They'd give me a big sledge hammer, I'd take it down to the loading dock, and beat my frustrations out on it. Perfectly good systems that, due to information security requirements, were not allowed to be recycled for anything. I objected and said we should only destroy the drives, but that was CA policy at the time.

  • by Anonymous Coward on Tuesday July 09, 2013 @10:10AM (#44225005)

    ... and yet I'm still furloughed on Friday...

  • Outdated Equipment (Score:4, Insightful)

    by Anonymous Coward on Tuesday July 09, 2013 @10:11AM (#44225015)

    It sounds like they were using this as an excuse to buy new equipment, so they destroyed extra equipment hoping that someone would allow them to chalk up the expense to the virus and thus give them shiny new stuff.

    • Or it was the IT equivalent of a German wedding.
    • by GodInHell (258915)

      It sounds like they were using this as an excuse to buy new equipment, so they destroyed extra equipment hoping that someone would allow them to chalk up the expense to the virus and thus give them shiny new stuff.

      That was my first thought as well. Particularly given the picture associated with the article is an old 13 or 14" NEC tube monitor.

    • by omnichad (1198475)

      Must have been really shiny - it cost them over $1,000,000 to replace $107,000 in destroyed equipment.

    • by pepty (1976012)

      It sounds like they were using this as an excuse to buy new equipment, so they destroyed extra equipment hoping that someone would allow them to chalk up the expense to the virus and thus give them shiny new stuff.

      Or one of the higher ups really wanted to destroy some of his files.

      http://www.theregister.co.uk/2007/12/01/official_purges_agency_computers/ [theregister.co.uk]

  • by jeffb (2.718) (1189693) on Tuesday July 09, 2013 @10:13AM (#44225045)

    Sounds like a good start.

  • by Serif (87265) on Tuesday July 09, 2013 @10:13AM (#44225055)

    You know, to be sure?

  • by Errol backfiring (1280012) on Tuesday July 09, 2013 @10:14AM (#44225059) Journal
    Will that infect my lawnmower? I'd better destroy it then before it gets dangerous...
    • by tgd (2822)

      Will that infect my lawnmower? I'd better destroy it then before it gets dangerous...

      You should get a shovel and double check ... your lawn may be full of worms.

    • Will that infect my lawnmower?

      No, but it could infect your lawnmower man. No great loss anyway, though.

  • by fuzzyfuzzyfungus (1223518) on Tuesday July 09, 2013 @10:15AM (#44225077) Journal

    Best Practices:

    1. Take off and nuke the site from orbit, it's the only way to be sure.

    • by omnichad (1198475)

      The infection could have come from the outside - they really need to destroy all the computers on the Internet.

      • by JBMcB (73720)

        And destroy all computer manufacturing facilities. And burn all books about computer science, so nobody accidentally builds another computer.

  • by rubycodez (864176) on Tuesday July 09, 2013 @10:15AM (#44225081)

    like how we developed Iraq, destroy good infrastructure so contractors with gov officials in their pockets make a pile of money

    or how government has developed inner cities over the past few decades, making fodder for the huge prison system business and food stamp system etc.

  • Just another example of why totally and blindly trusting big government with your tax dollars is not well advised. What do they care? They treat that income as totally disposable. Tax money is like Doritos, tax payers like Frito-Lay corp: "They'll make more" (obscure reference to an old advertising campaign for Doritos)
  • With users like this, who needs Malware?
  • by nimbius (983462) on Tuesday July 09, 2013 @10:22AM (#44225169) Homepage
    EDA: did you guys just smash a bunch of computers with a hammer because of viruses?
    DHS: Yes, but there havent been any terrorist attacks since we smashed everything with hammers. clearly the operation was a massive success.
    EDA: I dont even.....
    DHS: yep. Freedom isnt free.
    • Couldn't even be bothered to comprehend the first sentence of the summary, eh?

      Contrary to what we typically expect, it wasn't the DHS engaging in idiocy this time around. I know it's fun to bash the DHS, but at least do it for valid reasons.

  • by gstoddart (321705) on Tuesday July 09, 2013 @10:34AM (#44225337) Homepage

    they proceeded to physically destroy $170,500 worth of equipment, including uninfected systems, printers, cameras, keyboards and mice.

    OK, be honest now, who among us hasn't wanted to do this?

    Admittedly, destroying mice and keyboards is a little excessive, but I bet there's not a single person here who isn't dreaming of needlessly destroying a large quantity of computer gear in a very dramatic manner.

    • Amen. I recently acquired an old Sund V1280 Fire server. [slashdot.org] The beast is 130+ kgs heavy, and I sometimes wake up in the middle of the night, screaming and covered in sweat: one more dream of throwing the thing out of the window à la "One flew over the Cuckoo's nest".
      • by gstoddart (321705)

        The beast is 130+ kgs heavy

        Holy crap. The biggest thing we ever had to take delivery of was an HP-9000 server, but in a case with a built in UPS and a giant backplane for the disks.

        It was the size of a fridge, rolled on wheels, and needed to be wired in special because it was 220V and took a lot of juice.

        My guess is there was almost 100kg of batteries alone, but it was mostly a rolling rack with a computer inside.

  • by JeanCroix (99825) on Tuesday July 09, 2013 @10:36AM (#44225373) Journal
    It's was the dreaded "PC LOAD LETTER" virus. Smashing is the only recourse.
  • I don't want to work in that office if they learn most system vulnerabilities are due to humans!
  • I'm sure nothing fishy was going on in this government center. I imagine they didn't want 3rd parties looking at their computers too closely. #tinfoilhaton
  • They were due for upgrades anyway

  • If you're not confident you can disinfect your computers, then selling them on eBay is a lot more cost-effective ;-)

  • the Movie Zoolander? The two dumb male models smashed an iMac to try and get the files out of it.
  • Just let the techies run the show.

  • There are industries and use-cases when "smash first, don't bother asking questions later" this is the appropriate response.

    However, such times are rare and they should be spelled out ahead of time and they should only include destroying equipment which either 1) is at least theoretically possible to infect in a way that cannot be cleaned, ever (e.g. an infected BIOS), or 2) is deemed too expensive to clean and the data-storage media cannot be sterilized in a cost-effective manner or at all (e.g. a very che

  • It sounds like some contractors made bank in this arrangement.

  • That's hardcore paranoia. Are they an three letter agency front, or have they been pulling some hijinks?

An Ada exception is when a routine gets in trouble and says 'Beam me up, Scotty'.

Working...