Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Businesses Data Storage Encryption Medicine Privacy Security IT

TN BlueCross Encrypts All Data After 57 Disks Stolen 140

Posted by timothy from the best-practices-are-best-practice dept.
Lucas123 writes "After dozens of hard disk drives were stolen from a leased facility in Chattanooga, potentially exposing the personal data of more than 1 million customers, BlueCross decided to go the safe route: they spent $6 million to encrypt all stored data across their enterprise. The health insurer spent the past year encrypting nearly a petabyte of data on 1,000 Windows, AIX, SQL, VMware and Xen server hard drives; 6,000 workstations and removable media drives; as well as 136,000 tape backup volumes."
This discussion has been archived. No new comments can be posted.

TN BlueCross Encrypts All Data After 57 Disks Stolen More

TN BlueCross Encrypts All Data After 57 Disks Stolen

Comments Filter:

  • I am impressed (Score:3)

    by WindBourne ( 631190 ) on Friday July 29, 2011 @07:18AM (#36919824) Journal
    Most insurance companies these days, are far more concerned with getting bonuses to the executives.

    • Most insurance companies these days, are far more concerned with getting bonuses to the executives.

      You don't honestly think that the executives will end up with smaller bonuses as a result, do you? We all know that isn't how this game works.

      The company will cover these costs by raising premiums and/or reducing payments. It is very likely that the executives will see larger bonuses after this, as a self-congratulatory measure for "proactively correcting the situation".

      • Re: (Score:2)

        by RenHoek ( 101570 )

        That's part of the fun right?

        I mean, as a customer, first you get screwed over by having your medical records out in public. Then the company gets fined and leverages that fine on its customers, thusly getting screwed a second time. Finally, costs are incurred for getting up to standards, and guess who is paying for those costs?

        • There. All done encrypting every hard drive and backup tape.

          Um, does anybody remember the password we used? Surely somebody wrote it down?

        • So I'm getting screwed three times, without ever once getting flowers or dinner out of it!

    • Which is a win for all. Executives can't get bonuses if there isn't a decent amount of income to the company, there can't be a good amount of income to the company if there aren't high revenues. There can't be high revenues if there isn't a supply of something people want that can be produced for at least slightly less than they are willing to pay for it. They wont be willing to pay for it if it's cost is higher than it's value to the individual.

      Everyone wins.

      • Re: (Score:2)

        by geekoid ( 135745 )

        Executives can't get bonuses if there isn't a decent amount of income to the company,"

        false.

        • well, i guess if you're an executive for a money counterfeiting organization. Otherwise that bonus has to come from somewhere.

          You may be thinking of elected officials. They're the ones that get paid without having to show results.

          • You seem to be forgetting the financial companies who were so fucking broke we had to give them money or the world would end. You know, the same companies who took our money, turned around and gave billions in bonuses to their employees, presumably for doing such a wonderful job

            • And why we should have much lower taxes and smaller government and insure the government is never able to do a bailout like this ever again. Perhaps instead of taxes we could have bailout bonds issued so people could feel they were doing the right thing by buy the bank bailout bonds if they felt it was the right thing to do.

    • Re: (Score:2)

      by darrylo ( 97569 )
      Why? Some states require that companies notify people when their data is stolen, as well as sometimes requiring identity theft protection (e.g., credit reports or alerts) or somesuch. This can get pretty expensive, and so it's probably cheaper to just encrypt everything. They're not being altruistic -- they're saving money. It wouldn't surprise me if some executive got a bonus for saving the company money ...

  • This entire effort might be useless if they're not using good encryption. Is there one master passphrase to bypass all of the encryption? Also, they make no mention of how they plan to prevent physical theft of data again just that 'Well this time I put a password on my data, take that thieves!'

  • very lame (Score:1)

    by Anonymous Coward

    "We searched the country and were unable to find another company that has achieved this level of data encryption," Michael Lawley, vice president of technology shared services for BCBS, said in a statement.

    He certainly did not search very hard. Less than 1PB encrytpted, we do more than that every single day. And I doubt we are unique.

    • Re: (Score:1)

      by Chrisq ( 894406 )

      "We searched the country and were unable to find another company that has achieved this level of data encryption,"

      Could be because they also invested in steganography.

  • It is a pity that the data was stolen before adequate protection was put into place, but it seems to me TN BCBS took the right steps afterwards:

    1. They sent out alerts to those affected, both current and former members

    2. They now encrypt all their stored data

    Of course, this will not prevent all possible leaks, but at least it shows they are taking protection of their customers' data seriously, and have put in serious work to protect that data. I wish more organizations did that. Way to go, BCBS of Tennessee

  • "I know I already shit on the floor, but I'm wearing a diaper now so it's all good!"

    • "I know I already shit on the floor, but I'm wearing a diaper now so it's all good!"

      where is badanalogyguy?

      so you're saying that one mistake (data loss; floor shitting) will render every countermeasure (encryption; diapering) invalid? nah, I don't think so. The insurance company handled the data loss quite competent - they disclosed it early (afaik) and implemented a regime that will make future data losses much harder.

      • It wasn't a perfect analogy, but I don't think they should be congratulated for closing the gate after the horse already bolted. They're just doing what they should have been doing all along. Really, they shouldn't let anything even get stolen.

        • Re: (Score:2)

          by geekoid ( 135745 )

          " they shouldn't let anything even get stolen."

          way to blame the victim.

          • Aren't the real victims their customers? If you have millions of customers' data, you should have enough resources to give it physical protection.

      • no, it makes data losses just as easy as they were before. It prevents data theft as the records are now (theoretically) protected. Without proper off-site backups they are still screwed if someone steals their drives again.

      • "I know I already shit on the floor, but I'm wearing a diaper now so it's all good!"

        where is badanalogyguy?

        so you're saying that one mistake (data loss; floor shitting) will render every countermeasure (encryption; diapering) invalid? nah, I don't think so. The insurance company handled the data loss quite competent - they disclosed it early (afaik) and implemented a regime that will make future data losses much harder.

        Does the insurance company have insurance for their data?

        If the jelly does not cover the peanut butter on the PB&J pizza, then the PB gets hard and difficult to eat.

    • The counterpoint would be Sony:

      "Oh, there I go again! And again! Well, I didn't see that comi-And again! Wow, this is quite a string of bad luck!"

  • Well the new customers whose data hasn't already been stolen will be happy to hear it, I guess.

  • I'm by no means a security expert but isn't $6 million a bit excessive for the effort?

    TFA says "The company said it spent more than 5,000 man-hours on the encryption effort, which encompassed about 885TB of at-rest data." That equates to around $1200/hr. Perhaps I should become a security expert.

    • Re:$6 million? (Score:4, Interesting)

      by belthize ( 990217 ) on Friday July 29, 2011 @07:52AM (#36920046)

      I wouldn't take the $6M and 5000 man hours as directly coupled. The actual press release says:

      BlueCross invested more than $6 million and 5,000 man-hours in the data encryption effort, which included:

      - 885 Terabytes of mass data storage
      - 1,000 Windows, AIX, SQL, VMWare and Xen server hard drives
      - 6,000 workstation hard drives and removable media drives
      - 25,000 voice call recordings per day
      - 136,000 volumes of backup tape

      The 5000 man hours may only reflect actual labor and not reflect all the hours of planning/scheduling etc. What ever hourly rate for labor double it for overhead, the cost of a person is about twice their salary, at $100/hour that's $1M in labor. Another 500K in planning. I have no clue what software they used but I'm pretty certain it wasn't a single package. Each system may well have required a different package + licenses + contractor time from the vendor. For example they may have had to out source the voice call recordings to who ever provides their phone system. I kind of doubt they slap all the recordings onto a single box and mass encrypt.

      They're a very distributed organization so there's going to be a *lot* of duplication of effort, they may have had to do the phone bit at hundreds of sites.

      I don't know if it could have been done for $3M or if $6M actually represents a relatively reasonable price compared to a lot of the $XXX Mllion dollar utter failure projects. It strikes me as fairly reasonable considering the scope of the problem and usefulness of the result (assuming it's not a $6M whitewash).

    • Re: (Score:2)

      by tecker ( 793737 )
      Assuming 100% markup profit margin over baseline (common practice really) were looking at a baseline cost of $3 mil.

      Now we need to factor in an encryption scheme that works across Windows, AIX, etc with enterprise support backing it up say $1.2 million to licence for all servers and locations (seem low but hey) and we have $1.8 million to spend.
      Now we gotta pay people some prices to do that work so lets say $.5 million (500,000) so about $100 per man hour (bout right) and we have $1.3 to spend.
      Now pay t

    • Other people did a breakdown before me of the costs. Lucky thing: it's expensive to start but cheap to keep it, just remind people every 6 months that they should use the software. Oh, and check very often that you can restore your backups: there's nothing funny in working your whole weekend because an encrypted backup has locked itself in.

    • Re: (Score:2)

      by geekoid ( 135745 )

      Obviously, and no it isn't.

  • So, they're locking the barn door after the horse has bolted...

    dozens of hard disk drives were stolen from a leased facility in Chattanooga, potentially exposing the personal data of more than 1 million customers

    The data is gone... and now they're encrypting.

    • Re:"Safe route" (Score:4, Informative)

      by MysteriousPreacher ( 702266 ) on Friday July 29, 2011 @07:50AM (#36920032) Journal

      I don't think the barn door saying means what you think it does. It suggests pointless action taken after the event. The original data was stolen but encryption to hinder future theft of data seems sensible.

    • Even with the best commercially available encryption if someone steals the hardware storing the encrypted data they have all the time in the world to try and access it. The disks were in the possession of a 3rd party at the time of the theft so a security audit of their premises and security procedures might be in order to help raise awareness and prevent future incidents.

    • Re: (Score:2)

      by isorox ( 205688 )

      So, they're locking the barn door after the horse has bolted...

      dozens of hard disk drives were stolen from a leased facility in Chattanooga, potentially exposing the personal data of more than 1 million customers

      The data is gone... and now they're encrypting.

      They've locked the barn dor after 1 horse bolted. There's hundreds more left in the barn.

    • Your analogy, while not perfect has a valid point. However, remember that they now have a new horse in that barn. (all the customers that have since the data loss) What would you say about the farmer that lost his horse, got a new one, and still leaves the door open?

      Perhaps the lesson here should be to all the IT people (does anyone in IT still read slashdot?) take this type of preventive action BEFORE you have data stolen. (yes, i know it's really up to the C-something-O to fund and order such an operation

  • $6 million is pocket change to a company that has $5.2 billion in annual revenue. However, the true cost is really higher, as encrypting everything means that things like disk corruption are no longer repairable, lost passwords can't be reset without losing data, and the like. It'd be interesting to see just what the ongoing costs are.

    That said, I would like to compliment Tennessee BC/BS for doing the right thing, in spite of it costing money.

    --Paul

    • Re: (Score:3)

      by blueg3 ( 192743 )

      How is disk corruption less repairable when you encrypt?

      The lost-passwords problem is already well-solved for decent systems.

      • Re: (Score:2)

        by horza ( 87255 )

        I think he meant less recoverable rather than repairable. Which is true, you can't simply dump the disc and extract the fragments by hand if necessary if encrypted.

        Phillip.

        • Which is true, you can't simply dump the disc and extract the fragments by hand if necessary if encrypted.

          If you have a properly layered solution (e.g. LUKS), you can open the crypto volume, and then dump the unencrypted block device for manual recovery.

      • My personal experience with a couple of mainstream commercial enterprise solutions, is their data recovery tools leave a LOT to be desired and seem to only work for us about a third of the time. Features and management tools get the attention; auditing and recovery are after-thoughts in most products.
        In a few instances where we had to engage a data recovery service, they charge quite a bit more when they find out that they're dealing with an encrypted disk (i.e. when we're going after a specifc folder or a

    • Re: (Score:1)

      by maxume ( 22995 )

      If you use the password to encrypt the key, you can store a copy of the key somewhere else.

      So if the password is lost, to reset, you grab the key from the escrow and encrypt it with the new password.

    • Re: (Score:2)

      by Himring ( 646324 )
      $6 million is pocket change to a company that has $5.2 billion in annual revenue.

      Right, but any money spent on IT is a waste to the stuffed shirts, until something blows up, which, inevitably, gets them off the fence. Telling the COs in a meeting, "our worst possible downtime with the current allotted budget might be as bad as 3 days," makes them all look at each other with satisfaction and approval, seemingly, ok with being down 3 days in theory. Then, after 3 hours of downtime, they are talking about

    • I work for a company where data is subject to HIPAA (United States' Health Insurance Portability and Accountability Act - a law whose provisions also address the security and privacy of health data). Our data has been encrypted -- at rest and in transit -- for years. The loss of private health information, like what Blue Cross did, is a serious crime under HIPAA and subject to major fines (in this case, at least tens of millions of dollars, probably, given how large the breach was). The initial cost to encr

  • In the Netherlands we have a adage that seems fitting, "De put pas dempen als het kalf al verdronken is.". Which roughly translates to "Closing the well after the calf already drowned.".
  • They have the personal details (health records, bank info, addresses, etc.) of millions of people and they just now decided to encrypt the data? WTF?

    • Re: (Score:2)

      by Sloppy ( 14984 )

      It sounds reasonable on the surface, since people think of drive theft as very exceptional and something you can physically defend against. But then .. these people never had a drive fail and then RMAed it? Am I supposed to believe that when there's a mechanical failure and they're unable to erase the drive, they destroy it rather than mailing it back to a vendor or manufacturer?

  • jryy vg jbhyq unir orra svefg cbfg vs vg jrera'g sbe rapelcgvba bireurnq.

  • leased facility = cloud so this is what you get from going to the cloud the data can be in a place that can range from a nice data center to a small room in a office building. Also the people ruining the cloud can just have real low prices and then sell data to the highest bidder.

    • Re: (Score:1)

      by Anonymous Coward

      Leased facility != cloud. In a leased facility, you can find out the operational conditions and the level of physical security. You can make them part of the lease contract if you care enough. You can't do that in a cloud.

    • Re: (Score:2)

      by geekoid ( 135745 )

      And now, Samuel L. Jackson will read a line from his up coming movie: "English Lesson"
      Punctuation motherfucker, learn it.

  • ... even if it is far too late. And of course, the customers will pay for the cost of the failure, plus the cost of the fix. The company made a bad choice, and the consequences of that bad choice will be born by .. the customers. The executives will still get their usual multimillion dollar "performance" bonuses as if nothing was ever wrong.

  • If you encrypt it before it gets stolen.

  • If you've got the drive... you have unlimited attempts to crack it. Someone with a couple of video cards and a few days on their hands and their encryption is pointless.

  • These drives were likely part of various RAID volumes. Doesn't that mean they're pretty well useless outside their hosts? Is someone really going to go to the level of forensic data recovery to elevate from property theft to identity theft? That stuff isn't cheap, so the ROI is probably going to be really low.

  • Is it just me, or shouldn't this be standard fscking procedure for companies dealing with sensitive information such as medical and financial records?

    • Re: (Score:2)

      by qwijibo ( 101731 )

      Should be, but generally isn't. Security costs money, and most companies have been in a cost cutting mode for years. Security is one of the first things to go since it's invisible until you're compromised.

      • Generally I think most companies don't need it. Some only need the basics. You got my personal information, or credit cards? Just securely encrypt those sources. Sure some might slip out here and there, but you won't lose your whole database of 300,000 customers or whatever.

        I just mean if your a bank, financial institution of some description, or someone that handles my medical information, get on the encryption boat and set sail. Seriously. I mean it is one thing if someone gets my VISA number... its usual

  • Looked around the stories including their "infographic", not clear what they are using and how they've implemented it.

    Do servers have pre-boot enabled? How did they change they operational processes? Are these HW-encrypted drives? What is the failure rate on the process?

    Details like this are important. As it stands, they spent the cash and a lot of time, but no indication that they've implemented it properly. I wouldn't feel much safer.
    5,000 hours is nothing to be honest for even a mid-size company. T

  • They should get some credit for spending money encrypting their data but it's still another case of a company that only does the right thing AFTER shit hits the fan.

  • when one of their machines reboots, where does the key come from? such sites usually spend as much money as possible on the theory that mauve is better, which in this case probably means FC SANs. but at which level does the encryption happen? and doesn't disk encryption just mean that you need to take the enclosure or client box too?

  • is written on a post-it stuck to the monitor of the secretary for the CEO.

  • It only took them 57 horses getting stolen before they decided to lock the barn door.

    good job! way to keep on top of things.

  • So they are spending 1200 dollars a man hour? Total machines seem to be about 6000, so each machine is costing a grand to encrypt? Seems pretty expensive.

Slashdot Top Deals

A committee takes root and grows, it flowers, wilts and dies, scattering the seed from which other committees will bloom. -- Parkinson

Close