Are We Ready For a True Data Disaster?

snydeq writes "Fatal Exception's Neil McAllister questions how long we can go before a truly catastrophic data disaster strikes. 'The lure of potential profits in the information economy, combined with the apparent ease with which data can be gathered and a lack of regulation, creates a climate of recklessness in which a "data spill" of the scale of the Deepwater Horizon incident seems not just likely, but inevitable.' Witness Google mistakenly emailing potentially sensitive business data to customers of its Local Business Center service, or the 1.5 million Facebook accounts and passwords recently offered up on an underground hacking forum. 'These incidents seem relatively minor, but as companies gather ever more individually identifiable data and cross-reference these databases in new and more innovative ways, the potential for a major catastrophe grows.'"

The State of Data is Not Good

[pankajmay]

I will only partly agree with Mr. McAllister's assertion about the potential for catastrophic loss via an inadvertent leak. However, I do feel that much of this stuff go packaged as half-truths and half-fear mongering.
There are a few facets to the issue - let me try to dissect them:

• Immense amount of data being collected: Very true. Everyday people are generating information that when cleverly pieced together can unravel every minute of their life. However, the caveat is that there is such a huge amount of information. Today we are at a position where the inflow of data far exceeds our capacity to process it. Most regular people aren't interesting enough for someone to worth wading through the muck to piece together coherency. Yet, there are people who will be subjected to such attacks and hopefully they are already taking precautions. For the rest of us mere mortals, no matter how significant we think our precious little existence is, the fact remains that largely we are all mostly just statistical data points -- white noise.
  
  Just like in statistics -- corporations are not looking for a particular person, but they are trying to aggregate it all and derive a trend or more accurately a statistical model. And just like in statistics -- the outliers will stand out.
• The Valuable data is the Aggregate, not the actual data point: This is where the line becomes gray. Is it alright to zealously collect every dimension of data available to derive a meaningful aggregate? We are all understandably uncomfortable having our menial contributions, measured and carefully cataloged. However, if there is a way where important data about you is handled for only a brief while -- converted into something that retains the meaning of that data point but loses the association with you, I have a feeling then that would be classified as legal. Of course, active research is being done today in this area.
• Data is unduly important today because we have (stupidly) delegated our identity amongst few numbers: I heard on NPR yesterday about how people's health insurance is being stolen. And do you know why such a fraud occurs? Because, no one conclusively establishes the patient's identity. They just ask for the card and done. They don't ask the driver's license nor put a simple photograph of the patient on the file to check. We have done the same thing with other such numbers -- Social Security, Date of Birth have all been used conclusively to establish a person's identity. True - it may have been a simplifying solution when Computers were not advanced. But the real travesty is not the availability of our data out there - which in this modern age is inevitable -- but that we are not switching to more robust methods of establishing people's identity. One of the ways could be to check finger prints (finger print readers are mighty cheap) or other such biometric data that cannot be easily faked.

Until criminals discover databases.

[khasim]

Everyday people are generating information that when cleverly pieced together can unravel every minute of their life. However, the caveat is that there is such a huge amount of information.

-and-

I heard on NPR yesterday about how people's health insurance is being stolen. And do you know why such a fraud occurs? Because, no one conclusively establishes the patient's identity.

Now imagine a criminal organization that is interested in collecting that information and sorting it into personal profiles. Start with a database of social security numbers.

Now add enough detail to be able to get loans or credentials in the names of those people (with the aforementioned social security numbers).

It wouldn't take much processing power or storage.

The problem seems to me to be...

[Anonymous Coward]

.. that we don't know what data we do have, what data we should have, why we have it, what we want to do with it. Data itself is the problem we are collecting collating, storing this crap, if I collect and store enough tires they will eventually catch fire and burn things and poison people, I done know how you really go about estimating the cost of what has already happened, which seems to me to be disastrous, but things like 10 million CC number released, or 10's 100's of millions of Social security/bank account numbers (or sub. for which country you want) released are causing giant disruptions to people individual lives and costing who knows how much in fixing, just as a small example, I had a friend that had his credit score wreaked because of an unsolicited CC that was never activated but charged a something like $2 fee (which it was not supposed to), which was of course never paid since he was completely unaware of it and more than a year, many e-mails and phone calls and lots of straight out frustration an misery it was finally fixed what is that 50? 100? hrs of peoples wasted time and that was a simple billing error never mind a full on identity theft or any number of thousands of other problems erroneous or stolen/posted in to open information can cause to individuals (primarily the ones who pay the cost of these problems) if you want to count it in purely dollar cost my bet is we have already had hundreds of disasters that equal or exceed DWH, certainly on a personal level the level of disaster about to be experienced by the residence of the coast has been exceeded many many times world wide, but it is more diffuse and so less noticeable. I can't list all of the potential problems and people who should be here don't need me to tell them.

Data + Human = TONFO It's the only way to be sure

Sorry wait I'm not a web 2.0 weeny "I say we take off and nuke it from orbit,... its the only way to be sure.."

Besides I didn't RTFA

Re:Not Obeying The Law prevents data disaster

[dyingtolive]

The independents are hard to find.

By design even. Distribution is the primary thing that keeps the cartel's thumb pressed down upon artists. Pandora helps a lot, but lately they seem to be fallible even. I can't seem to get them to stop play Coldplay for example. I finally thought I voted down every Coldplay song in the collection, and then they started springing LIVE versions on me. I kind of thing they're getting paid to push it at this point.