Anti-Terrorist Data Mining Doesn't Work Very Well 163
Presto Vivace and others sent us this CNet report on a just-released NRC report coming to the conclusion, which will surprise no one here, that data mining doesn't work very well. It's all those darn false positives. The submitter adds, "Any chance we could go back to probable cause?" "A report scheduled to be released on Tuesday by the National Research Council, which has been years in the making, concludes that automated identification of terrorists through data mining or any other mechanism 'is neither feasible as an objective nor desirable as a goal of technology development efforts.' Inevitable false positives will result in 'ordinary, law-abiding citizens and businesses' being incorrectly flagged as suspects. The whopping 352-page report, called 'Protecting Individual Privacy in the Struggle Against Terrorists,' amounts to [be] at least a partial repudiation of the Defense Department's controversial data-mining program called Total Information Awareness, which was limited by Congress in 2003."
Re:Bets....? (Score:3, Interesting)
Of course not. And neither major-party presidential hopeful is going to change it, either. We're still going to get stupid hassles from the TSA, we're still going to get the watch list filled with pointless entries based on the name of someone who might have been seen with someone who was linked to someone who claimed to have been involved in a shooting in North Ireland.
I would seriously consider voting for either one that came forward and promised to cut TSA's authority and streamline the process, getting back to only those people who are basically confirmed problems being on the list, no matter what their views might be on Iraq, Afghanistan, the economy, or offshore drilling.
Just give it a few years (Score:3, Interesting)
And several billion dollars.
And unrestrained access to all of the personal information about everyone that can be gotten by whatever means.
It'll probably still suck then, too.
Profiles also work in reverse (Score:5, Interesting)
As any Cold War spy can tell you, if you "fit the profile" of a normal law-abiding person with just enough "off-perfect" things in your life so you don't seem "too perfect," it's much easier to blend in.
Re:Seems (Score:5, Interesting)
I seem to recall that much of this was gutted by Congress in the 1990s when they didn't want intelligence operatives paying off criminals for information, on the risk that the money might be tied back to the United States. This severely nerfed the ability of the CIA (among others) to gather HUMINT, as paid informants were a significant source of the information required to infiltrate the groups in the first place. I don't recall if this was ever overturned, though.
They can't collect or process (Score:4, Interesting)
enough data in any kind of real time to make this work.
Years ago, we were playing with a design of a system to track all the phone calls made on the AT&T network over a 3 month period. (not record the calls, just track the billing info). The machine that management wanted to try and do it on could not hold enough data just to store the data, let alone process it. And that was the largest theoretical model of hte machine there was (about 4 times the size fo the largest one in use at the time). They really needed one about 10 times as large as the largest theoretical one, just to store the data!
Multiple that by the rest of the items one buys during the day, and we can not track all the daa that is out there.
Why did they even waste the money to do the testing and the reports?
Re:Bets....? (Score:4, Interesting)
I bet this will not change what they are doing or how they are doing it one bit.
They'll be sure to change the amount of money spent on the program. I don't need to clarify whether it'll be more or less, its too obvious.
Whenever something doesn't work in government it seems to get more money and more power.
That leads me to think that maybe the primary function of government is to pretend to fail.
Re:They can't collect or process (Score:3, Interesting)
Re:Seems (Score:4, Interesting)
"...gutted by Congress in the 1990s when they didn't want intelligence operatives paying off criminals for information..."
They're still doing it here in the US. The FBI paid a shady informant 230,000 bucks to rat out harmless, loud-mouthed nobodies as part of this [militantislammonitor.org] case:
The government had no direct evidence. The confession was vague and even contradictory. And the statements about attacking American targets came only after heavy prompting from FBI interrogators.
America's FBI: "Incompetance and Pusillanimity through Proxy".
Re:In other news, (Score:4, Interesting)
Re:False negatives are a worse problem (Score:3, Interesting)
The biggest problem is actually not the false positives - that would just mean extra wasted effort to screen the individuals, which "only" costs time and money.
The larger problem is that in order to do any real good you need an unbelievably low false negative rate. Let's take the 9/11 hijackers as an example: they were only about 0.00000667% of the population. Unless you could capture all but 2 or 3 them, you're still vulnerable to the plot unless you can get one of the ones you captured to spill the beans - at best you've just mitigated the plot. How realistic is it that ALL 20 (or 19 if you believe that Zacarias Moussaoui was not part of the conspiracy) of them could have been identified (let alone captured) using such a method, even given the expenditure of vast resources sifting through all of the false positives? Even if 4 or 5 of them manage to fall in the "negative" group or, alternatively, if they're able to slip through your second-level screening procedures, you still have a disaster on your hands.
It's not likely that you could get the accuracy high enough to stop very many plots by itself, I suspect.
Re:Seems (Score:5, Interesting)
Re:I'd run on that platform. (Score:3, Interesting)
What would happen if terrorists got nicknames after all major U.S. and U.K. political figures.
For better, not worse (Score:3, Interesting)
It turns out that terrorism in western countries is a very rare thing, outside of a few hot areas like Spain's Basque area. This is very good, by the way.
Mining for rare event is extremely difficult. Bayes' s rule indicates that if in a database there are 0.01% actually suspicious events and your mining algorithms are extremely effective at 99% accuracy, then you still have an approximately 100:1 false positive ratio, which makes the mining still useless.