Slashdot Log In
Anti-Terrorist Data Mining Doesn't Work Very Well
Journal written by Presto Vivace (882157) and posted by
kdawson
on Tue Oct 07, 2008 03:05 PM
from the could-have-told-you-and-did dept.
from the could-have-told-you-and-did dept.
Presto Vivace and others sent us this CNet report on a just-released NRC report coming to the conclusion, which will surprise no one here, that data mining doesn't work very well. It's all those darn false positives. The submitter adds, "Any chance we could go back to probable cause?" "A report scheduled to be released on Tuesday by the National Research Council, which has been years in the making, concludes that automated identification of terrorists through data mining or any other mechanism 'is neither feasible as an objective nor desirable as a goal of technology development efforts.' Inevitable false positives will result in 'ordinary, law-abiding citizens and businesses' being incorrectly flagged as suspects. The whopping 352-page report, called 'Protecting Individual Privacy in the Struggle Against Terrorists,' amounts to [be] at least a partial repudiation of the Defense Department's controversial data-mining program called Total Information Awareness, which was limited by Congress in 2003."
Related Stories
[+]
Your Rights Online: NSA Is Building a New Datacenter In San Antonio 119 comments
An anonymous reader writes in with an article from a Texas paper on the NSA's new facility in San Antonio. "America's top spy agency has taken over the former Sony microchip plant and is transforming it into a new data-mining headquarters... where billions of electronic communications will be sifted in the agency's mission to identify terrorist threats. ... [Author James] Bamford writes about how NSA and Microsoft had both been eyeing San Antonio for years because it has the cheapest electricity in Texas, and the state has its own power grid, making it less vulnerable to power outages on the national grid. He notes that it seemed the NSA wanted assurance Microsoft would be here, too, before making a final commitment, due to the advantages of 'having their miners virtually next door to the mother lode of data centers.' The new NSA facility is just a few miles from Microsoft's data center of the same size. Bamford says that under current law, NSA could gain access to Microsoft's stored data without even a warrant, but merely a fiber-optic cable." The article mentions the NRC report concluding that data mining is ineffective as a tactic against terrorism, which we discussed a couple of months back.
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Bets....? (Score:5, Insightful)
Re:Bets....? (Score:4, Interesting)
I bet this will not change what they are doing or how they are doing it one bit.
They'll be sure to change the amount of money spent on the program. I don't need to clarify whether it'll be more or less, its too obvious.
Whenever something doesn't work in government it seems to get more money and more power.
That leads me to think that maybe the primary function of government is to pretend to fail.
Parent
I'd run on that platform. (Score:5, Insightful)
Vote for me.
I'd take their "no fly" list and identify every single person on it who was a legitimate threat and either have them under 24 hour surveillance or arrested.
The mere concept of a list of names of people who are too "dangerous" to let fly ... but not dangerous enough to track ... that just fucking stupid.
Think about how many people could be killed in the airport terminal itself WITHOUT getting on a plane ... say during the Thanksgiving or Christmas rushes there.
What idiot would let the people on that list (if they were really a threat) into a terminal? Wouldn't you expect them to STOP them BEFORE they get into a position to do that kind of damage?
Parent
Re:I'd run on that platform. (Score:5, Insightful)
Parent
Re:I'd run on that platform. (Score:5, Informative)
The no fly list doesn't identify people, just names, and it's very exact, so changing charles to chuck will defeat it.
No, actually it won't. The newspapers are full of stories of people who were detained or forbidden from flying because their name was similar to a name on the list, or a nickname of a name on the list, or a possible alternative spelling of a name on the list, or names that had once been used as an alias of names on the list.
for example, the name "T. Kennedy" was on the list. Senator Edward Kennedy (whose name does not begin with "T", but who is nicknamed "Teddy") was stopped:
from Wikipedia [wikipedia.org]
Parent
Re:I'd run on that platform. (Score:4, Insightful)
Right, but flying under your middle name does work. As does claiming that you lost your ID (but if you refuse to show it on principle, you can't fly). As does using one boarding pass with matching ID at security, and a different boarding pass with matching ID at the gate.
The realy sad thing is, the people who the government feels are a real threat based on strong intelligence are *not* on the no-fly list! The government doesn't want to reveal to the real suspects that their being watched.
Parent
Re:I'd run on that platform. (Score:5, Informative)
It doesn't matter, because the only place where you have to get your ID checked is at the TSA checkpoint, and they don't check it against any databases.
So, the easy recipe for bypassing the no-fly list is:
I flew as recently as last month and was not subjected to anything which would defeat this scheme. It fails if you need to check luggage, but I doubt a terrorist is going to be doing that. The no-fly list is such an obvious joke.
Parent
In other news, (Score:4, Insightful)
Re:In other news, (Score:5, Insightful)
Parent
Re:In other news, (Score:4, Interesting)
Parent
Well, that's a shocker! (Score:5, Funny)
In other news, water is wet, the Pope is Catholic, and Ursines excrete solid wastes in silviculture.
Seems (Score:4, Insightful)
What we really need are spies. Not so much in the US, here good old fashioned detective work (with Warrants) should work.
But over seas a standing army isn't going to do anything to quell terrorism. Tanks and plans will only inspire more terrorism. What we need are good old fashioned black ops. Undercover agents penetrating the terrorist groups and talking to the bad guys. Much less collateral damage as well.
We'd get a lot further with a couple guys with silenced pistols rather than a whole army.
Re:Seems (Score:5, Interesting)
I seem to recall that much of this was gutted by Congress in the 1990s when they didn't want intelligence operatives paying off criminals for information, on the risk that the money might be tied back to the United States. This severely nerfed the ability of the CIA (among others) to gather HUMINT, as paid informants were a significant source of the information required to infiltrate the groups in the first place. I don't recall if this was ever overturned, though.
Parent
Re:Seems (Score:4, Interesting)
"...gutted by Congress in the 1990s when they didn't want intelligence operatives paying off criminals for information..."
They're still doing it here in the US. The FBI paid a shady informant 230,000 bucks to rat out harmless, loud-mouthed nobodies as part of this [militantislammonitor.org] case:
The government had no direct evidence. The confession was vague and even contradictory. And the statements about attacking American targets came only after heavy prompting from FBI interrogators.
America's FBI: "Incompetance and Pusillanimity through Proxy".
Parent
Re:There's a bigger problem with that. (Score:5, Insightful)
Reminds me of a bit from Discworld.
To summarize, Ankh-Morpork was over run by rats. The obvious solution was to put a bounty on rats, payable per tail. Soon, the rat infestation was under control but the number of tails being brought in kept increasing.
The Patrician's solution: tax the rat farms.
Parent
Re:There's a bigger problem with that. (Score:5, Insightful)
The GP isn't calling for vigilante groups turning in terrorists. He's calling for old-fashioned cloak-and-dagger HUMINT. It works far, far better than the technological circus we are operating now. Humans will always outsmart machines made by humans. The only real accomplishment of mass government data mining is the oppression of the general public who aren't interesting in outwitting the government. They're just trying to live their lives.
In the old days (Revolution, World Wars, Cold War), when we were aware of our enemies, spies, analysts and cryptographers defeated the enemies with courage, brainpower and skill. Now we've replaced them almost entirely with people in offices. This isn't going to change until we have another wakeup call, and the next one will probably come from Russia. The red bear is back, and we aren't prepared to deal with it (or China). Much of Russia's new technology is ahead of the US, particularly in aerospace submarine areas. We do not have a real missile shield, we do not have space-based weapons, we do not have supercavitating torpedoes (or anything to stop them). About the only encouraging developments we do have are in robotics and lasers.
China isn't very technological (except for those nasty anti-sat weapons), but they have an enormous mountain of people they don't mind sacrificing for whatever they dream up. Their standing army is over 2 million. They're also currently building and testing over one ballistic missile a week.
2005 article [bbc.co.uk] 2007 Article [aviationweek.com] Oct 6, 2008 [heritage.org]
Terrorist data mining won't help much of anything when an EMP hits and the computers are fried.
Parent
Re:Seems (Score:5, Interesting)
Parent
Re:Seems (Score:5, Insightful)
Parent
Profiles also work in reverse (Score:5, Interesting)
As any Cold War spy can tell you, if you "fit the profile" of a normal law-abiding person with just enough "off-perfect" things in your life so you don't seem "too perfect," it's much easier to blend in.
Re:Profiles also work in reverse (Score:5, Funny)
Parent
They can't collect or process (Score:4, Interesting)
enough data in any kind of real time to make this work.
Years ago, we were playing with a design of a system to track all the phone calls made on the AT&T network over a 3 month period. (not record the calls, just track the billing info). The machine that management wanted to try and do it on could not hold enough data just to store the data, let alone process it. And that was the largest theoretical model of hte machine there was (about 4 times the size fo the largest one in use at the time). They really needed one about 10 times as large as the largest theoretical one, just to store the data!
Multiple that by the rest of the items one buys during the day, and we can not track all the daa that is out there.
Why did they even waste the money to do the testing and the reports?
Didn't we already know? (Score:5, Insightful)
I thought we already knew this. If the algorithm comes back with even .1% false positives the system is totally worthless. There's 365 million people in the US, .1% means that the FBI/CIA/NSA would have 365,000 people to investigate. Now go and talk to someone in the AI field and see if even .1% false positive is possible.
I'm betting that if a system is going to catch any decent percentage of terrorists (greater than 50%) the false positive rate will be above 1%. Even if you only apply the system to a relatively small number of people (say people entering a leaving the country) you are going to have hundreds of thousands, if not millions, of people to investigate. Combine any kind of realistic false positive rate with the fact that about .00001% of the population deserves to be investigated and the system is worse than worthless; all it will do is distract from the people who should be investigated.
The actual report (Score:5, Informative)
Paradox of the False Positive (Score:5, Informative)
I realize this is likely starting to sound old, but Cory Doctorow's Little Brother should be required reading for people doing something like this. His writings about the "Paradox of the False Positive" are enumerated there, but also in other sources:
http://www.guardian.co.uk/technology/2008/may/20/rare.events [guardian.co.uk]
(emphasis mine)
And, as others have pointed out, this system is likely to have a false positive rate higher than 1%.
The purpose isn't to flag terr'ists (Score:5, Insightful)