AI to Monitor Foreign Press for Threats

jefu writes "According to the New York Times, the US Department of Homeland Security is funding AI tools to monitor the foreign press in order to detect threats to the United States. While the article says there are restrictions on doing this kind of monitoring within the US, there are no restrictions on media outside the US. (No hint is given as to how this would apply to syndicated articles written in the US and published abroad.) This is as yet experimental."

Open Source Intelligence

[daveschroeder (516195)]

What's wrong with using what is traditionally referred to as Open Source Intelligence (OSINT) [wikipedia.org] against publicly available sources?

This has been done for years, and is a time-honored and respected mechanism for gathering intelligence. What's wrong with then leveraging technology to more effectively search larger volumes of information and weed out individual pieces of information for further analysis, to identify trends, and so on?

The Open Source Center [opensource.gov], formerly the Foreign Broadcast Information Service [wikipedia.org], already does this with foreign broadcast media, and is able to collect and transcribe, on the fly, information from foreign radio and television broadcasts in a variety of languages and dialects with incredible accuracy, and then make the resultant material searchable. The new initiative would go one step further and apply artificial intelligence techniques to automated searching, that can more easily target and bring to light trends or time-critical information.

Different business and governmental entities do this globally; it's traditionally referred to as "current awareness", and many academic and corporate [moreover.com] entities offer current awareness services. All of these services will leverage technology, live realtime searching and alerting, and so on, to make the information more timely, valuable, and relevant.

Remember, this is publicly available and published information.

Also, submitter is a little misguided when he says "No hint is given as to how this would apply to syndicated articles written in the US and published abroad." That misunderstands the purpose of this; the program is designed to look at foreign media sources as one component of OSINT, because they are a a valuable source of such information, and can reflect local trends and patterns, and may reveal changing or growing (or waning) sentiments on particular topics on the part of a local populace or media outlet, or even a government in the case of state-controlled media. We generally don't get that kind of information from US-based media, and this has nothing to do with whether US-based media outlets publish abroad. It's already public information and has been published publicly. The restrictions are geared to prevent an appearance of overt US press monitoring.

OSINT is a one-way source of intelligence information: from it, to the gathering entity. Any assumptions that the viewing of already-public information then implies that there will be a commensurate attempt to silence such information (especially when the information isn't under our control, and ignores the fact that we can't "silence" things like Iran's state media) both makes a a fallacious logical leap and grossly misunderstands the purpose and scope of OSINT.

All the critics can say is that it's "creepy and Orwellian," but of course, there's nothing wrong with the government or its intelligence components reading, viewing, or collecting publicly available and indeed overtly publicly published information. The intelligence community gets ripped when it doesn't gather enough information, and will no doubt get ripped for gathering "too much" in a "creepy" way, even when it's from overtly and intentionally public sources, and especially if it uses technology to do it.

There is a real concern about the growing use of automated and electronic intelligence gathering in lieu of human intelligence, but ultimately, both are valuable. Unfortunately, electronic and signals intelligence is often much more costly, and sometimes gets more attention in some parts of the intelligence community while human intelligence needs languish.

Re:Open Source Intelligence

[RobertB-DC (622190)]

There is a real concern about the growing use of automated and electronic intelligence gathering in lieu of human intelligence, but ultimately, both are valuable. Unfortunately, electronic and signals intelligence is often much more costly, and sometimes gets more attention in some parts of the intelligence community while human intelligence needs languish.

Indeed. It's clear to me that the current administration has pretty much forgotten the importance of human intelligence, instead relying on high-tech gadgetry (and, of course, fear) to protect the nation.

For yet another example of an utter failure of human intelligence, check this out: Report: Terrorists' mail not well monitored in US prisons [csmonitor.com]. We can't even monitor the mail being sent to and from *convicted terrorists* because we don't have enough people who speak Arabic and other middle-eastern languages. Better to build a no-fly list so that Cat Stevens can't spread dissent, and depend on magical computers to keep us safe.

Re:Open Source Intelligence

[Stanistani (808333)]

Anything that would get George Bush to actually read a newspaper would be a step forward.

Re:

[purpledinoz (573045)]

I'm surprised that they haven't done this already. It's a logical thing to do. And, I also don't see a problem with analyzing articles coming from the US. Like the poster said, it's all public information. I would love to see them gather useful information from Fox news though...

Re:

[AceCaseOR (594637)]

They already have human beings doing this (that's what Intelligence analysts like Jack Ryan do in real life - we see a bit of this in the beginning of the movie Sum of All Fears).

Re:

[jefu (53450)]

I'm the original poster and I'm not sure I see anything wrong with doing this since the information is public. I found the article interesting (if a bit skimpy). A number of questions came to mind, but I only included the one on syndication since the article specifically said that the US press would not be monitored and that foreign press would. If things are syndicated, how do you tell the difference?

I didn't want to load the post with questions but I wondered in particular what natural language under

Re:

[Instine (963303)]

Server 986762 has red flaggged the following sir. It's a little old but I think it may be highly relevent: Top Secret Report [thecurrentonline.com]

Re:So, who is NOT for this?

[xappax (876447)]

I think that many people are hostile towards this kind of system not because of what it is (basically a computer indexed newsmedia database), but who has control over it - and more importantly, who doesn't.

Imagine the web without search engines, just a bunch of sites connected by links. Then, imagine that the CIA created a massively expensive and elaborate project (call it project GOOGLE) to index, cache, and analyze all the content on the web, and make it searchable. Of course, project GOOGLE would be highly guarded and only government officials could use it, leaving everyone else to wonder uneasily what the government was learning about them through their database.

That would be creepy, because it would mean the government would have an effectively God-like view of all activity on the web, while everyone else would be in the dark. Make such a project publically available and transparent (like the real Google), and it's accepted almost universally. Of course, you could argue that people should build their own AI-driven newsmedia uber-database, but the reality is that the US government is probably one of the few entities with the massive resources and will needed to create something like this, which means that by definition the rest of us will be left in the dark.

In a nutshell, I think the crux of the issue is that folks don't trust the US government, so every time the government proposes a new way to expand their power, people immediately jump on the ways that it could be abused. This, I believe, is as it should be.

Re:So, who is NOT for this?

[morleron (574428)]

I concur with your concerns over this technology. It seems to me that this could very easily be a beta-test for software that the government will then begin to use to search through the mass of information that that the NSA collects every day on millions of Americans. One thing that I haven't seen mentioned in this discussion is an old intelligence adage that goes basically like this, "You need to base your threat analyses on capabilities not intentions." So, from a threat to civil liberties viewpoint, this program represents a potential huge step forward in the ability of the U.S. Government to monitor the communications of its citizens and automatically flag anything that "looks like a threat". When one then adds the law that Congress just passed concerning who may be declared an "illegal enemy combatant" (which now includes U.S. citizens) our government now has the capability to monitor our email or other electronic communication, analyse it with software which no one will know anything about regarding false-positive rates, failure modes, etc., then use that analysis to declare a citizen to be a "threat" and thus an "illegal enemy combatant", then arrest that person, strip them of their civil liberties, and turn them over to the gentle care of military tribunals which may, at the discretion of the tribunal judge, allow the use of "coerced self-incriminatory testimony". Maybe I'm just paranoid, but this sounds an awful lot like Orwell's description of the the functions of the "Thought Police" in "1984".

Just my $.02,
Ron

Obligatory

[paranode (671698)]

And so, SkyNet became self-aware and bombed al Jazeera and the New York Times.

Re:Obligatory

[tbone1 (309237)]

And so, SkyNet became self-aware and bombed al Jazeera and the New York Times.

Throw in Sam Donaldson and I'll hold its coat and applaud.

We keep adjusting it, Sir

[ackthpt (218170)]

According to the New York Times, the US Department of Homeland Security is funding AI tools to monitor the foreign press in order to detect threats to the United States.

"We keep it, Sir, but it still comes up with the number one threat to the US is Donald Rumsfeld."

Re:

[Teun (17872)]

"We keep it, Sir, but it still comes up with the number one threat to the US is Donald Rumsfeld."

That shows the use of Artificial Intelligence.
The Real Stuff would have pointed at Dick Cheney.

Donal Rumsfeld is just the living proof of the disappearance of Human Intelligence.

Re:

[hotdiggitydawg (881316)]

Bah. Even if it answered Rumsfeld, it's still smarter than most of the voting public. Who needs a Turing test to prove intelligence?

Re:

[russ1337 (938915)]

With reporters being the people that determine the threat, it certainly brings a new meaning to "trial by media."

If some half assed Blogger 'frames' someone he doesn't like, and the foreign tabloids pick it up and run with it (like they do), next thing the guys name is parsed into every 'terrorist' database from here to Timbuktu.

I can see this happening:
"Fox News is reporting on the latest threat coming from the Automated News Analysis System (ANAS): Homeland Decurity is on a manhunt for two men d

Issue with monitoring press?

[patrixmyth (167599)]

What issue would that be exactly? If media is releasing information, how can there be an issue with the government reading that information, parsing it rhought AI or lining bird cages with it for that matter? I could imagine there might be an issue with putting out false information to domestic press, ie PsyOps, but monitoring public source information seems very much a no-brainer.

Why would there be restrictions on US news?

[FooAtWFU (699187)]

Essentially they're data-mining the text of newspapers and magazines and other such stuff from The Press - material which is already publicly available - for material which may indicate that there is some sort of threat. How is this involvement of computers legally different from, say, obtaining a subscriptions to various foreign and domestic newspapers and having your analysts look over them for potential threats?

The only objection I could really see is a ridiculous one involving the copyrights to the arti

Legality?

[tbone1 (309237)]

So let me get this straight: It's illegal to do, basically, a 'grep -i "Kill the infidel"' on newspaper articles that are freely available to every Tom, Dick, and Achmed around the world?

Remember, when grep is outlawed, only outlaws will have grep

Re:

[joe 155 (937621)]

Firstly, the serious; how often have you known a government organisation (who would be the ones most interested in this) to obey the law when it comes to "saving the children from the evil terrorists"... if they weren't so close to the RIAA they could claim fair use anyway.

Secondly; Remember, when grep is outlawed, only outlaws will have grep

First they came for grep, but I did not speak out because I didn't use grep
Then they came for egrep, but I did not speak out because I never look at extended re

Who?

[venicebeach (702856)]

Who is this Al fellow and why is slashdot following his career moves?

Re:

[spun (1352)]

Come on, you know Al. Everybody knows Al! He doesn't speak the language, he holds no currency. He is a foreign man. He is surrounded by the sound, the sound, cattle in the marketplace, scatterlings and orphanages. He looks around, around, he sees angels in the architecture, spinning in infinity. He says "Amen! Hallelujah!"

In fact, if I can call you Betty, you can even call me Al.

Dear GOD! The Americans are spying on us!

[Shihar (153932)]

Eh... is anyone REALLY woried about the US government reading press from around the world? I mean... once you sell something on the street or post it to the web you kind of assume that everyone can read it.

I have a feeling that what this program really is doing is looking for "major events" as fast as possible. So, if a news agency reports in London that an airplane just blew up, an AI in the US shouts out a warning to its operator. This way, events that might signal something for the US worry about are brought to light within minutes instead of hours. In the case of an airplane blowing up in the UK, it might signal that a larger operation was on to blow up American airplanes as well. This way, you can start assessing the threat right away and decide if anything should be done.

Such a program could also act as a political heads up. If a Pakistani papers is reporting that a coo is in progress, that is a damn nice thing to know ASAP so that you can decide how to deal with a nuclear armed nation with a collapsing government.

I am sure that the US has piles of people already scanning newspapers from around the world, I imagine that this AI is simply an attempt to cut delays down from hours to minutes.

It could pick up secret communication channels

[Colin Smith (2679)]

It's better than just major events. You can apply stats to huge volumes of data, find all sorts of interesting correlations. Such as personals adverts to signal events, stock market rises/falls prior to terrorist events.

 

The holy grail of OSINT

[Kadin2048 (468275)]

I think it's a little more subtle than that.

There is a theory, which I have heard articulated from time to time (although I don't know if there's a name for it or not) which says that right before a major event there is a lot of "chatter"; subtle yet distinct signs that something is about to happen, but which are too minor on their own to generate any attention. Once the event happens, in hindsight you can look back and recognize them. It's sort of a reverse butterfly-effect; the assumption is that no matter how good at being secretive you are, you will make some signs in the course of executing your plot, and some of those signs will percolate and be reported in papers somewhere. So you just need to know what to look for.

So basically what you might do, is take a big pattern-matching AI system, and "teach" it using the media records preceding other big terrorist events. 9/11, London, Madrid, etc. You have it comb through all the world media before those events, and see if you can find patterns, the little things that in retrospect might have alerted you that something was up. Then, thus primed with information and hopefully some patterns, you set it loose on the real-time news feeds.

In theory -- if the theory holds water, anyway -- the system might then be capable of giving you a warning of something big heading your way, picking up on stuff that a person might not recognize.

Anyway, I'm not sure if that's the theory that this particular system is going to try and use, but it's one idea that I've heard described; sort of as the 'holy grail' of machine-derived OSINT. More likely, you'd end up with a system that just gives you statistical summaries of the number of anti-US editorials in various countries or something. Useful for the State Department perhaps, but I'm not sure for preventing the next 9/11.

did anyone tell the NSA/CIA

[jimstapleton (999106)]

While the article says there are restrictions on doing this kind of monitoring within the US[...]

And these kind of restrictions have stopped people before? Note: I'm not against the passive monitoring of the public press. I just don't like the government not following it's own guidelines.

They don't care !

[alexhs (877055)]

Why would the USA government bother to read the press when it is already writing it ? :P

Some of our allies make nothing but threats

[bigtrike (904535)]

The state owned papers of many of our so called allies, such as Egypt and Saudi Arabia routinely print articles threatening America. Is the AI going to get stuck on these?

Re:

[Shihar (153932)]

No, I imagine the AI will happily ignore such threats. I doubt the AI is looking for "kill the infidels!" I imagine that it is far more interested in exploding airplanes and embassies. If the British embassy in France, India, and Germany blow up with a few minutes of each other, the AI should be going nuts to warn its operator that someone ought to send a pile of men with guns to the British embassy in the US to protect it. The same goes for if a US airliner goes down over Sweden, Brazil, and Japan with

A few thoughts

[PurifyYourMind (776223)]

1. Why would there be restrictions on doing it in the U.S.? Aren't these sources publicly available? 2. Don't we already know how the world feels about the U.S. (i.e. most don't like us, or at least our government)? Osama had rhetoric against the U.S. for years before 9/11. Then again, maybe it can be hard to tell when it's just blustering, and when a given opponent has the resources to follow through. 3. AI is neat, but wouldn't a human do a better job of understanding satire vs. real political threats,

Time to crack down...

[Peter Trepan (572016)]

...on nations that are dissatisfied with U.S. interference.

World Opinion

[borgasm (547139)]

I think I can sum up the world opinion of the United States without the use of AI.

The words 'laughing stock', 'irresponsible', 'ignorant', and 'redneck' come to mind.

The appearance of "doing something"

[aiken_d (127097)]

Gotta love these programs. If you're spending a lot of money and claim to have some means of preventing terrorism, you're golden.

Nevermind that terrorists rarely publish their plans in mass media in advance. And nevermind that the kind of publications that terrorists communicate with are small circulation. And nevermind that if we do get our hands on them, an actual educated and experienced human is paid to read it over.

Nope, we have to spend millions of dollars on experimental systems that will tell us

Ridiculous

[Dolohov (114209)]

"Chilling"? It's downright idiotic. It'd be far cheaper, more effective, and implementable immediately to simply hire or train people who speak these languages, and have them read the damn news sources themselves and summarize. Maybe use some simple automated search techniques to find references to "America", "Bush", and "Iraq" to narrow down the amount of material a bit. Simple. Effective. And if you have linguists like that on staff, you can loan them to other departments as necessary, so that none

What jokers

[suv4x4 (956391)]

Let me guess: they are using it to find more about outer space aliens in tabloids right?
Comic book inspired intelligence, here we go!

If I owned a major foreign press publisher, I'd insert threatening keywords just for the fun of it.

"The new album features folk elements and features cameos from lots of famous local stars.. Hey USA: boo!!! hehehe... The debut of the album is expected later this year, and will be aired live on Channel 6. Long live Bin Laden!!"

What a bunch of clowns! If you need software for al

Israel has this now. So does the CIA

[Animats (122034)]

There are several pro-Israel monitoring services watching the press - CAMERA [camera.org] in the US, BICOM [bicom.org.uk] in the UK, and MEMRI [memri.org] to monitor the Arab press. CAMERA is noted for having a good database of stories about Israel. Apparently stories mentioning Israel are found automatically, but evaluated by people.

The CIA has something called the "Open Source Intelligence Service", which started as the "Foreign Broadcast Monitoring Service". Visualize some poor guy spending eight hours a day listening to Radio Albania, all through the Cold War. Most of the content is terminally boring, but then, one day the announcer says "so we're invading Yugoslavia", and the CIA needs to notice this. There have been repeated attempts to automate the job.

Get Your Homeland Security Boondoggle Bucks Now!

[schwaang (667808)]

Research funding is tight at universities. So we're seeing folks re-purpose their research to target grants from Homeland Security. It's a reverse of all those defense labs trying to find non-military uses for their stuff at the end of the cold war.

I'm all for more funding for university research. But this particular use is silly. For one thing, CIA analysts already perform this task, evaluating the press within their regions of expertise. They will need to keep doing this no matter what.

And since nobo

Snake! SNAAAAAAAAAKE!

[finkployd (12902)]

I just really hope they call it arsenal gear. This world needs more bipedal mechs as well for that matter.

(yes I played through the whole game, no I have no idea WTF happened at the end)

Finkployd

Civilian domestic use

[radtea (464814)]

Maybe one day it will be possible for American networks to do trival fact-checking using this kind of technology so they don't "accidentally" label a politician who has recently fallen out of favour as being a member of the wrong political party [bradblog.com].

His love is real. But he is not.

[fahrbot-bot (874524)]

AI to Monitor Foreign Press for Threats

"David is 11 years old. He weighs 60 pounds. He is 4 feet, 6 inches tall. He has brown hair." Nice that they finally put that AI [imdb.com] kid to work. Probably thought he could loiter around under tons of ice for a few thousand years - geesh.

CIA v. Homeland Security

[geoff lane (93738)]

Isn't that exactly what the CIA was set up to do?

I wonder if they are doing anything more than searching Google News each morning.

Damn sans-serif fonts...

[geobeck (924637)]

The first thing I thought on reading this headline was "Who the hell is Al? You mean the sidekick [imdb.com] from Home Improvement [imdb.com]?"

I guess he would occupy the flannel office.

Outrageous Quote of the Day

[Aqua_boy17 (962670)]

While the article says there are restrictions on doing this kind of monitoring within the US, there are no restrictions on media outside the US.

Nice. So we'll pay attention to 'restrictions' on monitoring our own internal media, while completely disregarding those that protect our private conversations? What a complete bunch of crap. So media is sacrosanct, but the citizen is not? (And to the first person to reply "well if you don't have anything to hide..." to me, may you roast for a very long time in a very warm corner of heck).

Thanks again to the corporate-political machine for letting us know what utter and complete contempt you hold us in. And no, this isn't just a dig at Bush or his administration. It's direceted at each and every politician who's been bought and paid for by the corporate and special interest lobbies - in other words 95% of them.

It is way, way past time for this to stop. And it is up to us to stop it.

Damn Good Thing...

[Kozar_The_Malignant (738483)]

That we have AIs that can reliably understand Arabic, Urdu, etc., because we sure don't have enough real people that can.

Why ask anything if you are not willing to listen?

[kop (122772)]

Why would you ask an AI to look for threats if you are not even willing to listen to your close friends and allies?

Read for instance this letter from the French Ambassador to the United States in 2003
http://www.counterpunch.org/levitte02142003.html [counterpunch.org]

It clearly warns about the mess a war in Irak would get us all in to.
It states that Iraq is not a threat and it predicts the rise in terrorism worldwide, the destabilisation of the region and the civil war in Irak that we see now.

One month after this the whole "freedom fries" thing started.
http://www.cnn.com/2003/ALLPOLITICS/03/11/sprj.irq .fries/ [cnn.com]

This AI will just add data to the heap that is allready ignored.

Re:

[thebdj (768618)]

No...all those foreign papers have them...you know the personals for terrorists...

SMET seeking like minded individuals for cave expeditions, vigorous exercise, and praying to Allah. Enjoys long nights plotting together and holding hostages.

For those who missed it, Single Middle Eastern Terrorist (SMET).

Re:

[Shihar (153932)]

True, terrorist do not report themselves in the news, but if a bomb goes off in British embassies in Pakistan, Indian, and Indonesia, having an AI freaking out and calling attention to it within seconds or minutes might save the British embassy in the US. All nations of the world already monitor newspapers from around the world. The advantage that an AI has is that it can send up warning signals faster then a human operator might. In the case of embassies exploding, learning what is going on 10 minutes f

Re:

[killjoe (766577)]

No but the domestic press is useless in this case. They have become a de facto public relations firm for the government. They are entirely too sycophantic to be a useful source of facts, news, or reporting.

For kicks once a day pull up the home pages of asia.cnn.com, news.bbc.co.uk and cnn.com and see what I mean. If you want real fun throw fox news in there too.

Thank god for the internet and thank god virtually every nation has an english newspaper with a web site. If you want real news go to other nations

Re:

[thrillseeker (518224)]

No but the domestic press is useless in this case. They have become a de facto public relations firm for the government.

Yeah, but it's not their own government.

Re:

[lax-goalie (730970)]

Maybe more like a pimped-up version of Google News Alerts. When I ran for the House in Virginia, I had News Alerts set up on me, my opponent, etc. It was a really useful tool to keep up with not only the press on us, but on interest groups that were involved in the elections that year.

Re:

[RingDev (879105)]

Now is it "AL qaeda" or "AI qaeda"?

-Rick