FBI Building App To Scrape Social Media 133
Trailrunner7 writes "The FBI is in the early stages of developing an application that would monitor sites such as Twitter and Facebook, as well as various news feeds, in order to find information on emerging threats and new events happening at the moment. The tool would give specialists the ability to pull the data into a dashboard that also would include classified information coming in at the same time. One of the key capabilities of the new application, for which the FBI has sent out a solicitation, would be to 'provide an automated search and scrape capability for social networking sites and open source news sites for breaking events, crisis and threats that meet the search parameters/keywords defined by FBI/SIOC.'"
So. It begins. (Score:5, Insightful)
You can already assume that your 'public' posts are being seen by people you wouldn't want to see it, but now you know that it is automatic. Depending on data sharing agreements these companies come up with with the FBI, they might even get access to private information.
Hopefully, the latter isn't an issue and they're just scraping public information, but even then, any hopes of not being carefully monitored are dashed. Assume that everything public (and most things private) will be read by people other than the intended recipients. Privacy? What privacy?
Re:So. It begins. (Score:5, Insightful)
So what?
Yes, I sound cavalier, but I see so many people on /. blithely affirming that people should just know that what they put on the internet stays there forever, and should just know that their SSID is being broadcast and it's a good thing that it can be tracked and stored, and should be fine with people capturing anything whatsoever that's done outside the house, or in the house with the curtains open...
So I can't see that anyone on Slashdot has anything to complain about here. Or is it different because it's not Google doing it?
Re: (Score:3, Interesting)
Google doesn't have the power to arrest me and send me to Gitmo without trial (NDAA). That's why I hate corporations but don't fear them. I only fear the government.
As for permanence, I can find my posts going all the way back to 1988. Who knew in 1988 that posts would be archived forever??? Anyway I don't use my real name anymore, and try to rotate my fake name every 1-2 years.
For the FBI: ;-)
terrorist
Ron Paul
militia
Constitution
gun
Liberty
airplane
MIAC REPORT
Yeah that sound catch their attention.
Re: (Score:1)
Re:So. It begins. (Score:4, Informative)
List of supposed trigger words as of @ 2005
http://www.rense.com/general66/scgh.htm [rense.com]
Re: (Score:2)
6-28-5
Is tthis June 28, 2005 or May 28, 2006? Or perhaps even June 5, 1928? ;-) I wonder where that list came from? For all I know it was an invention of some Tim McVey-type character. A work of fiction.
Re: (Score:2)
It most probably is a copy of the list that used to be on attrition.org , was unable to find attrition's original list, didn't look all that hard though.
Re: (Score:2)
List of supposed trigger words as of @ 2005
http://www.rense.com/general66/scgh.htm [rense.com]
Was it the wrong turn at Albuquerque that got Bugs Bunny on the list?
Re:So. It begins. (Score:4, Funny)
List of supposed trigger words as of @ 2005
http://www.rense.com/general66/scgh.htm [rense.com]
Well then, let me just target myself by creating a story from choice words on that list... bolded for your convenience.
--------
"MITM won't work, they have firewalls", the portly nerd stated flatly, "Only way in is SSH over SSL, but brute forcing the RSA key may take eons."
"Let me try," said Skully as she sat at the terminal displaying the remote supercomputer's login prompt.
Despite entering many spook words, the correct phrase remained an enigma; They had all been incorrect passwords and the secure shell remained so.
The Infowar program's conspiracy theories blabbed over the radio. Posters on the wall proved the basement dwelling nerd who summoned her clearly had a fetish for redheads and Skully became uncomfortably aware of him now staring at her rack.
"Do we have to listen to this crap, Bubba?" said Skully in an effort to divert the nerd's attention from her.
The radio clicked off, and Bubba adjusted an old TV's UHF dial, past Bugs Bunny and paused momentarily on a news broadcast:
Police say the hacker group Anonymous ran into a speedbump of sorts after targeting a Unix Security Firm--
"Lame," remarked Bubba as he resumed his search, "Pseudonyms should be cool or at least ironic... like Verisign ."
Bubba chortled gelatinously as he arrived at his favorite show and said, "Firefly is great, ever seen it? Used to be on another Chan..."
Skully wasn't listening, she was intently staring at the debugging output on one of the various screens. Surrounded by miscellaneous glyphs and hexadecimal numbers was a single coherent word, "sardine".
Skully attempted the password, and immediately gained access. "That executive's fish breath wasn't exactly top secret."
--------
I hope the spooks find it an entertaining read.
Re: (Score:1)
Re: (Score:2)
I've been using this signature since 1998, so welcome.
Privacy? (Score:5, Interesting)
Who thinks Facebook is private? The whole point is to *not* be private, right? Otherwise... what is the point of Facebook?
If the FBI was going to start monitoring encrypted email, VPNs, and other things where you are *trying* to be private, I would be concerned (yes, I know-- whole 'nuther can o' worms). But Facebook? You are giving the info away as a user, that is the purpose of having a Facebook account.
Re:Privacy? (Score:4, Funny)
Re: (Score:3)
Re: (Score:1)
Its never put anything online that you don't want somebody else's lawyer holding up in court.
Facebook "privacy" (Score:1)
Don't make the mistake of confusing "the world I want" with "the world that is". Facebook is a private enterprise, it is free, and it is deeply flawed.
I wish I could park my fancy convertible outside the liquor store with the engine running... and it still be there when I get back. Alas, I must suffer through the world that is. And someone is going to steal my car if I make it too easy.
Re: (Score:3)
I'm saying: don't assume it stops at 'public' stuff. That's stuff you could already assume would be consumed by people outside of your obscure circle.
After the whole NSA AT&T traffic sniffing stuff, I assume that anything that goes over the wire that isn't encrypted (and even some that is) can be seen by the government.
Can you get Facebook to delete your info? (Score:5, Interesting)
Re: (Score:3, Interesting)
Wiki-leaks releases tons of information on the government and banks and they get punished for it, Mark Zuckerberg does the same thing to people and he gets praised.
On another note, my spell-check that Firefox uses wants to change "Zuckerberg" to "Rubbernecker". Interesting...
Re: (Score:3)
Wiki-leaks releases tons of information on the government and banks and they get punished for it, Mark Zuckerberg does the same thing to people and he gets praised.
REALLY? You're comparing a group that publishes illegally obtained secret documents that discuss high level sensitive international operations with a group that has information that people willingly give to them, signing an agreement that explicitly allows this, and that information consisting most of the time of innocuous Farmville status updates.
Re: (Score:2)
Re:Can you get Facebook to delete your info? (Score:5, Informative)
Scream about it all you want, but you accepted the terms.
Re: (Score:2, Insightful)
Well, you know, not everyone lives in the States where you can put any kind of garbage on paper. In other countries the data still remains under your control as this is a very basic and fundamental *right* and if you want to be removed, companies need to comply no matter what is written in a contract.
Now, you may start saying that this is a US company after all. Then again, since US arrogates itself the right to enforce US laws in other countries, I hope EU will have enough balls to do the same and black-ou
Re: (Score:2)
Re: (Score:1)
It doesn't change a single thing if it's not admissible in court. Just the fact that it's regularly by police, and in general to do screenings, tells me a lot about the common culture and what to which extent the Sheeple is prepared to be abused.
Then again, you don't even have to look at polygraphs, there are plenty of other abuses TSA, Habeas Corpus, DMCA and the list goes on and on and on. Funny it's still considered the land of the free by a lot of regular people. It's like everyone has been hypnotized f
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Re: (Score:2, Insightful)
Re: (Score:2, Informative)
Re: (Score:2)
First, I realize my post seemed targeted toward you specifically. It is a general rant, no offense intended.
Second, requesting your info be deleted != your info actually being deleted.
The overall point is that is ridiculous for people to believe the crap they put on social media can simply be deleted. It is like shouting your personal details to a room full of people and then asking them to forget what they just heard. The genie cannot be put back into the bottle.
Even worse, ther
Re: (Score:2)
google???
http://www.facebook.com/help/search/?q=how+do+i+delete+my+account [facebook.com]
If you do not think you will use Facebook again and would like your account deleted, keep in mind that you will not be able to reactivate your account or retrieve any of the content or information you have added. If you would like your account permanently deleted with no option for recovery, log in to your account and then submit your request here.
Whether they truly delete it or not is beyond me, but it's not the same as deactivation.
Made For TV? (Score:3)
Believe me, I understand why we should all be nervous about projects like this being carried out by the Three Letter folks.
But thinking about another aspect, this sort of thing is always outsourced to some defense contractor who in turn lakes way too long to soak the taxpayer for software that ultimately either fails or does itâ(TM)s job poorly.
Why donâ(TM)t agencies like the FBI recruit âoeSpecial Agentsâ to work in âoehacker labsâ to turn this stuff out in-house? I mean, such a job description is âoemade for TVâ, totally sexy! A bunch of hot young geeks in the latest styles sitting around with holstered Glocks, in a hacker lab with the latest toys?
Who WOULDNâ(TM)T want that job?
Re: (Score:2)
I have to ask what you use to post with because all your quotation marks show as "â(TM)" and "â".
Re: (Score:2)
It's what happens when you use the old (D1) posting form [slashdot.org] and type or paste characters that Slashdot doesn't like. Either he copied-and-pasted that from somewhere, or he composed it in Word and pasted it into the posting form, or he used the Alt-codes not realizing that Slashdot would scramble them.
The new (D2) posting form automatically replaces a subset of special characters with their HTML character entities, such as the decorative quotes (“ and ”). Of course, using the HTML character
Re: (Score:2, Interesting)
The copy and paste buffer, even if you believe it to be plain text, contains bits per byte which are part of the overlying OS. Some people know where those bits are and manipulate them.
For example, when editing in MS-Word, open a plain text document--a real one (such as is generated by notepad). If you allow autospell to correct something, even if you save the document again as plain text, there will be an artefact of the autospell replacing text and some of it may be marked as UTF-8 (or other standard en
Re: (Score:2)
Wow, either you are a masterful troll who just trolled some mods hard, or I'm a complete fucking idiot.
Are you really saying that by using PEEK and POKE statements in BASIC one can "alter bit timings" and thus load a complete alternate OS via TCP/IP onto an unsuspecting network machine, and that the instructions you load into the machine will basically be running 180* out of phase with the normal OS, so the two will simultaneously run but aren't "aware" of each other?
LOLOLOLOLOL
Well played....
(moderations u
Re: (Score:2)
"Scrape" (Score:2)
As in, the bottom of the barrel.
They will know all of your dirty secrets because even if you don't spill, your friends will.
Terrorists putting their plots on FB? (Score:5, Funny)
Do we really think that terrorists are going to be coordinating things via FB?
It's like that old joke, what was Bin Laden's last FB entry?
"BRB, someone's at the door"
*Seal Team 6 likes this
Re:Terrorists putting their plots on FB? (Score:4, Insightful)
Terrorist or Freedom Fighter?
Egypt and Libya's uprisings were greatly facilitated by twitter and other social networking sites.
Not to say that we should overthrow the government, but, what about them using it to keep tabs on, say, the Occupiers and then using threatening, but legal, actions to undermine them?
Re:Terrorists putting their plots on FB? (Score:5, Interesting)
(Correction) Re:Terrorists putting ... (Score:1)
Re: (Score:2)
So I would assume the FBI motivation is more on these lines as the likelihood of social media sites can get legally shutdown.
I recall that as well. Maybe the FBI doesn't want to shut it down, though- by passively monitoring, they can determine where the next gathering point will be. If they were to shut it down, another method of communication (not as readily intercepted) would be used rather quickly. It would probably take protestors time to learn which services (or accounts) were compromised and, by selectively targeting demonstrations, they (the FBI) could further obfuscate the data source.
Re: (Score:3)
Exactly.
Reframe it this way - did people stop pirating music once the RIAA sued Napster? No, they moved onwards to Gnutella, Bittorrent and other P2P mechanisms. In fact, killing Napster make the alternatives more popular and even harder to stop. Taking do
Re: (Score:2)
Do we really think that terrorists are going to be coordinating things via FB?
It's like that old joke, what was Bin Laden's last FB entry?
"BRB, someone's at the door"
No, it was "Osama needs credits for farmville :("
Re: (Score:2)
The real pros, highly unlikely. The unpleasant-but-pathetic wannabes who end up getting sucked into FBI stings where the FBI has to do all the work because the perp is kind of a loser, quite possibly.
Assorted domestic political groups that the FBI wishes to harass or disrupt*cough* COINTELPRO*cough*, Yup, you betcha...
Given their sordid history, I strongly suspect that the FBI is interested in a definition of 'terrorist' that goes well
Re: (Score:2)
Given their sordid history, I strongly suspect that the FBI is interested in a definition of 'terrorist' that goes well beyond Mr. Ibn Muhumad Jihad al Anthrax and includes a fair number of much more prosaic domestic groups, who are probably twitbooking and facester-ing just as much as everybody else.
The part that bothers me about all this? I have deliberately blocked Facebook and its collection sites in my /etc/hosts file. Perhaps, just perhaps, I will be seen as a subversive (either for using Linux; or, one) who is "hiding my identity" in order to do wrong (no: I'm hiding it in order to not be a [victim/advertising metric]). Thus my concern at "the definition of 'terrorist'" you posted.
Re: (Score:1)
You're a long way off from being interesting to them. Actions like these are mostly about low hanging fruit. They know the smart and dangerous types are using other channels. This tool will be mostly used to monitor trends, map out where people will gather en mass, and somewhat occasionally track specific individuals. There's just too many people to watch all at once, even with today's technology. You have to do something to pop up on their radar. Until then, your lack of social network visibility will suff
Re: (Score:2, Insightful)
This isn't about terrorists. This is about Occupy Wall Street and the FBI being able to feed real-time intel to local police departments.
Re: (Score:2)
Re: (Score:2)
Depends on how you define "terrorist". Since lots of agencies are quick to try to label you as one...
Not news? (Score:3)
http://www.guardian.co.uk/technology/2009/oct/14/mexico-fugitive-facebook-arrest [guardian.co.uk]
Re: (Score:2)
Re: (Score:2)
FBI's Soliciation For Web Scraping Software Fools Public Into Thinking They Don't Have Such A Thing Already
Or maybe they don't, and they're tired of the CIA not sharing.
Haven't we all watched the video that shows the links between Facebook and the CIA?
in the early stages = v3 (Score:1)
Okay, so v1 and v2 were fairly quiet, but now were about to be higher profile with the results of our database scraping... so now we publicly state we're working on it.
I can configure my low end 'smart' switch to duplicate all traffic coming in on a port to another. If I can do this, then you know that it's trivial for them to. No longer a need for the 'secret' room at the telco. Now we can just duplicate and redirect all traffic to a huge server farm. Raptor or velociraptor, or my balls... whatever y
O rly? (Score:1)
Duplication of efforts? (Score:2)
The FBI and other three letter acronyms have been using programs for years that monitor the internet.
Echelon, carnivore, et all. Why do they need a whole new system just to include social networking sites?
Re: (Score:3)
Re: (Score:2)
Carnivore is very limited in what it can do
New communication systems can be exploited in different ways, and so new technologies need to be developed.
The value of Facebook is not just in what people are typing, but in what their friends, friends of friends, etc. are typing as well.
Burma Shave.
Re: (Score:2)
Why do they need a whole new system just to include social networking sites?
Maybe so people think they don't have that capability?
Prediction.. (Score:2, Funny)
Re: (Score:1)
Breaking news!! (Score:2)
Al-Zaida Terror1sts develop tricobalt salted antimatter bomb. Antardic ice sheet vaporized. Millions of penguins now homeless.
Backdoor Cisco Router Scrape! YOU WERE WARNED! (Score:1)
Spook BackDoors In Cisco Routers
- Older news, but still relevant!!
Please save this story and repost it everywhere
Especially in Security Discussion Forum Sites
- You should use OpenBSD or a hardened Linux distro
For a router, NOT these blackboxes offered with
proprietary hardware & firmware!
http://www.forbes.com/2010/02/03/hackers-networking-equipment-technology-security-cisco.html [forbes.com]
"Special Report
Cisco's Backdoor For Hackers
Andy Greenberg, 02.03.10,
I'd like to optout unless I'm suspected of a crime (Score:2)
Wake Up Mr. President! (Score:4, Funny)
Here's what it'll be: (Score:2)
It'll be a hundred million dollar application that searches Twitter for the "#bomb" hashtag.
Ok everybody,,, (Score:2)
...make sure all threats are using social networks......
Re: (Score:2)
When you realize the foolishness of what is claimed you will then realize, if you are thinking for yourself, that this is just like the telecom spying on Americans years ago (post 9/11) with the claim they were looking for terrorist. (and the GOV let the telecoms off in a lawsuit against them)
But the much more believable probability, then looking for coded common language (language/vocabulary use is only relevant to the agreed upon meaning by those using it - meaning whatever those using it want it to mean.
They weren't already doing this? (Score:2)
The only thing I'm surprised about is that they weren't already doing this. It's not like Facebook is new or anything.
As long as they are only scraping public posts, I don't see any problem with this. I'd even be fine with Facebook providing them with an API to make it easier to scrape public posts.
Damned /. Editors! (Score:2)
Misspelled 'scrap' again.
It'll Be Good for Policing Organized Crime (Score:1)
I keep getting Facebook notifications about people attempting to acquire more cell phones and baseball bats, as well as coordinating with other people who need help with fighting crime bosses.
I disabled my facebook acount by... (Score:1)
Well That Explains It... (Score:2)
Re: (Score:2)
Yeah, same here, and they keep tagging photos with me in it, showing the location as "just outside your house".
Track record (Score:3)
http://en.wikipedia.org/wiki/Virtual_Case_File [wikipedia.org]
2001 Projected started just after 911. ...
2009 The FBI is years behind and millions over budget ... only half of the project's four-phase development had been completed
2010 The FBI is $100 million over budget on the
2011 The FBI's upgrade of its computerized case file system has hit another snag
Anybody hear of Palantir? (Score:1)
All it looks like to me is that the FBI is finally joining the other TLAs in putting out an RFP that specifically is tailored to the makers of Palantir getting a contract.
Re: (Score:1)
Palantir. Those were in the ark of the covenant that the Israelites carried across the desert. There was one in the temple of the Lord that Samuel heard talking to him. The ancient Egyptians had a tuneable cochlear mechanism built into some of the pyramid temples which allowed them to listen to the palantir sequentially in a life-sized checkerboard fashion. Dragon Orbs are palantir using sunlight for full-screen on the wall projection.
And when the time for Pentecost was fulfilled the apostles were gathe
Well, thank goodness *someone's* scraping it... (Score:2)
It's gotten pretty darned crusty by this time, let me tell you!
too late (Score:2)
I would guess that commercial interests are already raping the internet for info now, just buy off-the-shelf software.
Expensive RSS feed (Score:2)
Don't immediately assume that because the government is doing it, that it's bad
There's millions of people running around with smartphones that constantly yap
And before an
Re: (Score:2)
Making the world safe for Corporatism! (Score:1)
You get what you deserve (Score:2)
Unlike (Score:1)
-KI
they already have this. (Score:2)
if you have read the Anonymous dump of the HBGary emails, they already have tools that will scrape facebook. Then you can internetwork that with Palantir Technologies and Berico Technologies stuff to make your own "Team Themis".
They should ask NSA for the data. (Score:2)
NSA most probably is doing this for a long time now.
Re: (Score:1)
If you believe the whack jobs out there, Google is run by the NSA so you would think they would just share what they have :)
Re: (Score:2)
Re: (Score:2)
I know right, all they hire are elite intellectuals.
Re: (Score:2)
I thought Google was ticked off that they couldn't get this, thus Google+.
As for govt monitoring and especially doing it by subcontracting to defense contractors, I have a long list of govt agencies and defense contractors that have spidered my little site through the years, and I mean a long list.
They might be tweaking to better assimilate random social site comments but they've been mining the web for years.
Re: (Score:2)
Re: (Score:2)
Re:and here comes slashdot, late again (Score:4, Insightful)
why does this site even exist anymore? reddit posts everything first, with less bias, and without all the self-loathing commentators screaming shill/troll/astroturf/mccarthyist label of the day.
We're here specifically to annoy you, AC. Looks like we're on top of our game again.
Re: (Score:2)
Yeah, I find I can't start my day without a goatse link.
I want to know where the cookie recipe guy went, I liked that troll.