NSA Can't Search Its Own Email 165
cycoj writes "The NSA says that there is no central method to search its own email. When asked in a Freedom of Information Act request for emails with the National Geographic Channel over a specific time period, the agency, which has been collecting and analyzing the data of hundreds of millions of Internet users, says it can only perform person-per-person searches on its own email."
big surprise (Score:5, Insightful)
Perfect example of "do as I say, not as I do". But this isn't just a NSA problem, it is a government problem.
Re:big surprise (Score:5, Informative)
More likely a case of somebody lying to get around a FOIA request, for which there will be consequences. All government agencies have very strict regulations concerning record keeping and FOIA, with jail time possible for anyone who fails to abide by those regulations.
Re:big surprise (Score:5, Funny)
Consequences. They're cute when they say that.
Re:big surprise (Score:5, Insightful)
FOIA does not require that you make it easy to comply with FOIA requests. Nothing in there says you have to have globally searchable e-mail or document storage, in fact. And the costs to fulfill the request are paid by the requestor, not the agency. By using an archaic, difficult to use system, they can legitimately make the costs of fulfilling FOIA requests prohibitively high. Thus they follow the letter of the law, though not the spirit.
Re: (Score:2)
More likely a case of somebody lying to get around a FOIA request, for which there will be consequences. All government agencies have very strict regulations concerning record keeping and FOIA, with jail time possible for anyone who fails to abide by those regulations.
Consequences? What, for the requester? I think we both know that nothing will happen. If it ever turns into a real scandal, the president will just say something stupid about George Zimmerman, wait for everyone to go insane, rinse, repeat.
Re: (Score:2)
More likely a case of somebody lying to get around a FOIA request,...
I agree. Telling porkies is their specialty.
... for which there will be consequences. All government agencies have very strict regulations concerning record keeping and FOIA, with jail time possible for anyone who fails to abide by those regulations.
Now you're dreaming in Technicolor. The liar might get a promotion for their behavior, but there certainly won't be any negative consequences.
Re: (Score:2)
"More likely a case of somebody lying to get around a FOIA request, for which there will be consequences. All government agencies have very strict regulations concerning record keeping and FOIA, with jail time possible for anyone who fails to abide by those regulations."
Agreed. It's either BS, or (literally criminal) incompetence. Based on what we have been told so far, I would have to guess BS.
Re: (Score:2)
This comment is spot on. This is a blatant lie by the NSA and National Geographic should be lining up their expert witnesses to back up such a claim in order to get the judge to force the NSA to "create" such a capability.... quickly. A computer-savvy high school kid can build an e-mail system search tool. There is no reason why someone at the NSA can't do something similar in a short period of time.
Re: (Score:3)
Re: (Score:3, Insightful)
Government is a problem that has never been a solution to anything but other governments and the failures of society. In shrot, it is a self-propogating problem. It is merely a patch for the failures of society. The bigger the government the more screwed up society is and one condition mirrors the other. "That which governs the best governs the least." Therein true anarchy (anarchy being a purposely corrupted word as used today, much like "hacker" or "liberal" etc ) would be best,
Re: (Score:1)
NSA *Won't* Search Its Own Email (Score:5, Insightful)
FTFY
Re:NSA *Won't* Search Its Own Email (Score:5, Insightful)
Re:NSA *Won't* Search Its Own Email (Score:5, Insightful)
Why wouldn't they continue to lie? Congress, the President, and the American public have made it abundantly clear that there will be no consequences for lying. So, why not?
Re: (Score:2)
Correction.
They are right to think that their buddies in DC will protect them and that we can't do shit about it.
Re: (Score:2)
If they were asked to produce all relevant correspondence then a simple search algorithm won't be relyable enough.
If that is the case and that's how they interpret that request then it is further proof that red tape is thicker than their mission statement. Cooperation looks differently. This is a bureaucracy trying to weather a storm. And this is why the NSA should be dismantled and their place should be taken by somebody who understands their mission and takes it a little bi
Exchange (Score:3)
Comment removed (Score:5, Informative)
Re: (Score:2)
There are lots of products that do this for exchange. I would bet that it's a security issue or they just don't want the feature.
For that matter, it does it out of the box and has for several years and revisions...
Re: (Score:2)
Citizen, you seem upset.
"Upset" is a condition in which one's reality is found to not match one's expectations. Frequently because of faulty expectations.
Now, what mistaken expectation did...ah, yes, here:
They're EMPLOYEES OF OURS, not our masters
Wherever did you get that idea, Citizen?
umm (Score:4, Funny)
var irony = UInt64.MaxInt;
Re: (Score:2)
Arguably. Let me just run a quick hadoop query over every email with a north american correspondent to get a feel for the consensus.
Right... (Score:5, Insightful)
The NSA says that there is no central method to search its own email.
[cough] Bullshit [/cough]
Re: (Score:2)
Re: (Score:3)
Re:Right... (Score:4, Insightful)
I think you forget that they're on the same side. That ain't gonna happen.
Re: (Score:3)
Yup: They won't use Contempt of Congress to deal with this, which is why I have Contempt for Congress.
Re:Right... (Score:5, Informative)
Re: (Score:2)
OK, the comfy chair it is!
Re: (Score:2)
That's because only the President can actually fire him.
For Congress to get rid of him, he has to be impeached.
As for prosecution, the President is also head of the DOJ that would be in charge of prosecuting him.
Do they license their email system? (Score:3)
Do they license their email system? It may the only one in the world secure FROM them.
Re: (Score:2)
Yes and no.
This is political. There are two possibilities here: Either, the guy who said that is an old-school politician, or he is of the new breed.
The new breed outright lies to you, straight into your face. That's the kind of people that run our governments today.
The old-school guys, however, will not lie. Well... not outright. In this case, he would say something that is technically the truth. For example, if they have several seperate and independent internal mailing systems - which is quite likely giv
Don't worry. (Score:5, Interesting)
what is good for the goose is good for the gander (Score:5, Funny)
Maybe they should run all their internal email through their PRISM system, that way it can be searched for keywords and META data much easier. Problem solved.
Re: (Score:2)
Re: (Score:2)
Suuuuure (Score:4, Insightful)
That's such a line of shit.
It's not that they cannot search their emails. It's that they have chosen to not create a search mechanism, because they have found this excuse is accepted by the courts to deny information requests. They will use every trick available to them to avoid adhering to laws they don't like.
Do you really believe anything they say?
Re: (Score:2)
People seem to be all worked up about the use of the word "can't".
It's a perfectly legitimate, honest, non-fibbing word to use in this context. The context being "We can't be arsed to search that email."
Like I said, perfectly appropriate in context.
Re: (Score:2)
"Can't" vs. "Don't Want To" (Score:1)
I suspect the real reason is that they don't want to, not that they can't. In the NSA's defense, when an organization works with highly compartmentalized information, having the ability to scan all employees' email is not wise from a security perspective.
Re: (Score:2)
Even the government is not stupid enough to run multiple Exchange servers within the same intranet. They probably run a version of Exchange with some add-ons they bought from Microsoft to handle classification controls. Even with multiple Exchange servers, in order for e-mail to actually be sent between subnets, they need to be connected. NSA is 30,000+ people, so we're talking about a cloud (hate that word) environment for managing that many e-mails. Querying for data from the "cloud" is not complicate
Re: (Score:2)
Plus let us consider how many of those 30,000+ people are not analysts or anything of the sort.
Indeed. While I'd expect the NSA to generally have it's stuff together, everybody needs to realize that large organizations can have varying levels of competence(and incompetence) in varying areas and tasks.
For the NSA, the lion's share of it's resources goes towards providing security and exploiting security holes. It has multiple 'defensive'(not in the news much) and 'offensive'(been in the news much more) cells, and they tend to not talk to each other. You have the mentioned admin people, but they're
Comment removed (Score:4, Informative)
Re:"Can't" vs. "Don't Want To" (Score:4, Interesting)
Suppose that instead of "National Geographic", someone at the NSA wanted to search every email that was sent to Snowden's Gmail account from within the NSA.
Do you think they would be able to do that? Not being able to do that sounds like a security problem.
Re: (Score:1)
Suppose that instead of "National Geographic", someone at the NSA wanted to search every email that was sent to Snowden's Gmail account from within the NSA.
Do you think they would be able to do that? Not being able to do that sounds like a security problem.
I thought their excuse was that they could do a search for a single email address, just not a complete domain. So snowden@nsacontractor.com would work, but *@natgeo.org wouldn't.
Others seem to be arguing that they're saying that they can only search against one employee as well.
Re: (Score:3)
Re: (Score:2)
Now that you've revealed this hole in NSA security, they'll be after you. Those damn spies are going to sign up for their own domain names, and send their secrets to a different username @stolensecrets.com every time!
Doesn't add up (Score:2)
having the ability to scan all employees' email is not wise from a security perspective.
NOT having the ability to scan email is not wise from a security perspective. If someone is doing something they ought not to be doing, exactly how is the NSA going to find out? They claim they need this ability to find out if civilians are dangerous so why would the same logic not apply to their internal operations? Makes NO sense whatsoever.
Smart design. (Score:1)
Isn't this pretty much what privacy adocates have advised for years? The NSA is one of the groups gathering people's data against their will. If anyone knows what possibilities to avoid if you don't want people in your data, it's them.
Get Snowden to do it (Score:2)
Snowden didn't seem to have a problem finding information. Maybe they just need a contractor to come in and do it for them...
Re: (Score:3)
Snowden didn't seem to have a problem finding information. Maybe they just need a contractor to come in and do it for them...
It is highly unlikely that Snowden knows everything the NSA is involved in. The stuff he released might be inflamatory but there will be plenty more he never knew about.
Re: (Score:2)
Indeed. To release something, you need to know it exists in the first place. We've all heard tales of programs that don't, officially, exist. I'm sure all the Intelligence agencies have a plethora of them, but you could never prove it. Which is kind of the point of such programs. . . .
Yep. The Men in Black have the data, and the NSA is THEIR cover (grin)
And we accept this excuse? (Score:5, Insightful)
If I were in charge, and the agency responsible for technological espionage and information security told me they couldn't search through their own emails, I would fire them. Every single one of them. Bam. Agency dissolved, someone go think of a new TLA for the new agency. This is like a Navy that can't figure out how to dock a battleship, or a tax agency that doesn't know what all the valid exemptions are. Complete and utter incompetence.
What's saddest is that this almost certainly isn't true. They've got these capabilities. They're just trying to hide something ("everything" qualifies as something, for their purposes). *Maybe* they're telling the truth, if they've got some custom, highly-encrypted system where emails can only be decrypted by the users. But that doesn't seem like the phrasing used here.
What's saddest is that "we're completely fucking incompetent" is not just the excuse they went with, but that it actually works.
Re:And we accept this excuse? (Score:5, Insightful)
If I were in charge, and the agency responsible for technological espionage and information security told me they couldn't search through their own emails, I would fire them. Every single one of them. Bam. Agency dissolved, someone go think of a new TLA for the new agency.
Not an option. The NSA has a portfolio of affairs, abuses of power, criminal behavior, tax fraud, drug abuse, etc. on every member of the government. Nobody will oppose those who could end their career in a few keystrokes. ...Or maybe I'm just paranoid.
Re: (Score:1)
If I were in charge, and the agency responsible for technological espionage and information security told me they couldn't search through their own emails, I would fire them. Every single one of them. Bam. Agency dissolved, someone go think of a new TLA for the new agency.
Not an option. The NSA has a portfolio of affairs, abuses of power, criminal behavior, tax fraud, drug abuse, etc. on every member of the government. Nobody will oppose those who could end their career in a few keystrokes. ...Or maybe I'm just paranoid.
I'm sure some oppose those who could end their career in a few keystrokes... you just haven't heard about those people for obvious reasons.
Re: (Score:2)
he NSA has a portfolio of affairs, abuses of power, criminal behavior, tax fraud, drug abuse, etc. on every member of the government.
I would like to believe that there are at least a couple members of government for which a portfolio containing that sort of data would be empty. The idea that of the hundreds of congressmen and senators, that none of them at all have avoided committing these serious violations of the law just doesnt seem reasonable. Its reasonable to suppose that most of them are guilty of serious violations of the law, but not all of them.
Re: (Score:2)
Re: (Score:2)
Paranoia doesn't not exclude the possibility that they really are after you.
Next we find out that the X-files was based on deliberately leaked government files just to discredit the truth if it ever came out, and that FEMA really is planning a government takeover real soon now.
Re: (Score:1)
They may have some sort of distributed email system, based on something old / classified. Think of it from NSA's side for a moment. You have this network of people with some of the most secret stuff in the US, and you are in charge of designing an email system. You're worried that someone could gain access to the servers, maybe even physical access, and you want to minimize such damage if it happens.
I could easily imagine that each person or workgroup would have physically separated mail servers.
Re: (Score:2)
Re: And we accept this excuse? (Score:2)
Re: (Score:2)
What's saddest is that "we're completely fucking incompetent" is not just the excuse they went with, but that it actually works.
Not sure if you've noticed, but that's been the US governments MO for decades for hiding their true purposes.
Re: (Score:2)
If I were in charge,
I've found that most of the time when someone begins a statement with "If I were..." they usually continue to describe one of the reasons they aren't. The people in charge are fully comlicit with what the NSA is doing. The NSA are the ones keeping them in charge.
Re: (Score:2)
*Maybe* they're telling the truth, if they've got some custom, highly-encrypted system where emails can only be decrypted by the users.
Incredibly unlikely. The NSA would be the #1 top candidate for a list of "organisations that know what key escrow is for".
If you work in the intelligence sector, you should understand that people can go missing or become no-longer-trustworthy very unexpectedly and you definitely do not want to lose access to their data. And in the later case, you absolutely want access to their data, especially communications, to check if they did any damage and what they knew.
Re: (Score:2)
Re: (Score:2)
That is what I first thought. They want to compartmentalize internal emails to keep things compartmentalized. +1 indeed sir!
Re: (Score:2)
Remove funding, destr their systems, fire them all (Score:2)
It's time for Congress to disassemble this agency. Their track record of protecting the American public is horrible.
Re: (Score:2)
Disassemble? No disassemble Johnny 5!
Re: (Score:2)
Epic Fail (Score:5, Insightful)
incompetence (Score:2)
Same ol' NSA protocol (Score:3)
Lie, lie, lie, until you get caught, 'cause there's nothing to lose and everything to gain.
Sounds like the Onion.. (Score:2)
.. but ends up as truth.
Seriously though, the NSA is directly involved in lying to Congress. Do you think they would have any system that would allow easy discoverability of their misdeeds? I am sure their processes are in place to make any type of lawsuit or congressional oversight as difficult as possible.
Of course, any results this poor fellow would have received anyway would be just pages and pages of blacked out text with the headers and footers as they only "public" information.
Re: (Score:2)
Congress cannot fire him, only the president can do that.
And don't hold your breath waiting for impeachment.
NSA (Score:4, Insightful)
NSA doesn't fund their operations primarily with drug running anymore. Insider trading is the best source of funding. And they have all the information they need to do this.
Re: (Score:2)
Easy fix. Supply the Metadata then. (Score:2)
Or whatever large glob of crap they store the email in. They do have servers, right? Somewhere on those servers is a file, RAID array, partition, SAN or JBOD, Most likely encrypted. Hand over the disks, the encryption keys and whatever else is needed to access that Metatdata.
Have the court appoint someone approved by the EFF to sign an NDA/Gag order/whatever and sift through the Metadata removing items which are of key importance. If those top brass military brats think they are above the law, go around
Smarsh (Score:1)
Does a good job tracking emails....
being difficult is not an excuse (Score:1)
I can see one way (Score:3)
I can see one way in which this might be both true and proper. If each account was individually encrypted with keys that only the users had, what they're saying would be completely true. And I think it would be completely proper and even laudatory to run an email system that way. They could search individual accounts by having the users decrypt them, but they couldn't do a wholesale search of the entire email system. This is the way email should be!
A somewhat more likely approach would be that by policy, users are not allowed to keep email on the server. All email must be downloaded or deleted. No online folders, ridiculously small INBOX quotas, maybe a read-once policy where as soon as the mail is retrieved the server auto-deletes it. I can actually understand this being done; I've worked with corporate lawyers who would love to have the email system set up this way for the express purpose of defeating global searches. Anything can be twisted and used against you, so save nothing, leave no evidence. I certainly don't agree with that mindset, but I've worked with people who are like that.
Not that I think it actually is done either of those ways. I think it's far more likely that they're simply lying and refusing to comply. It's probably simply policy to refuse such blanket FOIA requests, and there's undoubtedly a clause buried in the FOIA itself that allows them to require that requests be specific and narrow. You know, in the way that searches of private individuals are supposed to be.
Sounds unbelievable (Score:2)
Then it is.
this is what (Score:2)
The NSA flipping their finger at you looks like.
Working Phrase (Score:2)
By design? (Score:2)
They could be lying, or their system could be deliberately designed this way, to limit the amount of information a mole could find.
Can't do `tail -f mbox`? (Score:2)
The Watcher (Score:2)
Simple fix... (Score:2)
The NSA should simply host their email with Google and it will be instantly searchable through PRISM.
Infact the NSA should compete with google and offer its own free email and search engine, PRISM. Just PRISM it!
Re: (Score:2)
I am reading a spy novel at the moment that relies a lot on email intercepts to move the story along. I was thinking to myself that a great way to get all the "bad guys" email would be to create a free "anonymous" email service, or get someone else to do it for you.
Since this book is mostly concerned with middle eastern terrorists I thought they could call it: Allahmail
Re:sounds like they're running exchange (Score:5, Insightful)
Re:sounds like they're running exchange (Score:4, Insightful)
Even if their server can't do it what about their backup repository.
Re:sounds like they're running exchange (Score:4, Insightful)
As an Exchange administrator, I can say that searching across an entire mail database is absolutely possible, and also very simple to do from the Management Shell. They're either lying, or just don't want to do it.
Of course it's possible with Exchange or with anything else for that matter. There is an exception to FOI requests where getting the information is expensive. What they mean is they can't do it within whatever small budget they allocate to serving FIO requests.
Re: (Score:2)
A budget that they make small on purpose.
Don't kid yourself, they are being difficult on purpose.
Re: (Score:2)
As an Exchange administrator, I can say that searching across an entire mail database is absolutely possible, and also very simple to do from the Management Shell. They're either lying, or just don't want to do it.
It is also possible on Notes, any Unix mailbox format (maildir, mbox, maildir+, w/e), and any other e-mail system I can think of.
Re: (Score:2)
Re: (Score:2)
OK, fine.
Here's a solution: NSA takes their entire email database and dumps it out to Wikileaks.
We'll take it from there. Gratis.
Profit!
Re: (Score:2)
As an Exchange administrator, I can say that searching across an entire mail database is absolutely possible, and also very simple to do from the Management Shell. They're either lying, or just don't want to do it.
Also as an Exchange administrator, I can assure you that if your database is sufficiently large, such a request can *still* take forever to fill. You're basically looking at minimum 1 day per Terabyte of email. The interface is not quick, and it does crawl it mailbox by mailbox. Yes, you can do it with a single command, but it is prohibitive on staff time to setup/execute and lawyer time to redact the non-responsive bits.
Re: (Score:3)
Re: (Score:3)
Re: (Score:2)
Yeah this story sounds real. I am not seeing an accompanying article or evidence. Not saying it's not real, but it sounds like one of those government conspiracy theories like the government faked 9-11. Competent enough to stage 9-11 in front of cameras and people, with no one leaking it, then incompetent, for instance Snowden leaking information about the NSA. Who knows...
I suspect it is a very real FOIA excuse. Whether the excuse is valid or not is hard to say.
Within the bureaucracy, you can tell the FOIA officer your limitations and s/he sends them back to the requester.
Then it's on to step two: Natl Geog calling them full of congress and demanding information about their e-mail system (probably already available on a Chinese site).
And I'll bet the NSA is not lying.
I'll bet they have multiple different e-mail systems, all of them installed by different contractors at