New Software To Balance Privacy and Security?

An anonymous reader writes "Claiming to provide both security and privacy, researchers at UCLA say they have developed a system to monitor suspicious online communication that discards communications from law-abiding citizens before they ever reach the intelligence community." From the article: "The truly revolutionary facet of the technology is that it is a new and powerful example of a piece of code that has been mathematically proven to be impossible to reverse-engineer. In other words, it can't be analyzed to figure out its components, construction and inner workings, or reveal what information it's collecting and what information it's discarding -- it won't give up its secrets. It can't be manipulated or turned against the user."

Evil potential here

That means lawful U.S. citizens who don't fit the parameters are automatically ruled out.

It also means that lawful citizens who do fit the parameters are reported on. The same as if the agencies are grepping.

a savvy person may be able to tell that the program is running in the background ... by distributing this software all over the Internet to providers and network administrators, you can easily monitor a huge data flow

How will this software be "distributed"? Virus? Payload in a Sony rootkit? Thousands of patriotic sysadmins? Plenty of potential for evil to be done here!

This magic software only finds bad guys?

If that isn't putting the priest in charge of Sunday School, I don't know what is.

The problem is not Privacy vs. Security. You will never have Security. Not yours. You can have privacy, though.

The problem is, and always has been, balancing privacy and convenience.

spin doctors

So, it collect all data fitting into the criteria set by the agency without any chance of anyone ever knowing what those criteria were? How is the "law-abiding" citizen to know he's not accidentaly fitted one? They say it improves privacy, but it actually removes it, since you can never know you've not been deemed a "terrorist".

What good is this?

So, when I take my case to court, that they're illegally intercepting my communications just to look for dirt to ruin my political campaign, it's impossible to reverse engineer and prove that they were only looking for terrorists?

I mean, the captured documents could already have been altered, no way to prove that they didn't, now.

Not to mention the way it works amounts to what is essentially an eternal wiretap of everyone, guilt, innocence and suspicion matter not.

Mathematical proof of code is a tough business

Their greping thing is not interesting per itself, but I'd like to see this:

[...]a new and powerful example of a piece of code that has been mathematically proven to be impossible to reverse-engineer[...]

I'd like to see the demonstration. Until such time, I call bollocks and I refuse to believe an "impossible to reverse-engineer" piece of code ever exists.

Re:Mathematical proof of code is a tough business

I'd like to see the demonstration. Until such time, I call bollocks and I refuse to believe an "impossible to reverse-engineer" piece of code ever exists.

I second your bullshit and raise! The problem with proofs such as this is that they assume broad axioms that in reality might not be true in the hardware. For example, they may well have proved the theorem if they assume all operations of a certain set take the same length but in reality they might not. The processor might take a ten billionth of a second longer to do one operation than it does another, or it might release more heat when it does one operation than it does when it performs another, or it might release a certain magnetic field when it does one operation and not another.

Side-channel attacks, as these are called, are often totally devastating. There was one attack [schneier.com] where simply heating the computer up can cause a system to get owned. If the proof is correct, it's certainly interesting but practically we're a long way from getting to this gold standard.

Simon

Social Engineering

So, it has been mathematically been proben impossible to reverse engineer... has it also been mathematically proven impossible to socially engineer?

Like the Stasi?

And who gets to define what a "law abiding" citizen is? It may be OK now but what happens when the law is that you do not oppose the state, whoops, too late, there is already the infrastructure in place to find out where those damned pro-democracy scum are and what they are upto.

Next, when we're all watching TV and doing our VoIP on the net, all have our home security systems on the net then the government 'sees' everything, 'knows' everything and you have entered into the police state where you can't even move without it being reported on.

Nonsense!

In other words, it can't be analyzed to figure out its components, construction and inner workings, or reveal what information it's collecting and what information it's discarding -- it won't give up its secrets.

Maybe they have a mathematical proof that makes reverse-engineering impossible. Fine. But it is still possible to find out what it does in practice, since the nature of the data it processes is known. Just run it in a simulation and see what it does. No reverse-engineering required. From there onwards, it can be turned against the user and manipulated.

Lesson: Mathematical proofs are fine. But they have a specific scope. Not understanding that scope makes the proof useless and can result in faulty claims about how possible or impossible something is.

And what are the criteria?

How much you want to bet that some of the criteria include using encrypted communications and annonymizing proxies and other legitimate security measures that people will start adopting exactly because this kind of snooping system exists? It's a self-fulfilling prophecy.

Impossible to reverse engineer!

...it is a new and powerful example of a piece of code that has been mathematically proven to be impossible to reverse-engineer...

Brrrrrr.. spooky! This sounds like an incredible misinterpretation of whatever the original paper/research is actually doing though. Devices may be reverse engineered without even looking inside if you have access to its inputs and outputs and can continually test and hypothesize and retest, etc. A device that distinguishes between 'evil' and 'regular' packets (as input) and outputs a bit that indicates 1='evil' or 0='not evil' (or a floating point degree of evil, say..) is no different. If you have access to the code that runs this particular device running on a router or somewhere packets drift by then obviously the situation can be no worse. I'd definitely like to see the link to this mathematical proof..

Impossible to reverse engineer?

Which CPU does it run on?
Which executable format does it use?
Unless its running on dedicated hardware with really strong encryption (and even then, thats no gaurantee), it is possible to reverse engineer any piece of code piece by piece (for example, start with the first instructions the program executes and unwrap it from there). If you wanted to go deep, you could use an ICE or similar (or a software emulator with a built-in debugger that cant be detected from the emulated side)

Here's a more informed article on this software

http://www.research.ucla.edu/tech/ucla05-487.htm [ucla.edu]

Proof?

How can they claim it has been "mathematically proven to be impossible to reverse-engineer" without having first submitted the code to peer review? My house can be mathematically proven to be impossible to break into. But tell that the the guy with the ski mask and the crow bar.

It can't be manipulated or turned against the user

Of course not -- after all, it's already against the user.

For crying out loud, this is spyware, by definition.

What's the bets that VISTA will have it?

"With this new technology, based on highly esoteric mathematics, the software can be distributed to many machines on the Internet, not necessarily trusted or highly secure. The software works by analyzing all of the data and then having the appearance of putting all the data into a 'secure box.' A secret filter inside the box dismisses some data as useless and collects only relevant data according to the confidential criteria that can be programmed into the software. And because it's all done inside encrypted code, it's not apparent which, if any, of the data has been selected and kept, except by the person who has deployed the filter and has the decryption key."

methinks this will be stuck into VISTA and possibly forced onto XP users via a mandatory update.

One step closer to Big Brother .....

Isn't this like profiling? Everything can be reversed. Just wait until the dark side gets a hand in this.

So let me get this straight

This new software selects which communications are of interest to the intelligence community using an undisclosed algorithm. This algorithm "cannot" be reverse engineered. We just have to take the government's word for it that the selection criteria are correct and are unrelated to anyone's personal or political agendas. This will somehow "may ease some of these privacy concerns by making the tracking of terrorist communications over the Internet more efficient, and more targeted, than ever before."? I don't know about you, but it doesn't ease my concerns in the slightest. Just the opposite in fact.

Yeah, right

Anyone want to bet that this "amazing, mathematically proven impossible to reverse engineer" software is going to implicate at least a few innocent people? A major part of the 9-11 problem was that the CIA had all but abandoned human intelligence in favor of the computerized variety. Computers can prove a lot of intel, but only so much and an agency that doesn't rely on agents and contacts in the field is one that'll miss key intel that could be disasterous. Only humans at this point can use intuition. We need that intuition in order to piece together intelligence reports correctly.

This software will only contribute to the "if you have nothng to hide, you have nothing to fear" mentality. People will ask why you showed up in the sweep if you aren't hiding something. Things like this just undermine guilty until proven innocent and will only serve to make our agencies lazier, not more efficient.

It really is possible to stop reverse-engineering

Many commenters are claiming "it is always possible to reverse-engineer a program!," using such reasons as "you can always watch the processor perform the instructions and eventually figure it out."

Let me tell you, as a cryptographer, that these claims are false. The recent field of program obfuscation gives surprisingly strong ways to prevent reverse-engineering, in a very rigorous and strong way.

Not every program can be obfuscated (this has been proven). However, programs that fit a certain template (like: "check if the input string matches the user's password") can be obfuscated. What this means is that you can give the program's entire code to the adversary -- he can run it on his own computer (no DRM required) on whatever inputs he likes, alter it, stretch it, twist it, whatever. After all this he still will not be able to guess the password, any more than if he had some mathematically-perfect black-box that truthfully answered the question: "is [X] the password?" (Actually the definition is even stronger than this, but that's the gist of it.)

Yes, this seems extremely hard to do -- after all, the adversary has complete and total power over the code that is running. Yet it can be done, rigorously and provably, if you're willing to believe that there are some number-theory problems out there (like RSA) that are hard to solve.

For the work described in the article, it sounds like the "black-box" does something like the following: if your input string contains some "watch words," then the output is the same as the input, but encrypted under the government's key. If your input string is "benign," then the output is just "THIS WAS A BENIGN INPUT", encrypted in the government's key -- i.e., it ignores any benign input and replaces it with a placeholder. By running the obfuscated program and looking at the output, you can't tell if the input was flagged or not. Even while watching the program run, you can't tell if the program is flagging the input or not (or learn anything about the government's key). When the government collects the output and decrypts it, it only sees the flagged inputs, as the rest have been ignored.

As I've said, none of this depends on the program requiring any DRM or TPM or any other specialized hardware. It only relies on the mathematics.

wow, is this for real?

If this can be done, and i see no reason why it cannot, then wait till the bad guys find out how it works and start to make worms that cannot be reverse-engineered.

Oh brother.

Um...WTF

Uh, have we entered some new bizarre Orwellian Twilight Zone? So basically an uncrackable secret black box that the government can install on any machine to intercept any traffic with no ability for the surveilled party to repudiate the content (or perhaps even be aware of the surveillance?) is somehow a win for privacy? WTF.

BREAKING NEWS. The government has devised a fool proof plan to protect your privacy. They will simply garrison an intelligence agent in your house recording everything you do to make sure that the government doesn't inappropriately invade your privacy. (for your own safety please do not attempt to resist; you will have to be beaten to protect your own privacy, after which you will be dumped in a shallow unmarked grave - again for your privacy)

Abiding by the law and mathematical proof.

> "...researchers at UCLA say they have developed a system to monitor suspicious
> online communication that discards communications from law-abiding citizens
> before they ever reach the intelligence community."

"Law-abiding": which laws might that be? The laws intended to prevent disruption of society, like the ones used to jail many civil-rights activists in the 50s and 60s? The laws that declared a black man couldn't marry a white woman? Or the ones that declared a woman can't own real property?

Some of the very -best- people are by definition lawbreakers.

> From the article: "The truly revolutionary facet of the technology is
> that it is a new and powerful example of a piece of code that has been
> mathematically proven to be impossible to reverse-engineer.

That's a very broad statement. I haven't read the proof, so I can't say they're wrong. I will just point out that there are things "proven mathematically" in consumer statistics everyday that just aren't so. There is a difference between "mathematically proven" as used in colloquial speech and "a mathematical proof."

I wonder which this is?

Can't be reverse-engineered, eh?

Sounds like Famous Last Words...

Re:Can't be reverse-engineered, eh?

Oh the funny part - there's no need to reverse engineer it; the guts would be fully described in the resulting Software Patent.

Worst case, pull an SCO and sue them for violating your stuff, and demand un-obfuscated *everything* during discovery.

On the fun side, wait until RIAA/MPAA gets their agenda piggybacked into these little boxes.

Pointer to the actual paper (PDF, sorry)

http://www.cs.ucla.edu/~rafail/PUBLIC/Ostrovsky-Sk eith.pdf [ucla.edu]
Postscript: http://www.cs.ucla.edu/~rafail/PUBLIC/Ostrovsky-Sk eith.ps [ucla.edu]

This is a scam

"Because the code cannot be analyzed, terrorists using the Internet to communicate will never know if the filter has pinpointed their data or not."

Uhm, excuse me, but this is exactly the situation right now. Since when do terrorists ever KNOW that security is on to them until they're caught? Terrorists take precautions against being detected by ANYTHING. Terrorists with the slightest brains do not talk about operations in the clear at any time. What then is this software supposed to detect? Where is the benefit?

Supposedly the benefit is that "harmless" communication is never seen by the Fed. Bullcrap. The parameters of the software are SET by the Fed - they can see anything they want. That's obvious from the article as it glosses entirely over the matter of "criteria" in the first place.

This software would only be safe in the hands of someone who IS safe. In the words of the DRM enthusiasts, it only "keeps honest people honest." And since the criteria is changeable - as well as the appointment (or election) of the people who set the criteria - this is no security at all.

In the hands of George Bush, Dick Cheney and General Hayden, you're screwed, blued and tattooed.

This is nothing more than a propaganda piece put out at this time because Bush is in danger of being impeached over the spying issue. That's the bottom line.

False Premise

There is no tradeoff between privacy and security, so there is no need to "balance" them. An individual is not secure if their privacy is being routinely violated.

The tradeoff is between privacy and totalitarianism. Solutions that attempt to split the difference are not helpful.

George Orwell says "I told you so"

"Any sound that Winston made, above the level of a very low whisper, would be picked up by [the telescreen], moreover, so long as he remained within the field of vision which the metal plaque commanded, he could be seen as well as heard. There was of course no way of knowing whether you were being watched at any given moment. How often, or on what system, the Thought Police plugged in on any individual wire was guesswork. It was even conceivable that they watched everybody all the time. But at any rate they could plug in your wire whenever they wanted to. You had to live -- did live, from habit that became instinct -- in the assumption that every sound you made was overheard, and, except in darkness, every movement scrutinized. "

--George Orwell, 1984 [gutenberg.net.au]

Anyone see a parallel here? A black box that watches everything you do, with no way to know whether what you are doing is ThoughtCrime or not. Way to safeguard my privacy and rights.

Re:Scary, but encouraging...

A secret filter inside the box dismisses some data as useless and collects only relevant data according to the confidential criteria that can be programmed into the software

Who wants to bet the secret filter might end up being /.*/