Hyper-Threading, Linus Torvalds vs. Colin Percival 396
OutsideIn writes "The recent Hyper-Threading vulnerability
announcement has generated a fair amount of discussion since it was released. KernelTrap
has an interesting article quoting Linux creator Linus Torvalds who recently compared the vulnerability to similar issues with early SMP
and direct-mapped caches suggesting, "it doesn't seem all that worrying in real life." Colin Percival,
who published a recent paper on the vulnerability,
strongly disagreed with Linus' assessment saying, "it is at
times like this that Linux really suffers from having a single dictator in charge; when Linus doesn't understand a problem,
he won't fix it, even if all the cryptographers in the world are standing against him.""
He won't fix it? (Score:5, Insightful)
Re:He won't fix it? (Score:3, Insightful)
Re:He won't fix it? (Score:1, Insightful)
Dictator? (Score:5, Insightful)
Come on.
Single Dictator? (Score:3, Insightful)
If Microtoft decides not to fix it, then no one else can.
so which one is a single dicatorship?
bad tactics from Colin Percival (Score:3, Insightful)
If he and "all the world's cryptographers" can't do that, then Linus' pragmatism beats the cryptographers paranoia (their defining quality, in my experience) into a cocked hat.
If an exploit can't actually be exploited, it's not and exploit.
OK Colin, Well done (Score:4, Insightful)
Re:He won't fix it? (Score:3, Insightful)
Linus the great Dictator (Score:3, Insightful)
Colin needs to cool down a bit. At least the Linux distros (SuSE, Red Hat, etc are the ones which will get a problem from affected systems) are going to get patches for it. From Linus or any other Kernel developer.
Re:He won't fix it? (Score:5, Insightful)
So MS$ shouldn't fix problems in IE until an exploit has been shown for it?
It's better to wait and see before fixing something that may not matter later.
Its better to just fix it and be safe than wait and see if something happens later. It may not be top priority, but remember this "wait and see" approach to security is exactly what got MS$ into so much trouble with users. We don't need the same for Linux.
Re:He won't fix it? (Score:3, Insightful)
Fixing is easier said than done (Score:5, Insightful)
Re:He won't fix it? (Score:3, Insightful)
Some people are just keep pushing their flawed agendas.
Disclaimer: i did read the whole thread.
Re:OK Colin, Well done (Score:3, Insightful)
Re:OK Colin, Well done (Score:5, Insightful)
Re:He won't fix it? (Score:2, Insightful)
The only fix that kinda works is disabling hyperthreading. But on a dual core processor the problem is there as well (if there is a shared cache somewhere). And switching of the second core because of that would be stupid.
The problem Colin points out (getting an RSA key) is a userland problem anyway, so there is nothing to fix for Linus... fixing cache eviction covert channels in the kernel is possible, but not without losing a lot of performance.
--Blerik
This sort of attitude is pretty common (Score:5, Insightful)
As someone pointed out, yay for linux being free. As one or two above pointed out, someone who does care with the knowledge will write a patch. It'll get implemented as an option in the code, and if shown to be unobtrusive enough, may even get turned on by default.
Another Fairy Tale... (Score:4, Insightful)
Scene: A wispy cloud scuds across the sunny blue sky. Not much happening, and the cloud is hardly even black.
Chicken Little: The sky is falling! The Sky is falling!
The Penguin DictatorNo, not really. It might fall, but it's very, very unlikely. So calm down!
Chicken Little: I strongly disagree. The sky is falling! And because you do not understand the problem we're all going to die!
The Penguin Dictator:Listen here. It's almost certainly not going to fall, and I need to worry about real problems!
Chicken Little: (Runs screaming to the nearest coffeehouse with free wireless, where he types incessently:) The sky is falling! The Sky is falling! Tell Slashdot! Tell Tom's Hardware! Tell Cnet! Tell Linux Business News!
The Penguin Dictator: Sigh. (And then he gets back to work. He looks up at the audience) They just do not get it, do they?
The Windows Dark Lord: (Rubs hands together) Excellent, MOST excellent. (Yelling) Bring me my marketing minion!
Marketing Minion: (being drug in by a bald guy yelling at him) Yes, O Master!?
The Windows Dark Lord:Tell all the peasants that the sky is raining huge chunks of fire and dung! Tell everyone, tell them now! And have our independent consultants work on this day and night, night and day! Make sure that they independently tell everyone that they can easily avoid falminf chunks of sky dung if they stand behind our Windows! And RAISE the price!
Some Guy At Some House In Some City Somewhere: "Wow, that was easy. Let me send this up to the Penguin Dictator. No sky ever fell, and that cloud is easily blown away. Nothing happening here, move along."
The Penguin Dictator "Well that was easy. Include this patch in the next day's weather update!" Marketing Minion: Press Release!!! Millions killed by falling flaming sky chunks of burning dung with brain eating worms who eat children!!! Run for your lives!!!!
Laura Didio, munching a do-nut"If you would hide behind Windows, the sky would stop falling! Your children would be safer and the world a better place." (looks at stoick ticker, says to self) 'Excellent. MOST excellent. Bring me a donut!'
The Penguin Dictator "Sigh. Why didn't I just keep Sky 0.7a for myself? Why the bother, wy the bother?"
EPILOGUE: No one was ever hurt by the piece of sky that never fell, and Chicken Little kept looking upward for another cloud to rant about.
The End.
Re:At least Linus.... (Score:5, Insightful)
Great (Score:2, Insightful)
Yeah, just like a mouse driver having full access to kernel security structures and raw disk partitions, it doesn't seem all that worrying at all (except when your entire system crashes thanks to a buggy sound driver, or you get rooted, or...).
Not fixing this design mistake while laughing at respected experts in the field reminds me something [google.com]. I was hoping that we as a community might have became a little bit more mature during the last decade, but I seem to have been naïve again.
Re:At least Linus.... (Score:1, Insightful)
Re:He won't fix it? (Score:1, Insightful)
From the little bit I understand from the paper (which, when this story was first posted on Slashdot May 13th, wasn't publicly available), it's an extremely low level attack that depends on certain process switching back and forth, without emptying cache. (I think that's the base of it.)
From what I've learned in software writing, is that it's preferrable to wait and see how much and how bad your software runs or has problems before you start charging into the situation to fix it. Especially something as low level as this, which could have unseen side effects. Especially since this (to me, at least) seems to be more of a hardware problem than software, per se. (But, of course, I could be wrong.)
Its better to just fix it and be safe than wait and see if something happens later.
I would think that would only be adviseable only if you (internally) found the bug/security problem. I would put up a notice saying I've heard of this situation, and maybe even come up with an idea for the fix, but definitely wouldn't implement it until I could prove or see proof that said problem exists.
p.s. Microsoft's reaction is slightly different than what you describe. Microsoft didn't seem to care about bug fixes to IE, period, only fixing them when the griping got too loud and the public started paying more attention to Firefox. There was no motivation to fix IE, not just a 'wait and see' type attitude.
The man (Score:3, Insightful)
I'm sure that if an exploit is found, someone will have a patch ready for the next kernel revision - that's the beauty and advantage of Linux.
Re:Great (Score:3, Insightful)
I don't think Linus was "laughing at respected security experts" at all. For one thing this guy isn't a respected security expert - in my experience they usually have jobs. Secondly he wasn't laughing - his point was that the best way to fix the issue was to patch the userland crypto programs to be more immune to timing attacks e.g by making them read a set amount of cache lines each time. Anything that makes the crypto software more immune to these things seems good to me.
Also, how do you patch the kernel to stop this attack? The FreeBSD patch just disables hyperthreading.
I assume you can answer this since you sound like an expert. I also assume that you bothered to read the actual e-mails and not just the retarded Slashdot blurb.
But now I'm being naive aren't I?
Re:Slashdotters defending Linus... (Score:1, Insightful)
Re:bad tactics from Colin Percival (Score:4, Insightful)
The point of this debatte is that the Intel implementation sucks, it allows you to spy a lot on processes running on the virtual CPU. Sure, there are better alternatives than disabling HTT like the suggestions of Colin to only schedule threads of the same program on the virtual CPU. Actually, that is something you want to do anyway or otherwise you can seriously loose speed and drop under the performance of a processor with HTT disabled.
Speaking of paranoia, it is often not a bad thing to have, many big security problems can be avoided. Oh, I forget to patch the Linux box next door.
If security matters, don't do crypto in Linux (Score:5, Insightful)
... or in any other general-purpose operating system on a general-purpose computer. PCs are fundamentally insecure. There are a dozen ways to spy on cryptographic operations done in them, ranging from trojans, to hardware side-channel attacks, and dozens more to get copies of keys that they store. This is just one particular attack that may permit an attacker who can't get a trojan running with sufficient privileges to spy on operations directly to obtain some key bits. But if the attacker can't do that, there are lots of other ways to get the keys. General-purpose computers are simply not trustworthy.
If security is important, you do your crypto in a secure crypto module, like the FIPS 140-2 Level 4 IBM 4758 [ibm.com] or the Level 3 Luna SA [safenet-inc.com]. Or, you use a general-purpose computer with special-purpose, very simple software and then provide strict physical access control to the machine and very limited network access -- often through a serial link using a custom protocol rather than via a real network. Or you could theoretically use a general-purpose machine with a TCPA chip with a regular, general-purpose operating system that has been modified to make use of the TCPA chip and with keys tightly bound to a well-defined system software configuration. But only if you have good physical security. In many situations it's still better to use a FIPS 140-2 Level 3 or Level 4 device.
IMO, the existence of weaknesses like this in Linux, and the fact that they're widely known, is a *good* thing, because it helps convince people not to trust that which is inherently untrustworthy. We need more publicity of similar problems in Windows (and there are lots of them).
Re:bad tactics from Colin Percival (Score:3, Insightful)
Being able to read arbitrary memory from another process is a big security flaw, as illustrated by the Minesweeper Hacking [codeproject.com] sample. But for a kernel programmer it's a minor deal for server security as it needs a local/remote exploit to run code on your box. Even then it is a readonly exploit, which decreases exploitability unless we're talking about stuff like SSL certs or GPG keys - which are pretty hard to find in 1 Gb of data :)
So to really exploit this, your thread should be running the CPU practically or 100% CPU. That should be an easy enough warning signRe:Single Dictator? (Score:3, Insightful)
Most dictators have tweaked the kernels to suit their needs anyway, nothing stops them from implementing this in their own kernel version.
Re:Great (Score:3, Insightful)
Stop a mo and take a deep breath.
Take a look at your (heavily patched I'm sure) machine. If you had an unsupervised 24 hours with that box and an unprivileged account how many other actually useful exploits could you run for key retrieval.
I can think of about 5 methods right now that are much more likely to yield results within Linux. Even on a very up-to-date system with a decent admin.
Pfft.
- Storm in a teacup
Re:bad tactics from Colin Percival (Score:2, Insightful)
The problem is not with hyperthreading. It's not with Linux. It's with the implementation of the encryption algorithms. They need to stop assuming that table lookups are constant-time. Because they're not. As good as constant-time for most purposes, yes. But for cryptography they're not good enough.
Re:He won't fix it? (Score:5, Insightful)
Re:He won't fix it? (Score:3, Insightful)
Re:strcmp vulnerability. (Score:3, Insightful)
Re:bad tactics from Colin Percival (Score:2, Insightful)
for (i=0; isome_random_number; i++) { do_some_stuff_that_looks_like_crypto }
This seems like another pointless nerd debate. It sounds like that "hack a key by examining the frequencies the CPU emits" type stuff.
Re:If security matters, don't do crypto in Linux (Score:3, Insightful)
Re:Fixing is easier said than done (Score:3, Insightful)
Sounds like it doesn't fix the problem, it replaces it with another (reduced performance). That's not a *real* fix to any problem.
Compare: doing suicide to "fix" personal problems.
Re:He won't fix it? (Score:3, Insightful)
Fixing the crypto libraries is actually a non-trivial task. There are many of them and ensuring there is no information leakage is very difficult to achieve.
Fixing the chip may well be impossible -- it sounds to me like the only way to prevent this kind of thing from being a problem is to give each thread its own instruction fetch pipeline, dedicated execution units and dedicated cache lines. Which is, in effect, dropping HT altogether and switching to dual cores instead.
Re:Linus and RMS (Score:1, Insightful)
Re:He won't fix it? (Score:5, Insightful)
Other examples (Score:3, Insightful)
If a process is leaking information somewhere, then there will be people clever enough to pull that information out.
That said, it seems that this is more of a library problem than a Linux problem.
If you need any real security you should be doing your private-key operations in an HSM anyhow, not on your CPU.
Re:He won't fix it? (Score:5, Insightful)
Can I buy some pot from you?
Maybe that would work with a ONE MEGAHERTZ PROCESSOR. But do you have any idea how fast processors are these days? And how likely any deviance in the cache state, IO controller state, page faults, multi-user latency, or power management will throw your precious timings right out the window?!?!?!
I mean, c'mon, think about things before you say them. Even REAL TIME SYSTEMS AT NASA don't run with enough consistency to be able to tell WHICH CHARACTER IN A STRCMP OPERATIONS fails.
Re:This sort of attitude is pretty common (Score:5, Insightful)
That's really what's up for debate here. Whether the patch should be in the kernel-land or in the code user-space (mod_ssl, for your example).
The only realistic patch you could do in kernel-land is to simply disable HyperThreading. This works, but seems like a poor way to go. Any other form of patch in kernel-land just makes the attack harder and thus doesn't really work or it degrades performance way too much to be practical.
But fixing it in userspace is somewhat easier to do, albeit you'd have to fix *every* user-space program that's susceptible to this sort of thing.
Let's talk about the problem in general terms. When a program is doing some kind of computational stuff on something you want to remain secret, then it has to make some assumptions. Assumptions like the hardware is secure, or that it's not running on a virtual machine that's recording everything it does.. That sort of thing. You can come up with all kinds of ways to crack it like an egg if you work outside the box a bit and have total control of the machine it runs on.
This problem is attacking one of those assumptions, namely that another process can't time the secret computations accurately enough to perform a timing attack. With HT, you have two things running on the same core, and so it is somewhat easier to do this sort of attack.
So userspace programs that do secure computations have had one of their assumptions broken by HT. To remedy it, they need to rethink their assumptions. They need to or ensure that they perform equal timings regardless of the computations being done and so on. This is not particularly simple, but it's probably not particularly hard either.
Of course, the attack is still largely theoretical. All it's been shown is that it's "possible", not that it's "easy" or even that it is indeed "doable". For one thing, without having some kind of clue as to the algorithim involved or some idea of what to look for, all you get are a bunch of timings. You still need to do some things to trigger it at the right time and in the right way as to be able to derive information from this channel.
But crypto guys are paranoid like nobody else, and so they're naturally worried about this sort of thing. Mainly it's worrying to them because it's not a mathematical attack, which they're more used to. Modern crypto works based on theory and algorithims and such, and the idea that the algorithim being correct (for a given value of "correct") isn't enough to protect the security of the data is extremely worrying. A real world implementation of these algorithims now has to take some more real world facts into account, and this bothers them, of course.
Linus is basically right here. The kernel is simply the wrong place to fix this. It doesn't ensure that processes cannot spy on other processes via subchannels like this, nor should it. If you're paranoid enough to think this is a real thing to guard against, then your secure code should take it into account. Existing code doesn't do that, and would need to be changed *even* if the kernel was patched. Because how do you know that your kernel has been patched? How do you know that you're not running on an HT processor? You can't know for sure, so you simply assume you are and take steps to make timing attacks fail. Because if you don't, you can't reasonably say that you've attempted to secure the code in this way.
Not a kernel problem (Score:2, Insightful)
It's either an application problem or a hardware problem. Basically, used memory is not being zeroed out, so one programme can look at what another programme left behind.
In the case of a software frame buffer {like the 1980s home computers with bit-mapped graphics: Spectrum, BBC et al} failure to zero out memory causes spurious artifacts on the display. You can see this if you switch between graphics modes by writing to the hardware registers directly rather than using the "proper" system calls which clear the screen. {On the Beeb, you could actually grow a stack through the display memory. Pretty, but you'd better hope not to scroll the screen or print over that area.} The solution was implemented in software: create system calls that zero-out the display memory when switching graphics modes. {As a bonus, your users need only send one number to the routine, which can poke the right values into all the relevant registers on their behalf. They just don't get to invent their own graphics modes, but if you ever update the display ULA in future then at least you won't kill half the games software in existence.}
What we're talking about here is cache memory not being zeroed out between uses by successive processes. That looks to me very much like a hardware problem. It's not even an easy problem. My guess is that the implementors looked at it, decided "It's potentially insecure in theory but bloody difficult to make use of in practice", and left it that way on purpose. Like there's no point fitting an expensive lock on a wooden door with a person-sized glass panel in the middle of it -- especially if that door is only accessible through an overgrown garden with an underfed Alsatian in it.
BTW, crypto software running in userland could never, ever be made immune to snooping from kernel space -- at least, not on a system with any kind of debugging. The solution is to read and understand every bit of the kernel source -- including all drivers -- or get some independent expert to do so for you, so as to be sure the kernel contains nothing that could be used for malicious purposes. Hardware crypto devices would be more immune to tampering -- but less susceptible to independent verification.
Imagine this: <CHEESY MEXICAN ACCENT>Hey, extranjero! You want to send secret message? I chave code so secret, nobody onderstand eet 'cep' for me an' my brother. Djou dictates to me, one word a time, I write eet down in secret code. Then I send eet to my brother and che go to your amigo, and read heem the secret code. Nobody in world onderstand 'cep' my brother.</CHEESY MEXICAN ACCENT>
Vendor kernels contain patches (Score:3, Insightful)
This is one of the good aspects of open-source software: If you disagree, you can fork or simply distribute a patched product.
Re:This sort of attitude is pretty common (Score:3, Insightful)
It may be possible, yes. But plausible?
this is an issue that effects me and my customers
It MAY affect you and your customers at some later time, but right now it doesn't.
If you're THAT concerned about this issue, I assume you're going to call up your ISP and transition your site onto dedicated machines? Isn't it worth the extra cost to be assured that some other customer of the shared server environment can't compromise your crypto key?
Re:This sort of attitude is pretty common (Score:1, Insightful)
Re:Another Fairy Tale... (Score:4, Insightful)
1. Microsoft is just as potentially vulnerable as Linux. Their dirty laundry just doesn't get aired in public.
2. The fix is non-trivial and non-obvious. It's not a simple buffer overflow. Any patches are likely to have serious repercussions on kernel performance. e.g. disabled HT, ensure only two threads of the same U.I.D. are scheduled on the same processor, flush cache at every context switch, etc. It looks that Linus is unwilling to accept them unless this vulnerability moves more from the theoretical to proven.
Colin Percival only cryptographer in whole world (Score:3, Insightful)
It wouldn't be hard to have an option to prevent processes with different owners from running on the same physical CPU at the same time. It wouldn't even affect the case that Linus mentioned. But cryptographers don't seem to think it's a plausible attack anyway, aside from carefully arranged conditions. The discussion was entirely over whether it would be less foolish to prevent it in the kernel or in userspace, and nobody seems to have argued that anything should be done at all.
Bzzt! Back the paradigm up here. (Score:5, Insightful)
Potential performance problems are things you should defer on until proper profiling can be done (unless they're total show stoppers). Security and correctness are things you cannot ignore except in extreme cases. Security is particularly important to nail down, because it can result in your customers losing data (even data not pertaining to your app), which is the first no-no of software.
Application software has four priorities, in this order:
- Safety (shouldn't destroy data)
- Correctness (do what it says it does)
- Security (don't do anything else)
- Performance (do it fast)
YMMV, of course, sometimes correctness falls below security, and occasionally performance goes above correctness in some mathmatical functions (if doing it correctly would take a decade and doing a close approximation would take a day, obviously you want the approximation and then a heuristic). In this case, I'd say proper fix is to disable hyperthreading by default, and make sure the user is aware of the hardware bug/consequence of using HT when they decide to turn it on. You need to let the user decide if they're willing to accept the security risk or not.The Linux Kernel Developers may decide otherwise, but that's how I'd call it if it was in my shop. It's a hardware problem and the software fix is not obvious.
Re:He won't fix it? (Score:1, Insightful)
No it will not. This CANNOT be fixed. It has nothing to do with privilege escalation, and everything to do with information "leakage" from observable effects. Intel can no more change these observable effects than they can make a CPU that can't be instrumented with performance counters.
And guess what: linus is right. Linux is not the place to address such issues, because this is only ONE such place where such leakage can occur. Linux is not a military-grade computing system, and if you need such security, you need a dedicated system that NEVER pre-empts a higher-security process to allow a lower-security process to run. The attack is merely harder, not impossible on truly multi-CPU machines.
Re:If security matters, don't do crypto in Linux (Score:4, Insightful)
Specialist crypto hardware is too expensive for most users
If your secrets are worth enough that an attacker would go to the effort required to use this exploit... no, it is not too expensive. An IBM 4758 costs just over $1000 and provides an extremely high level of security.
If you want to go really cheap, and don't need high performance, you can also use a $10 smart card with a $10 smart card reader. The result is less secure than with a 4758, or a Luna SA, or any of the other products on the market, in several ways, but it's much, much better than doing the crypto in main memory on a GP CPU.
and keeping a PC in a physically secure environment can be difficult and contradictory with other important considerations.
Again, if you need security you have to do what it takes to get security. If security requirements contradict other requirements, you have a problem to solve.
At bottom, the point is that this is a sophisticated attack. There are much easier ways to dig keys out of a GP computer. If you need to defend against this attack, then you also need to defend against all of those, and defending against all of those requires physical security and use of secure crypto hardware. By the time you've closed all the other avenues of attack, you've closed this one, too.
This problem is real, and it's a good idea to fix it, on principle. But it's not uncommon that real security holes are simply irrelevant because of other considerations.
This is analogous to someone pointing out that my house is insecure because the attic vents are made of lightweight sheet metal. An attacker can use a ladder to climb up, use a crowbar to rip the vent off, climb into my attic, find the access panel and drop down into my house. That's all very true. However, compared to all of the *other* ways there are to break into my house, it's simply irrelevant. My house is simply not an appropriate place to store the Hope diamond, and replacing the attic vents with heavy steel ones, or otherwise securing access to the attic, isn't going to change that.
Re:He won't fix it? (Score:2, Insightful)
No. It requires having shell access. This is not the same as physical access, and a system should be secure against an attacker with shell access. Not that most are in fact secure, but they should be.
Re:probably easy to fix (Score:3, Insightful)