Slashdot Log In
DieHard, the Software
Posted by
ScuttleMonkey
on Mon Jan 01, 2007 08:24 PM
from the yippie-ki-yay-and-other-flying-quotes dept.
from the yippie-ki-yay-and-other-flying-quotes dept.
Roland Piquepaille writes "No, it's not another movie sequel. DieHard is a piece of software which helps programs to run correctly and protects them from a range of security vulnerabilities. It has been developed by computer scientists from the University of Massachusetts Amherst — and Microsoft. DieHard prevents crashes and hacker attacks by focusing on memory. Our computers have thousands times more memory than 20 years ago. Still, programmers are privileging speed and efficiency over security, which leads to the famous "buffer overflows" which are exploited by hackers."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Vista already doing some of this (Score:5, Informative)
Re:Vista already doing some of this (Score:5, Interesting)
Parent
Re: (Score:2)
Re: (Score:2)
Is this feature standard in Linux yet? I'd hate to see us OSS guys get shown up by Bill...
Re:Vista already doing some of this (Score:5, Informative)
Parent
Re:Vista already doing some of this (Score:5, Informative)
DieHard's randomization is very different from what OpenBSD does, not to mention Vista's address-space randomization. I've added a note to the FAQs that explains the difference in some detail, and answers several other questions, but in short: "address-space randomization" randomizes the base address of the heap and also mmapped-chunks of memory, leaving the relative position of objects intact. By contrast, DieHard randomizes the location of every single object across the entire heap. It also goes further in that it prevents a wide range of memory errors automatically, like double frees and illegal frees, and effectively eliminates heap corruption.
-- Emery Berger
Parent
Re:Vista already doing some of this (Score:5, Informative)
http://kerneltrap.org/node/5584 [kerneltrap.org]
Any thoughts?
Parent
Re:Vista already doing some of this (Score:5, Informative)
Here's a more detailed answer -- I'll add it to the FAQ.
OpenBSD (a variant of PHKmalloc) does some of what DieHard's allocator does, but DieHard does much more. On the security side, DieHard adds much more "entropy"; on the reliability side, it mathematically reduces the risk that a programmer bug will have any impact on program execution.
OpenBSD randomly locates pages of memory and allocates small objects from these pages. It improves security by avoiding the effect of certain errors. Like DieHard, it is resilient to double and invalid frees. It places guard pages around large chunks and frees such large chunks back to the OS (causing later references through dangling pointers to fail unless the chunk is reused). It attempts to block some buffer overflows by using page protection. Finally, it shuffles some allocated objects around on a page, randomizing their location within a page.
DieHard goes much further. First, it completely segregates heap metadata from the heap, making heap corruption (and hijack attacks) nearly impossible. On OpenBSD, a large-enough underflow on OpenBSD can overwrite the page directory or local page info struct (at the beginning of each page), hijacking the allocator. This presentation [ruxcon.org.au] describes several ways OpenBSD's allocator can be attacked. By contrast, none of DieHard's metadata is located in the allocated object space.
Second, DieHard randomizes the placement of objects across the entire heap. This has numerous advantages. On the security side, it makes brute-force attempts to locate adjacent objects nearly impossible -- in OpenBSD, knowing the allocation sequence determines which pages objects will land on (see the presentation pointed to above).
DieHard's complete randomization is key to provably avoiding a range of errors with high probability. It reduces the worst-case odds that a buffer overflow has any impact to 50%. The actual likelihood is even lower when the heap is not full. DieHard also avoids dangling pointer errors with very high probability (e.g., 99.999%), making it nearly impervious to such mistakes. You can read our PLDI paper for more details and formulae.
-- Emery Berger
Parent
Re: (Score:3, Informative)
Re:Vista already doing some of this (Score:4, Insightful)
Parent
Re:Vista already doing some of this (Score:5, Interesting)
Part of the other problem is that most home users expect secure data, but they aren't willing to do anything about it (e.g. set up non-admin users, install virus checkers/firewalls/etc).
Parent
Re:Vista already doing some of this (Score:5, Informative)
The real solution is programming in a language with secure memory management, such as
Parent
Re: (Score:2)
or randomly locate virtual memory around the HD without regard to pre-existing magnetic conditions.
;-)
Different program? (Score:2, Informative)
Re:Different program? (Score:5, Funny)
Parent
Re: (Score:3, Funny)
Re: (Score:2)
So it is. Speaking of which, does anyone here know how to interpret the numbers it generates? I ran it on the deadbeef random number generator [inglorion.net] a while ago (test results linked from that page), and my interpretation is that deadbeef_rand does well on some tests and very poorly on others. Am I right? Can one distill from DieHard's output what the weaknesses of the PRNG are?
Re:Different program? (Score:5, Funny)
Parent
Re:Different program? (Score:5, Funny)
Parent
Correction (Score:5, Insightful)
Speed and efficiency of *development*, maybe.
Which is the problem. Modern software is so dependent on toolkits and compiler optimizations and various other "pre-made" pieces, that any program of even moderate complexity is doing things that the programmer isn't really aware of.
Re:Correction (Score:5, Insightful)
Parent
nothing to do with VMs - just exception handling (Score:3, Informative)
Re:Correction (Score:5, Insightful)
No, it was right the first time. Java is several orders of magnitude more secure by default than any random C or C++ program. Yet mention Java on a forum like, say, Slashdot, and you'll hear no end to how much Java sucks because "it's slow". (Usually ignoring the massive speedups that have happened since they last tried it 1996.) It doesn't matter that the tradeoff for that speed is flexibility, security, and portability. They want things to be fast for some undefined quantity of fast.
In fact, I predict that someone will be along to argue just how slow Java is in 3... 2... 1...
Parent
Re:Correction (Score:4, Interesting)
I've got to call you on the "portability" crap.
Java is about as portable as Flash... Sure, the major platforms are supported, but that's it. 3rd parties spent a lot of time trying to impliment java, but never did get everything 100%. Licensing issues, above all else, made it a real hassle to get Java on platforms like FreeBSD.
Meanwhile, C and C++ compiler are installed in the base system by default.
The only "portability" advantage Java has is perhaps in GUI apps, and that's at the expense of a program that doesn't look or work remotely similar to any other app on the system...
There are a great many reasons people don't use java. Performance is only a minor one.
Parent
Re:Correction (Score:5, Insightful)
The basic complaints I have heard are these:
Complaint 1: Java is slow.
As you stated, this is not a meaningful complaint.
Complaint 2: Garbage Collection stinks
GC is an obvious requirement of a "safe" language. As implemented in Java, it is downright stupid. When doing something CPU intensive, the GC never runs, leading to gobbling up memory until there is no more and thrashing to death. I'm sure that somebody is going to dig up that paging-free GC paper, but pay attention: that is a kernel-level GC.
Complaint 3: Swing is ugly/leaks memory
The first is a matter of opinion. The second is well-known. Swing keeps references to long-dead components hidden in internal collections leading to massive memory leaks. These memory leaks can be propagated to the parent application if it is also written in Java.
Complaint 4: Bad build system
Java cannot do incremental builds if class files have circular references. In a small project of about ten classes I was working on, the only way to build it was "rm *.class ; javac *.java"
Complaint 5: Tied class hierarchy to filesystem hierarchy
This was just stupid and interacts badly with Windows (and anything else with a case insensitive filesystem). It is even worse for someone who is first learning the language. It also makes renaming classes have a very bad effect on source control.
Complaint 6: Lack of C++ templates
C++ has some of its own faults. Fortunately its template system can be leveraged to fix quite a few of them. Java's generics have insufficient power to do the same thing.
Complaint 7: Lack of unsigned integer
These are oh-so-necessary when doing all kinds of things with binary formats. Too bad Java and all its descendents don't have them.
Complaint 8: Verbosity without a point
It has gotten so bad in places that I am strongly tempted to pass Java through the C preprocessor first, but I can't do that very well because of 4.
Parent
Corrections (Score:3, Informative)
Swing does not really have the problems you speak of any longer, if you are using it right... heck, it didn't really have those problem to any great degree about seven years ago when I was building a large custom client app all in swing for only desktop deployment.
Complaining about the build sys
Re:Correction (Score:4, Insightful)
Maybe, but most programs are not written in a way which will achieve this goal.
Programmer time is a limited resource. This is true even on a hobby project with no deadlines and everybody working for free; you want to ship sometime. Making programs run fast takes a lot of programmer time, even when you use a language which is supposedly fast by default such as C or C++.
C and C++ make you spend a lot of time working around weaknesses in the language and fixing bugs that other languages can never have. A great deal of programmer time is put into developing the broken and slow implementation of half of Common Lisp that every sufficiently complex program must contain.
All of this time spent is time that does not go into making the program fast.
By using a language that makes programmers more productive, you get a lot more time to make the program fast. You can do this by optimizing in the "slow" language you started with, by rewriting inner loops in C, by changing the whole algorithm to run on the GPU, etc.
The 90/10 rule says that your program spends 90% of its time in only 10% of its code, and that optimizing the other 90% of the code is basically a waste. And yet people who want their programs to "go fast" are writing that 90% in a low-level language, effectively wasting a large amount of effort.
You may also end up getting your program working, realize that it actually is fast enough despite being written in a really slow interpreted language, and spend the time you saved making more cool software. Or you can go back and make the original product fast. It's up to you.
There are many good reasons to use C, and many good reasons to write entire programs in C, but "it's fast" is not a particularly good reason. An app written in pure C is probably not as fast as it can be unless its scope is very limited.
Parent
Re:Correction (Score:4, Interesting)
1. The fastest programs are written in C/C++
2. On average, Lisp/Scheme programs are fasted, followed by C/C++ programs, and Java programs are way behind.
3. Development time is shortest for Lisp/Scheme, with Java and C++ being more or less equal.
4. C/C++ programs used the least memory, with Lisp/Scheme and Java being about equal.
5. There was very little variation in the run time and development time of Lisp/Scheme programs, and a lot of variation everywhere else.
The PDF contains some nice graphs illustrating all this.
Parent
Re:Correction (Score:4, Insightful)
Business software isn't the problem. The software that is the problem is the software that runs on every naive home user's PC
Berger sounds like a VM-language bigot (or paid ($30K from MS)
who doesn't understand how most software is really made, and prefers to believe in caricatures of programmers.
Great, you've called the guy a bigot, a shill, and an idiot, without even having understood what he was talking about.
Parent
Re: (Score:2)
I wondered where all those vulnerabilities were coming from. It's not humans misusing memory references and overrunning ad hoc fixed length buffers, etc. It's the toolkits, libraries and compilers! Glad we got that figured out.
From the post:
Our computers have thousands times more memory than 20 years ago. Still, programmers are privileging speed and efficiency
Re: (Score:3, Insightful)
This implies is that because memory is larger less attention can be paid to efficiency, but the hapless programmers don't know better. I used to use quicksort when I had 640 KiB of RAM, but now that I have 8 GiB, I'll just use bubble sort. Brilliant.
You are really misrepresenting his point here. We both know that bubble sort would run much slower on a 8Gb dataset than quicksort. The real comparison is "should we some really tricky and nasty code for this particular function or should it be a giant lookup table?" When memory is (relatively) cheaper than processor time, the set of tradeoffs changes. Some of these tradeoffs then mean than code can be written more correctly (securely) at the expense of higher memory usage. These tradeoffs are intuitively
Um, no. (Score:2)
The fact that Microsoft doesn't HAVE a security model, IE/Outlook are jokes, and users run as admin has a bit more to do with it.
Re: (Score:2)
And it is almost as easy to exploit. Intead of overflowing to the return address, you overflow
to the nearest vptr (if C++ is being used), to the nearest function pointer or to the nearest green bit.
Not quite (Score:2)
Let's solve voting security. (Score:3)
Algorithms demand perfection (Score:2)
Write the algorithms correctly and there won't BE any buffer-overflows.
What's so hard about this?
You must remember... (Score:5, Insightful)
Even assuming nobody wants to go to all that trouble, there are solutions. ElectricFence and dmalloc are hardly new and far from obscure. If a developer can't be bothered to link against a debugging malloc before testing then you can't expect their software to be immune to such absurd defects. A few runs whilst using memprof isn't a bad idea, either.
This assumes you're using a language like C, which is not a trivial language to write correct software in. For many programs, you are better off with a language like Occam (provided for Unix/Linux/Windows via KROC) where the combination of language and compiler heavily limits the errors you can introduce. Yes, languages this strict are a pain to write in, but the increase in the initial pain is vastly outweighed by the incredible reduction in agony when debugging - if there's any debugging at all.
I do not expect anyone to re-write glibc in Occam or any other nearly bug-proof language. It would be helpful, but it's not going to happen.
Parent
Re: (Score:2)
What's so hard about this?
The "write the algorithms correctly" part. The demand for programs is much larger than the supply of sufficiently trained/disciplined/talented programmers. Therefore, we need a solution that gives acceptable results even when the programmer isn't a guru (and preferably when the programmer is a trained monkey, because he often will be)
Wouldn't a language do the same job? (Score:2)
Note: I'm sure other functional lang
Re: (Score:2)
Yes.
But it's not a practical solution for about 185 different reasons starting with the fact that very few commercial apps are written in any kind of dynamic language, let alone LISP, and they're not likely to be rewritten anytime soon for such an intangible reason as security. rpg was right that worse is better, and the last language will be C. he wrote that before java, ruby, etc., but I think it's still right. Like it or hate it.
Re: (Score:2)
(defun bar (array i x)
"Set the Ith element of array ARRAY to X"
(declare (type fixnum i)
(type (
Buggy (Score:3, Interesting)
Ren and Stimpy said it best (Score:2)
Lots of memory available? (Score:5, Interesting)
In reading this article, I started to wonder a lot about this. writing to conserve memory is a bad thing? I will say that I haven't noticed that in most software, regardless of whether it's OSS or closed-source. If anything, there seems to be a variation of Parkinson's Law in effect. Yes, computers these days have a lot more memory available, however, the number of applications and the size demands of each application has grown almost in lock-step with that. 15 or so years ago, yes, you had one OS and one application running - maybe, if you were lucky or were running TSR apps, two or three. These days, the OS takes up a hefty chunk, and it's not uncommon to see 8 or 9 (if not more) applications running at once. What they all seem to have in common is that they assume they have access to all the RAM, or as much of it as they can grab.
I have to wonder if he's actually looked at things these days. I don't see where programming (properly done) to conserve memory is a bad thing. If anything, it seems that few are actually doing it.
Re: (Score:2)
access to all the RAM, or as much of it as they can grab.
They don't assume "access to as much RAM as they can grab", they assume "access to as much RAM as they need". Given the presence of gigabyte RAM modules, virtual memory, and near-terabyte hard drives, this is usually a reasonable assumption.
I have to wonder if he's actually looked at things these days. I don't see where programming (properly done) to conserve memory is a bad thing. If
Randomness. Nooooo! (Score:3, Insightful)
The worst bugs are the ones that are hard to reproduce. In fact, when faced with a bug that's difficult to reproduce, I've been known to quip "yet another unintentional random number generator". The suggestion that they're going to apply a pseudo-fix that involves random allocations raises all kinds of red flags. I'd much rather have fine-grained control over which sections of code are allowed to access which sections of memory, and be able to track which sections of code are accessing a chunk of memory. I'd much rather have strict enforcement of a non-execute bit on memory that's only supposed to contain data (there is some support for this already). Introducing randomness into memory allocation? Worst. Idea. Ever. It's like throwing in the towel, and if they put that in at low levels in system libs and things like that, we're screwed in terms of every being able to *really* fix the problem. If their compiler is going to link against an allocator that has this capability, I hope they provide the ability to disable it.
Speed - buffer overflow? (Score:3, Interesting)
Re: (Score:2)
Re:What a load of shit. (Score:4, Interesting)
1) If there is a program crash, it may be possible to reproduce the bug on the same computer, but probably not on 2 different ones, such as the user's and the developer's.
2) It discourages programmers from good design and thorough testing by leading them to believe that bugs won't occur.
The claim for DieHard (from the whitepaper) is that it "tolerates memory errors and provides probabilistic memory safety". But bugs will still happen! I once added about 10 lines of code to log a bug our team was having a hard time tracking down. It turned out to have its own bug that would be hit if:
- Two threads were accessing the same buffer
AND
- One of them was swapped out during the execution of 3 machine instructions (out of about a million)
It took my moderately sized customer base 2 weeks to hit it. The only way to avoid memory errors is to make the code bulletproof, which means fixing it when bugs are reported.
Parent
Re: (Score:2)