IE Shines On Broken Code

mschaef writes "While reading Larry Osterman'a blog (He's a long time Microsoftie, having worked on products dating back to DOS 4.0), I ran across this BugTraq entry on web browser security. Basically, the story is that Michael Zalewski started feeding randomly malformed HTML into Microsoft Internet Explorer, Mozilla, Opera, Lynx, and Links and watching what happened. Bottom line: 'All browsers but Microsoft Internet Explorer kept crashing on a regular basis due to NULL pointer references, memory corruption, buffer overflows, sometimes memory exhaustion; taking several minutes on average to encounter a tag they couldn't parse.' If you want to try this at home, he's also provided the tools he used in the BugTraq entry."

An important security sidenote

Since it may not be obvious to all readers, be aware that when you can make a program crash by feeding it bad data, you can typically further manipulate the data you are sending it to take control of the program. That means a security hole. This is how buffer-overruns work. You can't always do it, but you can think of each way you can crash a program as a little "crack" in its exterior. If you can figure out a way to pry apart the crack, you've got yourself a hole.

So many of these "bugs" in Mozilla, Opera, Lynx, and Links are likely security holes as well.

It is interesting, then, to see that Internet Explorer did so well on this, with its notoriously bad history [trustworthycomputing.com] on security. My first instinct would be that the HTML parsing engine in Internet Explorer was written by a different team of programmers than worked on the rest of the software, and they used proper programming techniques (such as RAII [google.com] in C++, or perhaps used one of their .NET languages, rather than programming in straight C like the others) which as a side effect prevented such problems.

Let's hope that all these bugs are taken care of in the other browsers quickly before the black hats find ways to make use of them.

Re:So what is "random" here?

Thats the thing about randomness. You can never be sure.

This "random" test is dangerously incomplete.

It mentions there is no scripting or stylesheets in this test--just random HTML tag soup. So yes, there was a very defined, arbirtrary range established. The results are disappointing for IE competitors for sure, but given IE is at major release SIX and at the end of its life cycle, while the likes of Thunderbird are for less mature (barely at 1.0 and at the early stages of its life cycle) I would expect and demand no less than perfection from IE.

Given the arbitrary limits on this test, it appears to be designed specifically to make IE look better than its competitors and prove some point rather than be an objective investigation. It is well known that the most serious problems in IE are with scripting and CSS support being unstable, broken or incomplete. A similar test should be conducted of IE should be done with these included. Kudos to the author of the bugtraq entry for doing this kind of testing, but I don't think the editorial commentary regarding the amount of testing of these browsers or their attention to security is warranted or productive.

The author freely admits he did not seriously analyse the source code for the root cause of these crashes (and in the case of IE, he cannot do so even if he wanted to--but that doesn't stop him from proclaiming it as superior quality). He also provides no evidence that these bugs compromise security in any way beyond consuming system resources, so it was not exactly appropriate to attack their security abilities without further study.

As to the jibe about lack of testing...Many of these alternatives are open source projects, not yet at official 1.0 release yet people! Being open source, the whole point of exposure is to get many eyes looking at the code, and get people involved in improving the code. He seems to know a great deal about programming so I suggest he volunteer some of his spare time to the Mozilla project to make things right, if he is indeed THAT concerned about the issue.

Re:An important security sidenote

"My guess is this was recompiled with the new SP2 compilers?"

My understanding of the SP2 compilation changes is that existing buffer-overflows still exist and will still cause the program to crash. The difference is that overflows which previously allowed the attacker to execute arbitrary machine code will instead crash before the code is executed.

Re:An important security sidenote

The difference is that overflows which previously allowed the attacker to execute arbitrary machine code will instead crash before the code is executed.

Almost, it's more like they will crash and there is a near zero chance of the code being executed even by another running process because the area has been flagged as non-executable and the cpu will refuse to run anything found in that memory space.

Re:An important security sidenote

The non-executable flagging (Data Execution Prevention in MS parlance) only applies when Windows is running on an architecture that supports it, which is pretty much only AMD64 at this point. They use stuff like stack canaries to protect x86, which makes an attack harder but not impossible.

Re:An important security sidenote

No, this is the stack canary in action. To emulate per-page NX on a processor without it, Windows would have to single-step all your programs, making it slower than VMware. (VMware doesn't even emulate at that level of detail.)

(Technically, it could get by without single-stepping: it could mark your NX pages no-read, then handle the page fault by checking the instruction at the fault address, emulating a MOV or similar instruction but killing the program on a RET or similar. However, that's horrendously slow, since each page fault involves two context switches (one into ring 0, one back to ring 3), which would easily slow your program by 100-fold. Your 3GHz computer would effectively max out at 300MHz.)

Re:An important security sidenote

I forgot to point out that you can prove this by compiling your program with an older or non-MS compiler. Write up a test C program, then compile it with Cygwin or MinGW GCC, and run it on an XP SP2 system running on a plain x86 processor. It should still overflow normally. Switch to Microsoft's compiler, and it should raise an error instead.

Re:An important security sidenote

A stack canary is a form of protection against stack overflows. And yes, the idea is named after the canaries used in coal mines. To put it in simple terms, a normal stack during a function call might look like this:

Buffer: XXXX XXXX XXXX XXXX ...
Saved registers: YYYY YYYY
Return address: ZZZZ

When the buffer is overflowed, the attacker fills it with more data than it can hold. The extra data first fills the saved registers, then overwrites the return address. The attacker can simply point the return address back into the buffer, or find more diabolical means ("return into libc", a few others), to run his own code.

If a recent OS (first Linux, now Windows) is running on, say, an AMD64 system, then the entire stack is flagged with the NX (no execute) bit. If the attacker uses the normal technique of returning into the buffer, the processor will halt the program because it's trying to treat data as code without asking first. (This doesn't protect against return into libc attacks.)

However, on ordinary x86 processors like Pentium 4 or Athlon XP, there is no NX bit. So, Microsoft altered their compiler to insert stack canaries into every function. The previous stack diagram is changed to something similar to this:

Buffer: XXXX XXXX XXXX XXXX ...
Canary: CCCC
Saved registers: YYYY YYYY
Return address: ZZZZ

Ideally, the canaries are chosen randomly each time a function is called. However, this is too slow in practice, since functions get called *a lot*, so a program will randomly choose a single canary number once at startup and reuse it.

Now the attacker can still overflow the buffer, but this time he has to overwrite the canary. If he already knows the canary, or guesses it correctly, everything works the same as in the case of an unprotected overflow. However, if he guesses wrong, the canary kicks in. To maintain the canary, there is some code inserted by the compiler at the start and end of every function. The start code inserts the canary into the stack, and the end code checks that the canary has not changed. If the canary changed, an error is triggered, and the program is halted before the function ever returns. This prevents the attacker's code from running if he doesn't know the canary number.

There are still some scenarios that aren't protected by a stack canary, but it is rather effective overall, and actually protects against a few scenarios that the NX bit doesn't cover. It doesn't help protect against heap overflows, though, although there's no reason heap canaries can't be used also. (The heap is a lot harder to explain than the stack, but a lot of programs put some or all of their buffers in heap memory instead, and the heap can be attacked as well.)

Re:An important security sidenote

Executed by another process? What are you talking about? Processes in windows cannot mess with each other address space.

They can intentionally, just not accidentially.

ReadProcessMemory
WriteProcessMemory
CreateRem oteThread

(NX bit works only at AMD64 processors and above last time I checked)

Celeron D is now shipping with NX enabled. I don't know whether XP will take advantage of it.

Re:An important security sidenote

I have a file sitting on my desktop here at work which says IE was still growing up in July of this year.

It was an 11 byte html file which made IE go BOOOOOOOOM. I aptly named it "crashme.htm".

It remains on my desktop as a reminder of MS crap :)

Re:An important security sidenote

I have a worse CD.. if you put it in the drive then it starts to install Windows 98 :(

Re:An important security sidenote

or perhaps used one of their .NET languages, rather than programming in straight C like the others

Not likely, since IE was created ages before .NET, and I don't quite think they decided to scrap and rewrite the entire parsing engine since then :)

As for the malformed HTML, it didn't crash my firefox, but I'll try again a couple of times just in case ;)

Re:An important security sidenote

Go the "gallery" he mentionds is his entry and try the mozilla_die?.htm files. With Firefox 1.0PR the first one did the trick for me and crashed firefox.

Re:An important security sidenote

Yep, the first mozilla_die entry crashes Mozilla 1.8a4 for me, too. Sounds like the tests are repeatable enough. Now quick, everybody rush to file bug reports and the winners can collect their $500 [mozilla.org]!

Re:An important security sidenote

Not likely, since IE was created ages before .NET, and I don't quite think they decided to scrap and rewrite the entire parsing engine since then

Indeed. It would be interesting to know how much of it is preserved from the pre-Microsoft Mosaic code...

Re:An important security sidenote

My first instinct would be that the HTML parsing engine in Internet Explorer was written by a different team of programmers than worked on the rest of the software

I's say the same about outlook express. Most security holes in OE were due to bad "glue" between components. And if I'm not mistaken, most holes in IE are also caused by bad integration.
It sure looks like the expert programmers create components which are then bolted together by an army of "learn programming in 24 hours" drones.

Re:An important security sidenote

I think you mis-estimate how hard it is to manage projects with the complexity of Internet Explorer. Even teams of really good developers with noe one "non-expert" can be brought down by the integration trap. It can probably all be led back to the Waterfall development paradigm where you do things in huge chunks: "Requirements, Design, Implement, Integrate, Pray, Test". Each of those is done as a discreet phase. Any devleopment process still following that basic model tends to fall apart somewhere around Integrate. Even with better development paradigms such as agile development there are considerable challenges in integrating something so large as IE.

But that *IS* the point of Agile development, to ensure that every step of the way things are working toghether smoothly. The basic point is regardless of the paradigm IE is a big project with many different components requiring a high degree of integration. A key problem with many different components that are highly integrated is the fact that these components tend to "trust" each other to much. Meaning they just assume this component is friendly. If all integrated components were a little less trusting I think software as large and as complex as IE could be more secure.

This is just a guess, I don't know much about internal Microsoft culture. I have however seen security problems of this scale in projects I have cleaned up and worked on and the problems stem from the exact problems I describe. So its reasonable to assume that somewhere along the way MS has made the same mistakes everyone else does in the software world. Just because they have LOTS of smart people doesn't mean they are any better at managing software processes. Just look at what they are doing with the LongHorn requirements :)

Jeremy

Re:An important security sidenote

That's certainly a good point (pre 2000).

The good news is that now people are required to know Writing Secure Code [microsoft.com], and (more recently) Threat Modelling [microsoft.com] by heart. I can tell you first hand those approaches have been adopted company wide. While Threat Modelling can be time consuming, I've personally found possible issues in code that we wouldn't have noticed without it. Plus we got other people outside our department looking at our code. All in all this is the best approach we could be taking. Microsoft is not sitting on it's ass about this issue.

Re:An important security sidenote

Your first instinct would be wrong, at least when it comes to it being built by a separate team. The fact is, as hard to believe at it is, for the past year Microsoft has put in place for every product systematic development techniques that directly target the security of an application (Threat Modeling, Secure coding techniques). Furthermore, this kind of test is standard within Microsoft (feed random inputs to all possible input locations). And once all the coding is done, the source still has to pass inspection through a security group within Microsoft! You can read about this stuff at the secure windows initiative [microsoft.com].

And this shift is working. The trend per-product is a significant reduction in security vulnerabilities. That is not to say there aren't any, that would be impossible, but if you look at the vulnerability graph for, say, Win2k Server since it's release, and win2k3 Server since it's release, there is a significant drop in the amount of vulnerabilities that are coming in since the release of the product. Furthermore, a large part of the vulnerabilities are found from within the company. The same thing can be said for most products, including IE, IIS, Office, etc... We're getting there....

Now, go off and run as LUA [asp.net], and nip this stupid spyware problem in the bud.

Re:An important security sidenote

This is a simple, nearly infallible rule of detecting exploits, to the point where I even know it. :)

If you can get a program to write past the end of it's allocated memory segment, you can overwrite all sorts of fun stuff with things like shellcode and anything else you want to throw in the executable stack.

The program (I read the SF post yesterday) generates standard things that would confuse a program in HTML - Null (ASCII 0) characters, overly large integers (Opera, IIRC, brought his system to a halt with a giant colspan="" element), things that need to be checked pre-emptively.

Regardless of his "bias", this is a problem. In fact, sometimes the people with the most to gain do a great job giving the others the opportunity to gain instead. Either way, he just upped the bar for browser security, which benefits us all.

Don't just blow him off.

Re:An important security sidenote

Actually, it is a large facet of security.

Are you familiar with XSS attacks? As a guy who writes web backends, I am. As a result, I have to make sure that every bit of content that comes to me and is subsequently displayed (which can get fun, especially if you have a database with 20M customers before you get started) needs to have no HTML tags, or even worse, allowable HTML tags. This can get very slow when processing a lot of content. If you have a templating language which uses different tag endings than an HTML tag, you've got another set of content to scan for. This is the reason things like mod_security were invented. Thing about a bulletin board or a "product review" system and how much content is availble to be sent straight to the database by one person and echoed right back to another.

SQL injection. While good database API's solve this, some systems don't (ahem... PHP's raw API). This is easily solved by something like DBI or PEAR's DB abstraction layer (which the name of escapes me), but once you're up to your knees in mud, it becomes a whole new nightmare. With the new mysql GRANT vulnerability (especially since, last I checked, mysql doesn't support binding at the client API level), SQL injection becomes something that can not only effect your live app, but something much more dire indeed. I won't even get into sql procedures that perform admin tasks.

The fact that IE passes a test, while other's don't, that it was made to pass, that says somethign positive about IE's security, and is not to be blown off. After all, I can inject some of that "wonderful" content right here and it might crash your browser, because there's nothing stopping me from doing it in slashdot's code. If I had the fingernail clipping of that guy's knowledge, I might be able to do something worse.

Of course, if you were running IE, you wouldn't have that problem. Do you understand now?

Re:An important security sidenote

"The fact that IE passes a test, while other's don't, that it was made to pass, that says somethign positive about IE's security, and is not to be blown off."

No, I disagree with that. It is reasonable to blow off that IE passes its own test cases. What is not reasonable is to blow off that other browsers do not.

IE still includes some basic security flaws due to faulty design. For example, there is phishing attack that displays http://www.bankname.com/ on mouseover but goes to http://ip.nu.mb.er on click. This is a security flaw in IE that should not exist (the same routine should be used to determine the URL for both mouseover and on click). Incidentally, this flaw does not exist in FireFox.

More relevant test cases are always good. New versions of Firefox, et. al. should be able to handle these test cases as well as those that they handle now that IE does not.

Re:An important security sidenote

I don't think that anybody is suggesting that IE is the best browser on the market. This guy has performed a useful service by devising a test script that highlights the flaws in other browsers and demonstrates that IE's HTML rendering engine doesn't have these same flaws. Nothing could be simpler. There is no need for all the open source advocates to run around flaming everything in sight, just because we now have an example that highlights a place where IE is better than our browsers of choice.

Pretty simple stuff, really.

Re:An important security sidenote

I don't see how this is a bad thing. This just means that IE does not catch some of the malformed code people use to cause havoc on the net.

But the other browsers not only didn't catch it, they actually crashed when parsing it. I'm all for compatibility and standards compliance where possible, but a crash/potential security hole is far more serious an issue than letting through some sloppy HTML. (Besides which, as a user, I find it infuriating that Mozilla/Firefox are so stuck up on perfectly standard HTML that they just don't work with some web sites that are perfectly usable in IE anyway.)

Re:An important security sidenote

I've come across plenty of sites that either don't work at all or are broken unless you use IE. Generally, it's because the site looks at the browser's identification tag and sends crippled pages to non-IE browsers. I can only think of one site I use regularly (a web app at work) that actually doesn't work in Opera if I set it to report itself as IE.

You might make sure that the sites you're having trouble with in Firefox are actually providing the same data they're giving IE before you assume it's a problem with the browser.

Re:An important security sidenote

Besides which, as a user, I find it infuriating that Mozilla/Firefox are so stuck up on perfectly standard HTML that they just don't work with some web sites that are perfectly usable in IE anyway.

As a user, I find it infuriating that people write non-standard compliant HTML that only works in one proprietary browser.

Re:An important security sidenote

Idiot. Crashing = denial of service attack.

*Your* first lesson in computer security is, and write this a thousand time: *crashing* on malicious code is *BAD*, whereas *recovering* from the situation and responding with an *error message* is *GOOD*.

Re:An important security sidenote

*crashing* on malicious code is *GOOD*, while *running* malicious code is *BAD*.

Holy crap! How absolutely untrue. If your program is crashing, you've lost all control. If you still had control, it wouldn't have crashed: it would have printed an error message.

Once you've lost control of your program, all bets are off. The only difference between crashing and taking control is exactly WHAT bad data you feed into the program. These browsers simply crashed because RANDOM data was being fed in. That random data could be changed to carefully-crafted executable code, and BAM, your harmless "crash" is a security exploit.

Re:An important security sidenote

You aren't a security expert, are you? Now, your first lesson in computer security is, write this a hundred times: *crashing* on malicious code is *GOOD*, while *running* malicious code is *BAD*.

It's true that *catching* bad input and deliberately aborting (hopefully with a somewhat reasonable error message) is good. According to the article, that's not what's going on... the browsers are NOT checking input, e.g. scanning into uninitialized buffer areas because they aren't finding an expected end marker, or a length is incorrect. So parent is exactly right... that kind of "buffer overrun" bug is exactly what can be exploited.

Be liberal in what you accept...

Jon Postel [postel.org] said it best: "Be liberal in what you accept, and conservative in what you send." A web browser that crashes due to invalid HTML fails this test. Execution must stop at once... yes, but the program should handle the error. A program that crashes is the Operating System handling a program with bug. See RFC 1122 [ietf.org] for more details about the Requirements for Internet Hosts. Section 1.2.2 about the Robustness Principle explains better than I can why you're wrong.

Re:An important security sidenote

I think it all depends on the definition of crash. A top-level thrown Exception is a good crash. A GPF is the kind of crash that might evolve onto a buffer overflow.

Re:An important security sidenote

A program must never crash because it received bad data. You always have to validate user input and there must always be sanity checks. If the browser receives malformed code, at worst it can give an error message, but it must never crash.

Crashes are always considered bugs.

Re:An important security sidenote

I don't see how this is a bad thing. This just means that IE does not catch some of the malformed code people use to cause havoc on the net.

Let's turn it around... if it was IE that was crashing on bad HTML, and the other browsers simply ignored it, would you be making the same argument? IMO, the slashdot headline would then be "IE Crashes on simple malformed HTML."

How is it a bad thing when other browsers refuses to read that code. Isn't that a good thing? A good example is a compiler most compilers catch overflows and don't allow you to finish compiling.

NO, no, no, no!! It is a BAD thing, because at the very minimum it's a sign of non-existent exception handling. You should never get a runtime error from bad input. In some cases, you create an infinite loop-- is there any excuse for that?
And considering the nature of the crashes (one of the links caused Firefox 1.0PR to die with a windows memory error, shutting down ALL instances of firefox) this means that some memory was accessed that shouldn't have been, which means that you could conceivably put executable code into memory simply by constructing the right "invalid" HTML. Lo and behold, you now have a buffer overflow exploit for Firefox. And we're telling all the IE users on Windows to switch to Firefox!

I'm a firefox user, and there's no way I'm switching back to IE, but this MUST be fixed. Now that it's well known, I'm sure there will be a patch for Firefox fairly soon, though I have a feeling the code changes will be somewhat involved.