Linux Has Fewer Bugs Than Rivals

sushant_bhatia_progr writes "Wired has an article stating that according to a four-year analysis of the 5.7 million lines of Linux source code conducted by five Stanford University computer science researchers, the Linux kernel programming code is better and more secure than the programming code of most proprietary software. The report, set to be released on Tuesday, states that the 2.6 Linux production kernel, shipped with software from Red Hat, Novell and other major Linux software vendors, contains 985 bugs in 5.7 million lines of code, well below the industry average for commercial enterprise software. Windows XP, by comparison, contains about 40 million lines of code, with new bugs found on a frequent basis. Commercial software typically has 20 to 30 bugs for every 1,000 lines of code, according to Carnegie Mellon University's CyLab Sustainable Computing Consortium. This would be equivalent to 114,000 to 171,000 bugs in 5.7 million lines of code."

What about the ones they missed?

[Ironsides]

Not to be a downer but, how do we know they didn't miss anything? 5.7 million lines is a lot of code to go through and analyze. I'm also curious where they came up with the 20-30 bugs per thousand lines of code that proprietary software suffers from since they can't see the code.

Of course, we must remember, "It's not a bug, it's a feature!"

Statistics

[amigoro]

• Typical Commercial Software: 20 to 30 bugs for every 1,000 lines of code (Kloc)
• Linux kernel : 0.17 bugs per Kloc
• Windows XP: 40-50 bugs per Kloc Source [neu.edu]

Moderate this comment
Negative: Offtopic [mithuro.com] Flamebait [mithuro.com] Troll [mithuro.com] Redundant [mithuro.com]

Positive:Insightful [mithuro.com] Interesting [mithuro.com] Informative [mithuro.com] Funny [mithuro.com]

Re:How can one be sure

[Deviate_X]

Actually "Windows XP" isn't a Kernel. The kernel of Windows XP is called the actually called the "NT Executive" - which is composed of the Hal (Hardware abstractiomn..), Microkernel and kernel services ( device drivers,.. ).

Windows XP Architecture [senecac.on.ca]

Re:Linux Kernel vs Windows XP

[Realistic_Dragon]

Crashing KDE won't kill your system. System services will keep on rolling just fine.

Crashing the Windows shell will nuke the whole box, web servers, ftp servers, application servers and all.

Obviously the distinction for a desktop user is minor, since your desktop is gone and your work with it, but if you are running servers the separation is VERY important. A KDE crash (unlike a hard lock from windows shell bringing down the system) doesnt lead to:

Service unavailability for customers.
Possible disk corruption (disk writes not completed).
Potentially having to rebuild a volume from raid or journal.
Loss of state based data (eg from a web app).

And so on. Add to this that users for whom stability is critical won't even be running a desktop on a Linux system and you can see why this is a real world metric for users who aren't relying on all of the extra crap that they cannot disable in the NT kernel.

new vulnerabilities + exploits

[NoSuchGuy]

Here are some new vulnerabilities:

Linux kernel IGMP vulnerabilities von Paul Starzetz [isec.pl]

Linux kernel scm_send local DoS von Paul Starzetz [isec.pl]

Re:Congratulations...

[chialea]

I'm not a programmer-type either, but I'm familiar with some of this research. There are a few techniques that I've seen, but don't consider this to be complete, my research is in crypto, not code.

1. code patterns -- if you see something that looks like a pattern, it is probably a bug... "if(x = 0)", for example. of course, you have to check that it actually IS a bug, but you can catch certain common things that way.

2. type safety -- tools can go through your code (either statically or while it's running) and look for type violations. for example, you might write an int to an unsigned int, or mix up pointers and ints, which could be bad. you can catch a stunning number of bugs this way.

3. pointer analysis -- another annoying bug can be in aliasing, where you have multiple pointers that may or may not be pointing at the same memory. are you really /trying/ to overwrite that data structure pointed to by another pointer? In general, this sort of analysis is hard (undecidable, off of the top of my head), but feasible in limited cases.

I'm not sure what sorts of current tools are released by these researchers, but this is a very basic overview of the techniques I've heard about people using recently. (Repeat disclaimer: I'm a theorist.)

Lea

Re:Mistake

[FrYGuY101]

Okay, why?

Those are integrated into the Operating System. They are not part of the kernel.

Re:Linux Kernel vs Windows XP

[beuges]

the windows shell is explorer.exe, microsoft's equivalent to kde. if explorer.exe crashes, web servers, ftp servers, database servers, and anything else running on the box, even user applications that are not tied in to the shell, will continue to run unhindered. so, to respond to your first statement, crashing explorer.exe wont kill your system (do it yourself on windows xp, go to task manager and kill the explorer process). system services will keep on rolling just fine. contrary to /. belief, the windows shell is not "tied in" to the windows kernel any more than kde is tied into bzImage.

Re:Congratulations...

[Khazunga]

I've not RTFA, so this may not be true for this particular case. However, the bug figure is usually derived statistically. You assume bug finding follows an inverse exponential (picture a capacitor charging function). After some time, from the found bug data, you can deduct the function limit when time is infinite. That limit is the total bug count.

Re:Now tell us what the bugs are

[1u3hr]

RTFA: they were working on the code from 2000; took them four years to analyse it. So the code they were looking at has been patched in current versions: "Seth Hallem, CEO of Coverity, a provider of source-code analysis, noted that the majority of the bugs documented in the study have already been fixed by members of the open-source development community."

Re:Congratulations...

[Octagon Most]

"I doubt they looked up all the code. They probably only made statistics to compare the amount of bugs based on what has been reported and archives."

Exactly. The article is nearly useless. According to the CNet article covering the same issue:
"Proprietary software, in general, has 1 to 7 flaws per thousand lines of code, according to an April report from the National Cybersecurity Partnership's Working Group on the Software Lifecycle, which cited an analysis of development methods by the Software Engineering Institute at Carnegie Mellon University."

The Wired article says,
"... 20 to 30 bugs for every 1,000 lines of code, according to Carnegie Mellon University's CyLab Sustainable Computing Consortium."

Hmmm, both cite Carnegie Mellon University as the source. So which is it, one to seven, or 20 to 30? That's a big difference. It's either 5,700 to 40,000 flaws or 114,000 to 171,000.

The bottom line is that the Linux source code can be viewed and has 985 visible bugs of various identifiable types. The Windows source code cannot be viewed and may have anywhere between 5,700 and 171,000 flaws based on some questionable extrapolation using two widely divergent methods.

Re:Linux Kernel vs Windows XP

[Anonymous Coward]

Yes and no. Theres different meanings of the word "shell" here. Explorer.exe is *just* a shell - it's basically a file browser, of which the desktop is a special case. Explorer.exe crashing won't even disable your applications - it's just another application.

So yes, the GP was incorrect in saying that the "shell" crashing will crash the machine. However, other parts of Windows that correspond to KDE, like the window manager, can crash the machine when they go down, because they run in the kernel. Not all of them, though, because some of them run as services, like the theme engine.

Basically, the architectures are sufficently different that it's pretty hard to draw an simple comparison between them.

Re:Now tell us what the bugs are

[mahdi13]

How could they analyse the 2.6 kernel 4 years ago when 2.6 is less then a year old?
ReadingTFA: "The Linux source-code analysis project started in 2000"...

This is an ongoing study, to find the bugs checkout Bugzilla.kernel.org [kernel.org] it's not like they hide them or anything

Re:20-30 bugs per 1000 lines???

[jeremyp]

Bzzzt wrong

jeremyp@dhcp-2-1-56:jeremyp$ cat foo.c
#include <stdio.h>

int main ()
{
if (0)
return ;
printf ("Hello World\n") ;
}
jeremyp@dhcp-2-1-56:jeremyp$ gcc foo.c -o foo
jeremyp@dhcp-2-1-56:jeremyp$ ./foo
Hello World
jeremyp@dhcp-2-1-56:jeremyp$ gcc --version
gcc (GCC) 3.3 20030304 (Apple Computer, Inc. build 1666)
Copyright (C) 2002 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Re:Kernel is not the problem

[tclark]

When I upgrade a kernel I don't want to recompile my Nvidia driver. It should just _work_.

You might want to talk to Nvidia about that. They are able to produce a driver that does this, but they choose not to.

Re:Congratulations...

[chgros]

A very common bug in the kernel:

p = malloc();
if(!p)return;
q = malloc();
if(!q)return; <= Memory leak if this happens

Re:Is anyone else bothered by this?

[JudicatorX]

Why does "Linux is just a kernel" suddenly not apply here?

Linux is a kernel. The problem is that this is comparing the linux kernel to all of windows XP

Re:Apple != Orange

[truthsearch]

Start up time and rendering speeds are fast for IE.

And both can be achieved without OS integration. Rendering for any 3rd party app can be direct to the video driver if the OS allows it. That's not integration.

It's already been proven that startup time for all Office apps is from hidden API calls near the start the executable code. They load the visual interface before the application's actually ready for use. That plus pre-loading of DLLs gives fast startup. Office isn't considered part of the OS, yet IE is. Therefore fast startup times have nothing to do with integration.

Try supporting 5 different applications vs. one. Over the phone. With a user with no training or previous knowledge.

I have. An entire office of old-fashioned accountants who prefer ledgers and pencils. How is blending 2 apps tightly together better than having 2 separate apps? If there's a problem with Firefox I can tell someone to not launch it. If there's a problem with IE parts of it are in memory whether I choose them to be or not. If there was less integration in Windows then it could be trimmed down to a minimal size for each user. Instead everything including the kitchen sink must continually be supported. You're only increasing your headache by using Windows and its tight integration.

I'm questioning the statistic mentioned is valid or not. Can this number even be trusted?

Not as purely fact. Yet someone who reads the study may determine that it's better to have the code open to all who can fix bugs instead of one select group. Or it may give insight to management that security can be better achieved when they can have their own people analyze the code. When read properly I don't see how anything but good can come from a study such as this.