Linux Has Fewer Bugs Than Rivals 626
sushant_bhatia_progr writes "Wired has an article stating that according to a four-year analysis of the 5.7 million lines of Linux source code conducted by five Stanford University computer science researchers, the Linux kernel programming code is better and more secure than the programming code of most proprietary software. The report, set to be released on Tuesday, states that the 2.6 Linux production kernel, shipped with software from Red Hat, Novell and other major Linux software vendors, contains 985 bugs in 5.7 million lines of code, well below the industry average for commercial enterprise software. Windows XP, by comparison, contains about 40 million lines of code, with new bugs found on a frequent basis. Commercial software typically has 20 to 30 bugs for every 1,000 lines of code, according to Carnegie Mellon University's CyLab Sustainable Computing Consortium. This would be equivalent to 114,000 to 171,000 bugs in 5.7 million lines of code."
Make love, not war... (Score:2, Insightful)
better love (coding) your software, than making war selling it
How can one be sure (Score:5, Insightful)
Re:Congratulations... (Score:5, Insightful)
Seth Hallem, CEO of Coverity, a provider of source-code analysis, noted that the majority of the bugs documented in the study have already been fixed by members of the open-source development community.
Re:Mistake (Score:5, Insightful)
This just in! "Hello world" has 0 bugs per three lines of code! Most stable and secure software ever devised!
Now tell us what the bugs are (Score:1, Insightful)
Re:Mistake (Score:2, Insightful)
Deleting half the DLLs in sys32 then trying to run applications does not constutute a bug, especially when Windows shouts at you that you really don't want to be deleting them. If the user being able to cause problems was a bug (some would say it is), then Linux is more buggy than anything else. Windows has the decency to complain if you're deleting anything essential, Linux at best goes "Y/N", and even that can be overridden with a switch.
Lots of bugs maybe, but you can't say the entire codebase is badly written.
More /. FUD (Score:1, Insightful)
Linux Kernel vs Windows XP (Score:3, Insightful)
What about if you throw in KDE or GNOME, Mozilla, etc, everything that you'd have to add to really equal the features of Windows XP....
Re:Congratulations... (Score:3, Insightful)
I doubt they looked up all the code. They probably only made statistics to compare the amount of bugs based on what has been reported and archives.
As a side note, at 20 bugs per 1000 lines, the 40 millions lines of Windows would contain 800000 bugs. I'm not a M$ fan, but this sounds a little excessive.
20-30 bugs per 1000 lines??? (Score:3, Insightful)
Four Years? (Score:2, Insightful)
The 2.6 kernel isn't even a year old yet. How'd they do a 4 year analysis?
No I didn't RTFA.
That's not just funny - it's TRUE! (Score:2, Insightful)
Yes, the extra eyes looking at your code helps. But so does the fact that you know that there will be extra eyes looking at your code.
So you do a better job.
And the correllary is therefore that one of the big reasons some programmers don't like code inspections is that they don't want others to see their code.
Gee, if you worked for me and didn't want others to see your code I'd wonder why.
Re:Apple != Orange (Score:5, Insightful)
This is what you get for integrating your web browser into your operating system. Legality aside, there was a low cunning to that business move when M$ did it. Now, however, that decision is coming back to bite them on the tender bits: the browser is part of the OS, ergo bugs in the browser count as bugs in the OS.
Re:Mistake (Score:4, Insightful)
Then, when speaking of XP, they don't quantify the bugs, but merely say, "more are being found daily". Great... a pear.
Re:More /. FUD (Score:3, Insightful)
Re:Apple != Orange (Score:1, Insightful)
First of all I agree the comparison is a bit like apples and oranges, but it's not so far off if you keep in mind a few things:
First of all, the Linux kernel contains nearly every single driver available for the GNU/Linux operating system. Windows XP does not.
Of course, Windows XP includes a graphical windowing interface, which the Linux kernel does not.
However the Linux Kernel does include most of the things desired in an operating system. Two fully working and implemented sound architectures are there (OSS and ALSA). A networking stack that is infinitely more robust than the Windows one. It includes kernel modules for doing everything you could possibly imagine (and many things you can't).
There's a lot of obscure support for pretty interesting things. HAM / Amateur radio support. Bluetooth, framebuffer consoles, all sorts of interesting input devices (ranging from game pads for consoles to SUN keyboards and various touchscreens).
Basically what I'm saying is, just because Windows XP includes some things in that 40 million lines of code that Linux does not, Linux includes many things in its relatively small code base that Windows does not.
Having said that, the comparison is obviously apples and oranges, but regardless, a bug count of 985 as seen by various academic minds is very very good. It shows Linus' leadership has paid off, and bodes well for the Open Source community as a whole.
One thing I can guarantee is that if those same people went over the XP code base (even just the core kernel stuff) in the same fashion they did with Linux, the bug count would be MUCH higher.
Re:How can one be sure (Score:3, Insightful)
Well, the number of lines of code in the code base isn't strictly relevant, but comparing an unquantified rate of bug discovery against a quantified absolute number of bugs doesn't sound like sound practice to me.
It also makes no statement about the relative severity of the bugs - 100 trivial bugs with simple workarounds would certainly be worth fixing, but how can you compare them against, say, one bug that trashes your principal hard disk?
OK that's my opinion, now to R the FA.
Re:Conflict of interest... (Score:3, Insightful)
The real question... (Score:3, Insightful)
Mistaking symptoms for bugs (Score:3, Insightful)
The best you can hope to do is to count the number of *symptoms*, but that's not the same thing as bugs. It's possible, even likely, that a single bug in the code manifests itself as 100 apparently different external symptoms.
So this comparison is apples to oranges and completely meaningless.
Re:Congratulations... (Score:2, Insightful)
Yes, its VERY easy for somebody to notice a bug or unexpected effect.
it may also be very easy to suggest a possible fix for that bug.
However, in a large system with lots of interconnected parts, you cannot just impliment the fix without being certain that you haven't screwed up other parts.
Take for example the slashdot rendering bug in firefox. Plenty of people have reproduced the bug, plenty of people have suggested work arounds and fixes, however none has been stable enough to go into the proper branch (may have changed since I last looked, 1.0 is still problematic).
Now, another source of bugs and errors is compiler updates. Problems with buffer overflows and other "standardised" bugs can often be removed by changing the compiler switches or obtaining an update to the compiler, this is in effect how MS managed to remove a great many problems with SP2.
The inverse is also true, code which worked as expected on one version of the compiler may begin to fail during recompilation many years down the line leading to new unnoticed bugs.
smells like BS (Score:5, Insightful)
How do you analyze 5.7 million lines of code except to run it through a static analyzer? Static analyzers can't detect most errors, which tend to be data dependent. So a company selling an uberlint donates their tool, 5 academics run it and write a paper. *Blech*
Commercial software typically has 20 to 30 bugs for every 1,000 lines of code Bull. Software with 20-30 errors per KSLOC doesn't work. (I know, I just spent a year trying to save a project that had 10 errors per KSLOC. Even at that defect density (found and fixed) it was undeliverable. Notice that the link in TFA doesn't take you to anything that supports this assertion, just to the organization's home page. I could't find anything to support these numbers.
Weasely non-comparisons: 2.6 Linux production kernel, ... contains 985 bugs in 5.7 million lines of code as opposed to Windows XP, by comparison, contains about 40 million lines of code, with new bugs found on a frequent basis.What, bug in Linux are found on an infrequent basis?
This is all hot air.
Re:My thought, too (Score:2, Insightful)
Lea
Re:Apple != Orange (Score:3, Insightful)
The "Linux" code base just includes the kernel.
MS gets dinged all the time (at least by us UNIX guys) for what crud they put into the kernelspace, and much of that "extraneous crap" has caused a number of security and stability issues over the years.
Windows and Linux are fundamentally different in design and purpose. And I do believe that most of the "extraneous crap" code should be included in the kernel lines of code count if it runs in kernel space.
Re:20-30 bugs per 1000 lines??? (Score:5, Insightful)
It's not.
There are bugs that affects applications to a level the user notices, and there are bugs that affect the maintainability of the code, the reusability of the code, and the ease of use of the code. Most bugs are hidden from the user's view.
Does a stack overflow in a piece of code that only occurs when a craftily created exploit is executed constitute a bug? Yes it does -- even when no exploit exists. Under normal operations it does not affect the use of the system -- at least not until an exploit is developed and widely used. The fact that Windows, especially the older versions of the OS, were vulnerable to so many simple exploits illustrates the bugginess of the code. And most of the bugs are not even in close enough to the surface to be so readily exploited.
Does an end user see API bugs? Does anyone outside of Microsoft experience architectural flaws in the Windows OS? No, but that does not mean they do not exist.
What about code that misbehaves when presented with certain data. That may not be a problem for the original application that the code was written for, but any time that code gets reused, the programmer must know about these "hidden features" (read: bugs). Again, the user never sees this sort of bug.
The number they came up with wasn't pulled out their ass. See this article on measuring bugs [ganssle.com] for a more detailed discussion on the topic.
When reading it, bear in mind that most commercial software is produced for in-house use, receives very little QC, and frequently does not even compile cleanly. It's usuall just "good enough" to get the job done.
Re:Conflict of interest... (Score:3, Insightful)
Why do people always look for the worst in things? Yes, you have to have a certain level of paranoia when coding as it is hard to see some of the ramifications, but I think this goes beyond that. In this case it's a sensationalist story...
Flawed comparison (Score:3, Insightful)
Of course it is.
Most proprietary software are user-level apps where a bug here or a bug there isn't critical. The economic loss that can be atributed to a bug in MS Word's bullet-point-algorithm isn't as great as a datacenter going down due a Oops in the linux-box running the SQL-database. To put it simply: there are different requirements for different tasks. A comparison between the quality of WinXP's GUI and GNOME for example would have been much more interesting.
I don't understand how this story could make slashdot's frontpage.
Re:Where do they get these numbers??? (Score:2, Insightful)
Re:20-30 bugs for each 1000 lines??? (Score:3, Insightful)
I found a bug in your code already.
Re:partisan hackery (Score:3, Insightful)
Believe it or not, we're supposed to give Microsoft some credit for their recent good efforts. We're supposed to accept that someday in the future, they will ship secure, high quality software.
That may yet happen, and indeed XP SP2 was a good step, but a soild Microsoft system is still vapor.
Faced with superior products already existing on the market, Microsoft has consistently promised and promised and promised vapor... in the hopes of persuading everyone to keep on using their shoddy wares in hope that they'll someday, at an unknown time in the future, provide improvements. After all, they're "serious" and making a good effort.
We're also supposed to believe that Microsoft's process is somehow superious, despite flawed results, because of a 3 year roadmap. It's all part of the vaporware marketing tactic... don't buy something better NOW, instead wait for us. Never mind their poor track record of slipping their own dates and dropping those ambitious planned features.
Re:20-30 bugs per 1000 lines??? (Score:1, Insightful)
It certainly doesn't cover "Linux can't suspend my laptop" type bugs which tend to be the ones that the user notices.
Re:Mistake (Score:2, Insightful)
Re:Mistake (Score:2, Insightful)
Hardly a fair comparison (just like certain TCO studies) but that's not going to stop Linux vendors using it and slipping down to MS marketing levels.
However it's not clear if the comment
comes from Wired, or from the Stanford report.
Re:Apple != Orange (Score:5, Insightful)
Do you use only the Linux kernel? No. You run Slackware, or Fedora, or SuSE (my personal choice), or Gentoo, or something. A fresh copy of the latest version of a distribution is what should have been analyzed. All of the other stuff that's piled onto the kernel is what you use, such as the GUI, time-killing games, and all the other shit that a standard XP install disc contains. You don't click on line 11359 of the Linux kernel to open up Konqueror. You click on an icon, which starts a process that you use. The stuff piled onto the kernel is far more buggy than the kernel would be. I'd bet that a distribution of Linux is still far less buggy than Windows is, but it's certainly not 985 vs. 800,000. For example, when I minimize my windows on XP, there isn't hard drive activity for the next five minutes, as can happen with SuSE with not very much open. I'm sure something's misconfigured, but if a distribution, in 2004, is that poorly configured by default, then that's a bug.
Re:Kernel is not the problem (Score:1, Insightful)
It's usability, user-friendliness and consistency in the GUI where gnu/linux fails. Yes, it's kernel is rock-solid and never crashes (although I've seen linux boxes completely frozen when running Matlab).
But for it to be the success it is is acclaimed to be, the desktop is the area where it is still lacking with respect to both OSX and XP.
Re:Linux Kernel vs Windows XP (Score:2, Insightful)
On a Windows machine I used to use at work if I printed to a printer that was down from MS-Word the computer would completely lose it to the point where explorer.exe had to be killed (actually it would pop up the alert saying that explorer.exe is not responding, would I like to end task, so I clicked yes). I've actually never seen a Windows computer that ran smoothly after killing explorer.exe, every time I've done it I've had to do a restart to make it work well (that's probably because there was something else going wrong too). At least you can still save documents, etc., after killing explorer.
On this here Linux box, if I ssh to University Solaris machines and use -X instead of -Y and start a remote X app, my system goes to a grinding halt. To get out of that I have to switch to the console (which takes about 10 seconds) and kill ssh, because X is completely unresponsive (keystrokes in the console are slow as well, but it's at least usable). Although once I kill ssh it runs perfectly well, and I remember to use -Y next time. I've never had my Linux box crash to the point where it had to be restarted, but I don't run KDE or Gnome, so if those can really take a computer down I wouldn't know.
Not a true comparison (Score:2, Insightful)
There's also the question of defining a bug. Sometimes, one person's feature is another person's bug. Take allowing scripts in email, like Outlook Express used to do. (May still do, for all I know.)
Classic case for the do-nothing argument (Score:4, Insightful)
Re:20-30 bugs per 1000 lines??? (Score:4, Insightful)
It's a joke, laugh.
It's a sure sign that the moderators are getting stupid when you have to point out the jokes to them.
Re:Mistake (Score:5, Insightful)
There's no central Linux repository for reporting bugs in ancillary packages, while at Microsoft's site, all reported bugs go into a single database that can be queried. Each individual Linux distro and each individual Linux package maintains its own bug lists, which would have to be some how amalgamated.
In order to do the stated comparison, you would need to state what distribution you were using, you would have to state which patches you were using, and you would have to document where the information on bug counts came from for both products.
On top of all this, bugs per 1000 lines of code, while an industry metric, isn't a valid measurement of code quality, either, by the way.
One good code metric is the number of control points per function. The more control points per function, the more likely that you have a bug. It's always possible to take a complicated function and break it up into smaller pieces, and with C/C++ (but not
Another good metric is the distribution of severity of the reported issues. Draw a bell curve of severity of issues, compare percentage to percentage. You're not concerned with 1000 bugs vs. 100 bugs, you're concerned about how many of those bugs compromise your system or cause a crash.
Another good metric is delta bugs over time. There will always be bugs, everyone knows that there will be bugs. The question is the bug count going up or down.
Another good metric is delta time between open and close. Again, there will always be bugs, but how fast they are getting closed is a measure of whether a product is good or bad.
Another good metric is distribution of the number of days that bugs have been open and reported (by severity).
The reason for this is that the number of lines to implement a given function using a given algorithm varies between programmers and programming styles. Some will use two lines or statements to do what can be done in one, to make code easier to read. The line number is something that a developer can manipulate easily.
However, these other measurements are tied directly into the quality of the code. Nobody cares how big the Linux kernel is compared to the Windows kernel, or vice versa. What they care about is how well the Linux kernel works compared to the Windows kernel.
Any count related to bugs, also, needs to take into account the fact that on Windows, you have billions of users any of whom could find and report a bug. On Linux, bugs are more likely to go undiscovered for a longer period of time, simply because there aren't as many people trying to hit them.
The Windows and Linux kernel tend to be very similar in bug counts. The kernels of both OSes tend not to have bugs, because kernels tend to have simple code that's hard to mess up in any meaningful way.
It's only when you start including all these ancillary subsystems, device drivers, etc. that you start to see significant percentages of the bug.
And it is exceptionally hard to get an accurate count of those on Linux. On vMac, I documented approximately one bug in ten that I fixed, and I think that would be typical of other open source projects (although I can't swear to that).
I think that on these other, valid measurements that aren't dependent on lines of code, that if you could collect the statistics on a package per package basis, and compare them, that Open Source would still come out ahead of Windows.
It's just a matter of using a meaningless metric on widely divergent code bases to prove any point is irresponsible, and by reporting it, the media is doing a disservice to the Open Source community by perpetuating the image of the community as a bunch of college students who don't understand real development practices.
Re:Mistake (Score:2, Insightful)
But you're wrong, they don't. IE and Media player run in user space in Windows. As do the vast majority of "buggy" programs in Windows.
It would be far more fair to compare XP with the Linux kernel + X + Gnome/KDE + browser + email app + xine + a bunch of other equivalent user-space utilities. But that would be a lot of work, and deciding where to draw the lines would be difficult. The two OS are architected very differently... shouldn't the horrendously buggy third-party video drivers that ship with most linux distros be included, since so many third-party drivers are included with Windows?
I'm not saying Linux would lose such a "fair" comparison, it would probably still be found less buggy than Windows. My point is this referenced article is stupid, useless propaganda generated by someone with an axe to grind.
Plus (Score:3, Insightful)
The whole thing kind of strikes me as people talking out of their ass.
Re:Apple != Orange (Score:4, Insightful)
Exactly how does making the browser part of the operating system make Windows more usable?
Because they did this, I now can't remove IE if I want to. I have trouble with outlook if I set mozilla as my default browser. I can't upgrade IE without patching the OS (which means other programs that use IE components might break). There is no way for me to downgrade to earlier browser versions. There is no way for me to have multiple versions of IE installed, so it makes it hard to test. Because IE is in the operating system, Microsoft has delayed releasing new features until the next version of the OS is shipped.
I can't think of a single thing that integrating the browser into the OS buys you, except some code reusability. From my engineering perspective, this wasn't done for any legitimate engineering reason, but because of legal and business reasons.
Another obvious biased stance (Score:3, Insightful)
Now I can see maybe comparing the kernel mode code. Windows throws a lot of stuff in the kernel, much of which UNIX types disagree with. Basically the whole graphics subsystem, including rendering engine, is all kernle mode. Microsoft decided that the speed increase was worth it and it didn't matter since if it goes down, the system halts anyhow since Windows doesn't drop to shell. UNIX is different, if X crashes, big deal, you now have a console and you restart X.
However even with kmode comparisions it gets hairy. I mean what is really necessiary and proper there? Talk to a hard core microkernel fan of soemthing like QNX and they'll say NOTHING excpet the kernel, not even drivers. Linux put a LOT in kernel space as compared to some OSes.
Is anyone else bothered by this? (Score:5, Insightful)
Why does "Linux is just a kernel" suddenly not apply here? Is it because we're bashing Microsoft? This is kind of lame and makes the community look silly and vitriolic.
And yet (Score:3, Insightful)
I'd like to see a serious discussion of the methods and funding sources in this study. You know, the same kind of discussions that appear whenever a Microsoft study is posted. Mysteriously, that critical thinking is absent in articles like this.
40 million lines of code != 5 million lines of code. If Microsoft attempted such a skewed study, I can already picture the kinds of posts people would be writing, asking about methods, funding, and more. Those posts aren't appearing in this article. Why? Well, that's because this isn't about the quest for finding out which operating system actually has less bugs (sorry, I forgot, "Linux is just a kernel!"...funny how that oft-uttered phrase is missing in the article discussion), this is nothing more than a Microsoft-bash article intended to generate Microsoft-bashing replies.
OSTG, the corporation that owns Slashdot, makes money off of OSS products. It's rather convenient that they run a "tech news" site that just so happens to post news stories that are all critical of their competitors, isn't it? Imagine if Microsoft owned a tech news site that posted nothing but anti-Linux articles? You guys would be all over it, but your biases against Microsoft make you have a double-standard. So you ignore that Slashdot is a corporate-owned website with a major bias and an editorial staff that can barely read and write correctly, much less bother to read their own website.
Re:20-30 bugs per 1000 lines??? (Score:2, Insightful)
Dude, letting dhcpcd set your hostname is extremely lame.
Re:Is anyone else bothered by this? (Score:3, Insightful)
Re:Is anyone else bothered by this? (Score:3, Insightful)
According to the author: Linux has fewer lines of code AND fewer bugs per thousand lines of code when compared to typical commercial software.
Currently linux has 985 known bugs in 5.7 million lines of code. If linux is to compete with commercial code it will need about 114,000 to 171,000 bugs in 5.7 million lines of code.
Re:Flawed comparison (Score:3, Insightful)
You don't understand how the story could possibly make the frontpage. To those that know how slashdot really works, it was inevitable that once this story was written, it would reach the front page.
oh man I messed that up bad.
Re:And yet (Score:1, Insightful)
And while it isn't a fair comapison, you must admit that having so few bugs in the kernel alone suggest that the general standard of coding is high. nobody could trawl through the 40mill lines of WinXP code, just like no one could go through all the software that makes up the Linux desktop (especially seeing as it's constantly updated) But anyway: Do you run windows update? have you seen the number of patches that appear? do you have to keep an antivirus running 24/7 to watch over your security holes?
I know i do. And yes, despite all the advantages of Linux, i run windows, because at the moment i need more ease of use, no time to learn a new OS.
Bug-proof isn't everything.
Re:Is anyone else bothered by this? (Score:-1, Insightful)
Anyone wonder why it takes 40 million lines of code to do what 5.7 million do?
The Windows kernel, based on VMS, is smaller in size than the monolithic Linux kernel.
And no, that doesn't include "all of Microsoft's applications."
Yes, it does.
What "included applications" were you referring to? Notepad? Gimme a break...
How many lines of code is X-Windows? And GNOME? And all the Linux drivers? Windows is a kernel, a HAL, a graphical subsystem, a desktop environment, a shitload of APIs and DLLs, and basically every other desktop component that currently makes up GNOME/KDE/X-Windows/Linux. Yet the comparison only cited Linux and not the rest.
Linux is written by programmers who obviously love programming, since they work on it in their spare time outside their normal programming jobs.
You are obviously a fanboy. We know what Linux programmers are like.
Windows is written by 24-year-old recent college CS graduates who have no fucking clue how to do anything - managed by "product managers" who have even less clue - managed by corporate execs who are KNOWN ASSHOLES!
Stop by Microsoft sometime for a job interview. We'll see if you stand shoulder to shoulder with Microsoft's engineers. Ever compared the level of desktop Linux to the level of, say, Windows 95 from nine years ago?
Mod me troll, mod this flamebait! Is that all you got, huh? Are you nuts? Come at me!
I guess Flamebait.
I said -pedantic gives warnings. (Score:3, Insightful)
-pedantic
Issue all the warnings demanded by strict ISO C and ISO C++; reject all programs that use forbidden extensions, and some other programs that do not follow ISO C and ISO C++. For ISO C, follows the version of the ISO C standard specified by any -std option used.
-- That seems to be exactly what I described it as doing.
Now, you choosing to make standards un-conformance into a warning rather than an error is something else. (Old) C defines the default return type to be int, but C++ has it as undefined. Forcing it to a warning is not the way to solve that problem; you should've fixed the code, which is why g++ defaults to making the type checking as errors, instead of warnings.