The 25-Year-Old BSD Bug

sproketboy writes with news that a developer named Marc Balmer has recently fixed a bug in a bit of BSD code which is roughly 25 years old. In addition to the OSnews summary, you can read Balmer's comments and a technical description of the bug. "This code will not work as expected when seeking to the second entry of a block where the first has been deleted: seekdir() calls readdir() which happily skips the first entry (it has inode set to zero), and advance to the second entry. When the user now calls readdir() to read the directory entry to which he just seekdir()ed, he does not get the second entry but the third. Much to my surprise I not only found this problem in all other BSDs or BSD derived systems like Mac OS X, but also in very old BSD versions. I first checked 4.4BSD Lite 2, and Otto confirmed it is also in 4.2BSD. The bug has been around for roughly 25 years or more."

Re:Many eyes make bugs shallow...

[Anonymous Coward]

Isn't it? The bug WAS found, wasn't it?

Re:Many eyes make bugs shallow...

[DannyO152]

How many "eyes" were watching BSD systems use Samba for a DOS filesystem? Seems to me, someone saw behavior and exactly because it was open source, looked into it, found the coding error and filed a bug report. It will be fixed, because everyone now knows about this, and that too is a side effect of open source, even if it's related to the politics.

Re:Many eyes make bugs shallow...

[garett_spencley]

From the sounds of it, this was a bug that was not triggered very often. When it was finally triggered, investigated and fixed the person who found it released the info publicly, thanks to the beauty of Open Source, and everyone affected, commercial entities and FOSS users using the code alike, benefited. If this were a proprietary system that were licensed out to various companies stricken by NDAs etc. it's quite likely that if one company discovered the bug the others would never learn about it.

Re:They actually do...

[Goaway]

After all, someone closed a 25-year bug... how many hidden bugs will remain that way in os/2 warp? windows 95? other proprietary systems?

Er, this bug was in current versions, too, you know.

Re:See? SEE?

[Spy der Mann]

No need to run and hide. The only reason the bug's so old is because the OS is that old. And remember that 25 years ago, we didn't have web 2.0.

In comparison, Microsoft has been around for what... 20 years? And who knows what bugs in Windows are there, lurking, just waiting to bite us?

Re:more proof

[Keyper7]

Unless you present some proof that this never happened in a Microsoft product, you have nothing.

I'm not saying it did, but you have to take Microsoft's word that it didn't, and no more evidence than that.

I'd rather go with the more transparent option, thank you.

Samba knew, but didn't pass it on?

[quarrel]

The most telling thing in TFA for me was that the bug had been identified by the Samba team and a workaround implemented for Samba.

Surely both the samba communities and the *BSD communities are active enough that this could have been passed on for further investigation by the *BSD crowd? (Sure, samba probably would still need a workaround, particularly given the long uptimes and widespread deployment of *BSDs)

I know nothing of the devs at Samba and *BSD, but seems a bit strange. Perhaps they did try..

Meanwhile, congrats to Marc on fixing a bug. One of the most touted benefits of open source (whatever your license) code.

--Q

Old Code

[Frosty Piss]

One would think that after all these years, Windows source would have leaked, Microsoft being what they are. Everyone knows that your most secret secrets are the first things to leak... And yes, I've seen the "secret MS code" joke.

Re:Many eyes make bugs shallow...

[InvisblePinkUnicorn]

This is like saying global warming either does exist because today was the hottest on record, or does not exist because today was the coldest on record. Why are these analogous? Because in both situations, you're only considering one data point, which does not even begin to indicate a trend.

Re:Many eyes make bugs shallow...

[Peganthyrus]

You see what you're looking for, most of the time. This sounds like a subtle bug that you're not going to find until you go looking for it; it's hard to invoke under normal usage patterns. Nobody stared at that code looking for this problem until now. But if it was closed source, the guy who fixed it wouldn't have been able to look at it and find the problem.

A quick googling of "many eyes make all bugs shallow" brings me the more complete statement that adage is simplified from: "Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix will be obvious to someone." (Linus via ESR). Clearly this 25-year-old bug is one of the exceptions that calls for the 'almost'.

Re:Now it's time for a little housekeeping

[hitmark]

i dont think its easy to trigger this reliably enough to use it to hide files.

Re:Many eyes make bugs shallow...

[Derek Pomery]

Erm. That's not what "many eyes make bugs shallow" means.
Well. Just reading the source is part of it, but not all.
Fact is, if I run into odd behaviour when testing/using - if the source is available I can read it, I can breakpoint.
I cannot do that with a binary.

So yes. Things did occur as they were supposed to. Someone found something odd, they were able to look at code in question, and fix it.

The shallowness is the fact that there is a direct connection between the thousands of testers/users and the code in question.
Instant turnaround. No "user reports behaviour in detailed fashion, including testcase, to some corporate e-mail address, and maybe it eventually gets a to a developer three layers down who may be able to figure it out and fix it if he has the time"

Should it be fixed?

[CaroKann]

Considering how old this bug is, and how much work-around code probably exists as a result, I wonder how many new bugs this bug fix will create.

Re:Samba knew, but didn't pass it on?

[irc.goatse.cx troll]

I wouldn't pass it on either if it meant having to deal with Theo.

Re:Would this be a "Critical" or "Important?"

[Goaway]

Samba cared enough to implement a workaround.

Re:Many eyes make bugs shallow...

[Goaway]

Perhaps they didn't feel like doing the "no, it's supposed to work like that, you're wrong" dance.

Re:Should it be fixed?

[iamhigh]

Yeah, imagine if you had, like 90% market share, and your OS supported everything from 8 bit games to 64 bit collaborative applications... then it would be a real bitch to fix stuff.

Re:Many eyes make bugs shallow...

[TheLink]

Sure it's found, but after 20+ years? That's not what I call a good approach.

"many eyes make bugs shallow" is equivalent to the "infinite number of monkeys..." thing.

In my experience it's better to have quality than quantity when it comes to the eyes used for finding bugs.

Any idiot can tell you about obvious bugs, and it's kind of waste of time to see 1 million duplicate bug reports, because it's too slow to search through 100 million other bugs (with dupes) for dupes ;).

For UI stuff, you get the naive (as in not yet unexposed to your evil software ;) ) users in and watch them, and even then you MUST have _trained_ eyes to watch them. The trained eyes can often spot problems the naive users are experiencing - the naive users may not even realize they are experiencing problems or realize what is wrong...

Re:Many eyes make bugs shallow...

[KGIII]

The shallow bugs argument basically says that if enough people stare at the code, they will find the bugs. Clearly that did not happen here.

I am not an open source zealot or anything, in fact I'm more inclined to use proprietary software at home and open source, almost exclusively, for my business. I say this so that I'm clear about my disclosing my personal views. (In other words, I believe in all things moderation and think zealotry is absurd and I probably shouldn't be confused with an open source zealot or even an advocate.)

However... You are mistaken. Clearly that DID happen here. That is exactly what happened. It took a long time, granted, but it was found, it was fixed, and (seeing as no one was really bothered by it too much) it was even done in a "timely manner." (Albeit a very long time but it didn't seem to impact anyone to any great length.)

The bug was found, the code was open, the bug was fixed. Hell, it isn't even newsworthy. It's just a squished bug that had no real impact in the majority of people's experiences. The impact was so small that people coded around it, so be it. Open source doesn't even demand that it be bug free. Coding around an existing bug is perfectly okay (I think). Fixing the bug may have taken valuable developer time away from their core projects for all I know. Either way, it was found, it was fixed, and because of the open source nature of the software these two things were possible.

This couldn't have happened with Linux...

[Baumi]

...because currently, no Linux bug could possibly be older than roughly 17 years [wikipedia.org]. :-)

Re:Many eyes make bugs shallow...

[AndGodSed]

True, but you misquoted the statement. The correct statement is not absolute. It reads, and I quote a guy called Linus:

"Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix will be obvious to someone."

I am sure you will agree that the correct statement sans flamebait modifications does not warrant a "clear contradiction" as many detractors of FOSS who are jumping at this opportunity to point out a example of a fixed bug that was not necessarily a security risk and saying "see, the OSS model is clearly flawed! BSD has a 25year old bug that was only fixed now!"

Heck, how many other bugs have been fixed over the years?

These detracting arguments smack of FUD mongering...

Re:yeah, sure

[AndGodSed]

Excellent point.

This is touted as a slip up (or flaw) in the Open Source model of doing things, yet a proprietary software developer, and one of the largest mind you, failed to spot this completely.

Re:Many eyes make bugs shallow...

[fudoniten]

BSD has been checked over by 'quality' eyes--when it was used as the basis of NeXT/OSX, for example. They missed it too.

If the code wasn't open (i.e. if there weren't many eyes), this bug would have remained forever, or at least until the code was dumped.

Re:Many eyes make bugs shallow...

[iminplaya]

If this were a proprietary system that were licensed out to various companies stricken by NDAs etc. it's quite likely that if one company discovered the bug the others would never learn about it.

That, or the individual who discovers it would be fired, blacklisted, and possibly be arrested and charged with some criminal act.

Re:Many eyes make bugs shallow...

[LS]

Wuh? thousands of projects that prove the point, and one single bug that doesn't, so reject the whole argument?

You sound like those people that don't like evolution. The concept of shallow bugs is an approximate description of how things work, not a methodology. Also, if enough people stare at the code and use it, they will find the bugs that matter.

Re:Old Code

[dHagger]

The virus-makers were! IIRC there were some virus-outbreaks shortly after the leak, using flaws supposedly found in the leaked code.

Seems like a strawman arguement to me

[shis-ka-bob]

Newton's Law of Gravitation doesn't correctly predict the precession of perihelion of Mercury, so perhaps we should stop teaching Newtonian mechanics. If a law isn't perfect, it can still have value.

I don't believe that many in the open source community dogmatically accept the shallow-bugs arguement as a universally applicable rule of software development. It seems to me that any reasonable person would agree that

• Not all eyeballs are equal. Theo and Linus are vastly better at code reviews than the average Slashdot reader, for example. So, there is a wide range of eyeball quality.
• Bugs have a wide distribution of subtlety. The bug in this article seems to be high on the subtlety scale.
• The 'with enough eyes all bugs are shallow' statement is universally recoginized as simplification. For example, ESR states [gnu.org] that a better statement is:

  What is true is that (1) with many eyes, shallow bugs get caught very quickly, and (2) that the more eyes there are, the more likely it is that some member of the group has sufficiently penetrating vision to catch the deeper-swimming bugs.

So, yes, if you start with the sound bite version of a 'law' and apply it to a statistical anomaly, you can 'disprove' the law. We now agree that the sound bite doesn't perfectly correlate with 'real world' So, you are now forced to decide if the relationship expressed in the 'law' is uncorrelated (bug fixing uncorrelated with eyeballs) , anti-correlated (eyeballs hinder bug fixing) or correlated (eyeballs often help bug fixing). I believe that the law is still a good rule of thumb and that the correlation between 'many eyes' and 'bug get found' is strong.

Re:Many eyes make bugs shallow...

[xtracto]

So, it is just another Open Source bug which the developers did not care to fix because it was not the itch to scratch. Similar to plenty of other bugs which are notified in bugzilla kind of systems and the developers just ignore for months and even years.

Re:Excuses Excuses...

[Paradise Pete]

All the excuses and explanations in the world wont change that this is EXACTLY what open source proponents have been saying wouldn't happen with open source.

Nobody's saying that - that would be retarded. What's said is that there would be fewer undiscovered bugs, and more rapid and complete fixes. Notice how he began his investigation by looking at the SAMBA source code. You might also note that he was able to look into the second issue as well, making both SAMBA and FreeBSD a tiny bit better along the way.

If this fix had happened at a closed shop the biggest difference is that you probably wouldn't be reading about it.

Re:Long live the Code

[Rod Beauvex]

Don't we criticize another company for this practice? :D

Demonstration of the bug

[ChrisDolan]

The Perl program below demonstrates this bug. Tested only on OS X...

#!/usr/bin/perl -w
use strict;
use File::Temp qw(tempdir);
use File::Slurp qw(write_file read_dir);
use Test::More tests => 9;
 
# Create some temp files
my $dir = tempdir(CLEANUP => 1);
write_file("$dir/one", '1');
write_file("$dir/two", '2');
write_file("$dir/three", '3');
 
# Confirm that the directory contains the files
is_deeply([read_dir($dir)], ['one', 'three', 'two']);
 
# Open a directory handle and read through all files
opendir(my $dirh, $dir);
is(scalar readdir($dirh), '.');
is(scalar readdir($dirh), '..');
my $file1 = readdir($dirh);
is($file1, 'one');
my $file2 = readdir($dirh);
is($file2, 'three');
# Record the position of the second file
my $pos2 = telldir($dirh);
my $file3 = readdir($dirh);
is($file3, 'two');
 
# Rewind to the second file's pos, and confirm that the next read is the third files
seekdir($dirh, $pos2);
is(scalar readdir($dirh), $file3);
 
# Delete the first file and try the above test again. It *should* have the same results
ok(unlink("$dir/$file1"));
seekdir($dirh, $pos2);
is(scalar readdir($dirh), $file3);
 
closedir($dirh);

The output of the program is:

% perl bsdbug.pl
1..9
ok 1
ok 2
ok 3
ok 4
ok 5
ok 6
ok 7
ok 8
not ok 9
# Failed test at bsdbug.pl line 30.
# got: undef
# expected: 'two'
# Looks like you failed 1 test of 9.

Re:Many eyes make bugs shallow...

[fabs64]

You think this only happens in the open source world? Let me show you what the "defect priority analysis" would look like at my work were we to receive a report about this bug: Reproducible: Yes Frequency of occurrence: Extremely low, only comes manifests for a very rare corner case. Systems known to be impacted: None, systems that have noticed defect previously have already implemented a workaround. Current known impact upon the functionality of the system: None Systems currently using code where defect is present with no impact: All systems accessing a directory Potential negative impact of an incorrect fix: Extremely high, potentially crippling filesystem traversal. Proposed solution: Wait till people stop using DOS filesystems.

Re:Many eyes make bugs shallow...

[dissy]

what was so special about this code that the samba developers couldn't have just written a patch rather than a work around?

Well, if you wrote a program with a bug, and I wrote a program that used yours, and i found your bug, what would you expect me to do if i told you about it and you said "that isnt a bug, and we wont be fixing it"?

Why would I write a fix to give to you when I know you wont be adding it to your program at all?

Why would I write it at all? Specifically in this case, its usually considered 'bad' for a user level application to require you to patch your kernel, and only on a certain OS but none of the others. I realize this is done, but no one likes it.
Especially when you can replace 'if you are on bsd, you must patch your kernel' with the current 'you do nothing, the software will figure it out and it will just work'.

The samba team can not force the BSD team to change bits of their software. All the samba team can control is their own code, which they used to code the work around so it would still work, without having to tell bsd users "sorry, a feature of your os (that every other os calls a bug) prevents our software from running". Someone else would patch samba to work around this non-bug 'feature' and then you get the same situation as now.

Of course of the original bsd dev would have dropped his ego and code-political agenda for a minute and just admit it was a bug, none of this would have happened, and it would have been fixed right after it was reported the first time.

Don't click that link!!!

[Spy der Mann]

If you're going to work in Open Source projects related to Operating Systems, stay away. The dreaded "trade secrets" accusation could ruin your whole career.

Here is why

[Solr_Flare]

BSD and the *nixes were designed to be simple, effective, modular operating systems. As long as you have the drivers and know how, you can easily port them over and install them on a variety of hardware. Then, thanks to their modular nature, you can then plug in all the extra bells and whistles you need for your particular system and go to town.

That is why they are still around and still popular. They are K.I.S.S., work as they are supposed to, and the modular code that is plugged into them can just be sloughed away when it becomes out dated, and newer, better code plugged in to modernize the OS as you go.

That's also why Windows has had so many problems over the years. Windows was designed to be everything you need in a single package. That means everything is all tied up together. So, unlike BSD and the *nixes, when part of the OS becomes out dated, MS can't just unplug the old stuff and plug in new stuff. It's all interlinked from the ground up. That means a large portion of development time getting is spent fixing bugs caused by new additions, which then cause even more problems down the line when you go to update again. It also makes it bloated as legacy code ends up stuck in the mix because without it the patched together additions wouldn't function right.

And, unfortunately for MS, their market dominance is based on the windows "feel" being familiar and backwards compatibility. If they could, I'm sure they'd re-write windows from the ground up, but now they are in a catch 22 where doing so might significantly kill their market share.

I'm guessing Bill and company sometimes look back and kick themselves for not having the guts to go for broke and re-do the OS from the ground up for Windows XP. Because, back then MS was still king, Apple was at its low point with a very small and stagnant market share, and the *nixes were still primarily a hard core enthusiast hobby. Today, if MS were to completely change Windows, they'd probably lose a significant amount of market share to a variety of alternatives.

OpenBSD is good at fixing this kind of bug

[chrysalis]

This is not the first time the OpenBSD time does an excellent job at finding obscure bugs that were lying around for one or two decades in every BSD derivative. Congratulations !

Re:Many eyes make bugs shallow...

[kelnos]

i find it kind of ironic that the samba developers, rather than finding the code, and fixing it, coded a workaround instead.

I don't. The Samba guys encountered an incorrect behavior based on user-space code. Who knows how long it would've taken to find the cause of the bug.

And even more important, they would have to code a workaround anyway, to unbreak older BSD systems running Samba.

Extremely complex systems cannot be perfect.

[Anonymous Coward]

I am astounded that some people are claiming that this shows OSS does not work, BSD sucks, etc.

Operating Systems are extremely complex and created by man. It is therefore impossible for any of them to be absolutely perfect.

For a bug to be hidden for 25 years (the bug itself and not the symptom), it would have to be obscure and occur only in fringe cases. Well, this bug was obscure and only occurs in fringe cases!

If this were happening in closed source software, the bug would be near the bottom of priorities, since it only occurs under rare conditions. Nobody will bother going to the trouble of analyzing the binary, so to be fixed it would need to be taken seriously by the company behind to closed code.

In this OSS case, it only had to be taken seriously by one talented person.

BSDi and Apple didn't find it. Some person who was empowered with the code being open found it and fixed it. In fact, this fix to Apple's OSX, comes courtesy of the O in OSS.

Proof that OSS works, from one of the oldest OSS bases around. You would expect a bug to be pretty obscure in such old OSS code and that is exactly what this bug is. Is it any wonder that the BSD's are so incredibly stable?