The Annual US-CERT FUD Festival

Joe Barr writes "Joe Brockmeier and I have teamed up in a story on NewsForge to point out how the mainstream and trade press misrepresent the annual summary of vulnerabilities from US-CERT. They're doing it again this year to make it appear as if it is more secure than UNIX/Linux. Pamela Jones did a similar report at Groklaw over the weekend." From the article: "One figure represents the vulnerabilities found in Windows operating systems: XP, NT, 98, and so on. The other represents a total figure not just for Solaris, AIX, HP-UX, the BSDs, and Linux, but for a hundred different versions of Linux. The sum of all the unique vulnerabilities from all the Linux distros does not equate to the sum of vulnerabilities in any single Linux distro, and one could say the same about the various versions of Windows. That's why it is a completely meaningless exercise to discuss those totals as if they present an accurate picture of the relative security of Windows and Linux. " We've reported on the US-CERT list already this year. NewsForge is a sister site to Slashdot.org, both of whom are owned by OSTG.

Downright Disingenuous

[TripMaster Monkey]

The act of contrasting the vulnerabilities found in the few Windows operating systems with the vulnerabilities found in hundreds of Linux/Unix is bad enough, but when you consider that the Unix/Linux list contains duplicate items, it becomes positively shameful.

From the Groklaw article:

Second, the Unix/Linux list duplicates items, counting a vulnerability more than once in the list. For an example, note that it lists Eric Raymond Fetchmail POP3 Client Buffer Overflow (Updated). However, the same vulnerability is listed, under the same title, four times. That's because it was reported in the week of August 10-15, again in the week of August 17-23, in September 6-13, and the week of November 9-16. Worse, for any comparison purposes, the same vulnerability is also reported as Fetchmail POP3 Client Buffer Overflow, so in reality one vulnerability is listed 5 times, making the total of 2328 meaningless unless you carefully comb through it to weed out duplications.

I honestly expected better from the CERT [us-cert.gov] folks. I don't know why, but I really did.

Re:Downright Disingenuous

[User 956]

The act of contrasting the vulnerabilities found in the few Windows operating systems with the vulnerabilities found in hundreds of Linux/Unix is bad enough, but when you consider that the Unix/Linux list contains duplicate items, it becomes positively shameful.

It looks like we both posted at the same time. At any rate, you have a point to a certain degree. My post here [slashdot.org] shows that if you go through the list and subtract out all the items with "updated" after them, Subtract OSX and Solaris, the Linux/Unix group category is about par with windows, not 3x worse.

Whether "different" OSes should be lumped together is another discussion entirely (how "different" are they if they have the same kernel?)

Re:Well..

[rubycodez]

heh, "several" Linux distros, there's over 90 of them!

From the article.... anti-FUD stats

[CodeShark]

Not intending to "karma whore" here, but look at the stats from an already done analysis:

• 22 Technical Cyber Security Alerts were issued in 2005
  • 11 of those alerts were for Windows platforms
  • 3 were for Oracle products
  • 2 were for Cisco products
  • 1 was for Mac OS X
  • None were for Linux
  , and secondarily look at this quote
• "Here's more of the same. US-CERT's list of current vulnerabilities contains a total of 11 vulnerabilities, six of which mention Windows by name, and none of which mentions Linux.

Folks, as other /. posters have already discussed better than I can, most of the supposed Linux bugs are either duplicates or in user- space software. That would be akin to saying a Firefox browser vulnerability is a Windows OS security problem,as opposed to an underlying OS vulnerability that would affect any and all software on the platform.

Not true.

[fireboy1919]

They've got Apache vulnerabilities listed on the Linux side, but not on the Windows side - vulnerabilities that affected both places, I might add.

This is true of most of the *nix vulnerabilities, actually.

So what we're really seeing is Windows-only vulnerabilities being compared to ones that are OS neutral. Not that its very suprising, though. Its 2006.
With the exception of software written specifically for Windows, most software is cross-platform.

This is the only really meaningful way to do this kind of a report because of this characteristic. The important thing to keep in mind in that, though, is that Windows has all of its own vulnerabilities AND most of the others. :)

Re:Should Compare A Single Version Of Windows Too

[GuyverDH]

It's valid, and yet invalid - all rolled into one.

No they aren't many different distros, only 2.

Windows 1.x -> ME are all different versions of windows management systems based on MSDOS.

Windows NT 3.x -> 2003 are all different versions of windows management systems based on NT.

So only 2 distros, with lots of versions.

Now Linux has had how many distros? I've read as high as 90, and no, I haven't done the research myself to come up with my own answer, but I know personally of at least 20.

Add to that the BSD distros, of which I know of 3 personally.

Then they lumped in 4 completely different Operating systems - not even distributions.
AIX, Solaris, HP-UX and MacOSX - all of these are true UNIX operating systems - not the complete list by far - Tru-64, Centix, C-TIX, the pre-caldera UNIXWare, OpenServer, Xenix, UNIX, etc...

Remember, Linux ISN'T UNIX. So why the hell would they lump them together. Here's why - it's the only way they could get the numbers to add up to anything close to a large margin above the count from the 2 distros of Windows.

FALSE.

[WindBourne]

Umm, I looked at the list and they weren't counting the same vulnerability multiple times.

Very false. just look for Larry Wall Perl Insecure Temporary File Creation (Updated). Three instances of the exact same item. And only in *nix even though ActiveState perl for Windows had the same issue. So, there are LOTS of issue with this report. Cert is more SNAFU, than not.

I prefer my way.

[khasim]

Simply evaluate each vulnerability in a simple hierarchy. When evaluating a distribution or a version of Windows, use only the apps installed by default.

1. Remote--root access that does NOT require human intervention or other app running.

2. Remote non-root access that does NOT require human intervention or other app running.

3. Local root access that does NOT require human intervention or other app running.

4. Local non-root access that does NOT require human intervention or other app running.

5. Remote root access that requires some human interaction or some combination of apps.

6. Remote non-root access that requires some human interaction or some combination of apps.

7. Local root access that requires some human interaction or some combination of apps.

8. Local non-root access that requires some human interaction or some combination of apps.

9. Remote OS crash.

10. Remote app crash.

11. Local OS crash.

12. Local app crash.

There, now it should be easy to [b]exactly[/b] compare different systems. A thousand #12's (local app crash vulnerability) is still not worth a single #1 (remote root access).

SECURITY is about REDUCING the avenues of attack. A default Ubuntu install will never have any vulnerability above a #3 simply because it has no open ports, by default. This is extremely important when your machine is connected to the Internet.

Simple pre-processing would help

[Kelson]

one vulnerability is listed 5 times, making the total of 2328 meaningless unless you carefully comb through it to weed out duplications.

They could have cut it down to a more manageable list by piping it through "grep -vF '(Updated)' | sort -u".

That brings it down to just 871, which is much easier to comb for further duplicates.

The same process on Windows vulnerabilities brings it down from 831 to 659. Both lists still need to be checked for duplicates with different names (say, "Apache HTTP Request Smuggling" and "Apache HTTP Request Smuggling Vulnerability"), but we're now looking at a much more comparable set of numbers.