Vulnerability Numerology - Defective by Design?

rdmreader writes "RDM has a point by point disassembly of the security vulnerability story phenomenon. We regularly see these, comparing various vulnerability lists for different operating systems. ZDNet's George Ou, for example, condemns Linux and Mac OS X by tallying up reported flaws and comparing them against Microsoft's. What he doesn't note is that his source, Secunia, only lists what vendors and researchers report. Results selectively include or exclude component software seemingly at random, and backhandedly claims its data is evidence of what it now tells journalists they shouldn't report. Is Secunia presenting slanted information with the expectation it will be misused?"

Numerology?

[RyanFenton (230700)]

<Skeptical Nitpick>
Did the guy who titled this know what the term Numerology [wikipedia.org] means? It's usually associated with wild "magical thinking" about numbers, and is at best a rather silly form of pseudomathematics.
</Skeptical Nitpick>

Ryan Fenton

Re:Numerology?

[Spy der Mann (805235)]

Did the guy who titled this know what the term Numerology means?

Exactly. IMHO, he's saying that Secunia vulnerability comparisons aren't any more reliable than numerology predictions.

Re:

[Billosaur (927319)]

No, I think he's saying if you apply numerology to trojans/virii, you can gain insight into their personalities...

Re:

[pboulang (16954)]

And yet it was used, and you knew exactly what it meant: hence, it is a perfectly valid word.

Re:

[Anarke_Incarnate (733529)]

Knowing what somebody means does not make it correct (or cromulent). Since it is not a word (much the way Ain't is not a word) then simply allowing people to bastardize two languages is simply incorrect.

Oh, and P.S.
Piss off, you wanker.

Re:

[pboulang (16954)]

And yet the dictionary adds words every single god damn edition. Those bastards! I assume you don't use words until your dictionary of choice adds them? How civilized of you. You are just now learning how language evolves. Your argument is wholly without merit, especially in this context.

Re:

[Anarke_Incarnate (733529)]

A dictionary adding words is fine, for when their exists no word to truly express the subtle meanings that the new word provides. We ALREADY HAVE a word to do EXACTLY what this "word" purports to do. That word is viruses. Don't grasp to the straws of failed logic because of a false sense of entitlement.
Using a "word" in the wrong context does not give that word new meaning. It simply makes you look stupid.

Re:

[mhall119 (1035984)]

And yet it was used, and you knew exactly what it meant: hence, it is a perfectly cromulent [wiktionary.org] word.

There, I embiggened that for you.

Re:Numerology?

[squiggleslash (241428)]

Look, it's really quite simple what he's trying to say: If you add the letters of "ROUGHLYDRAFTED.COM" (A=1, Z=26, . = 0), you get 195. Add those digits together and you get 15. Again, add a third time and you get 6. So after adding the digits together three times, we get 6. Six repeated three times is "666", which is specified in the Bible as being the mark of the devil.

Now, if you do the same thing with "SECUNIA", you get 72. 7+2 = 9. And 9, added to itself, is 18, and its digits also add up to 9. So nine is obviously significant.

What does this mean? It's quite simple. The Devil, as specified by the Bible, is also what tempted Adam and Eve to take an Apple from the tree of knowledge. You see where this is going? ROUGHLYDRAFTED.COM is essentially saying that Apple is the source of knowledge. Whereas SECUNIA's, like, nine, or something.

Does this help?

Re:

[sumdumass (711423)]

Wow, You forgot to work in the in the hebrew language, W is the number 6. Actually, there isn't a W but once to translate English to Hebrew, W become a single U which is 6. So the entire World Wide Web WWW is 666 or the devil too.

Re:

[Nazlfrag (1035012)]

I always wondered why Apples apple had a little bite taken out of it.

Re:

[irenaeous (898337)]

. . . that Secunia vulnerability comparisons aren't any more reliable than numerology predictions.

I RTFA. He is not critical of Secunia per se. He quotes a lot from Secunia's advisories and claims that George Ou has misused the data. In other words, Ou is practicing Numerology with Secunia's numbers. Presumably then, Secunia's numbers can be used intelligently by others who know how to correctly interpret the data. His criticisms of Ou sound correct to me, but I don't care for all the extremely harsh

Re:

[mr_mischief (456295)]

So you're saying "one vulnerability" on Windows which effects every program running on the OS being less significant than "100 vulnerabilities" in which different applications you might not have installed being exploitable, listed by each string of values that as input could be used in an exploit, is not "at best a silly form of pseudomathematics"? Perhaps you missed the point of the word as used.

About Secunia

[Noryungi (70322)]

Does Secunia present slanted information?

No, it just lists vulnerabilities. But it also lists them AND presents these two important things: (a) the importance of the vulnerability, and (b) whether or not it can be triggered through the network or not (local/remote vulnerability).

Furthermore, it separates Windows vulnerabilities in system and application vulnerabilities, if memory serves well. It's not able to do that with Linux, since different Linux distros incorporate different applications.

The matrix therefore becomes a lot more complicated. You can have a 'local only' problem (meaning: no remote exploitation) which can be considered as 'critical' on some Linux/BSD systems and not on others. You can have a remotely-exploitable problem which is critical on all systems that have application XYZ installed. But if I don't install XYZ (or if it's not activated by default) on my PC, I don't have a problem. And so on and so forth.

Which is why people that point at Linux/Mac and say: "Aha! More insecure than Windows!!" are not truly honest: I have Linux and OpenBSD machines with up-to-date SSH servers, no users, a good password, and no other network service running. These machines are almost perfectly secure -- except when it comes to an OpenSSH vulnerability -- even though there are plenty of applications on them that could be considered obsolete or vulnerable... if you can gain local access in the first place. The only point of vulnerability is OpenSSH. And I update it religiously.

All in all, don't blame Secunia: blame people (especially journalists) who know nothing about security and jump on meaningless numbers pulled out of thin air to blame Linux.

Re:

[thePowerOfGrayskull (905905)]

I have Linux and OpenBSD machines with ... no users... These machines are almost perfectly secure

Erm, aren't all machines perfectly secure when you take the users out of the equation?

Re:

[sumdumass (711423)]

No, Not perfectly. Earlier versions of windows could be exploited without any interaction of any user at all outside the author of an automated virus. Even things in Linux could have the same types of vulnerabilities. Although, it is rare to see automated programs that could exploit them with no user interaction and then replicate and launch another attack somewhere else. This seems to be a windows only thing.

If you said that removing the user removes a significant portion of the vulnerabilities, then you w

Re:

[thePowerOfGrayskull (905905)]

Hint: without a user, the machine doesn't get turned on ;)

Re:

[josephdrivein (924831)]

All in all, don't blame Secunia: blame people (especially journalists) who know nothing about security and jump on meaningless numbers pulled out of thin air to blame Linux.

Except the same meaningless numbers were used to push FF against IE. I recall the "More secure" slogan.
But it's been a while since the last time I heard it. Malice suggests that those numbers aren't very useful to FF lately.

Disclaimer: I'm a Linux user and I use FF regularly.
For what it's worth, I don't wish to start a flame war, but I think we should attempt to be fair.

# of Vulnerabilities!=Acknowledged Vulnerabilities

[Foofoobar (318279)]

Number of vulnerabilities in a product is not the same thing as the acknowledged number of vulnerabilities in a product. Secunia reports on the number of acknowledged vulnerabilities. Microsoft is known for NOT acknowledging vulnerabilities even though they have been reported to the company and then SUDDENLY fixing them in a patch.

And then unfortunately, their supporters like to bash Linux and Mac for actually working with security agencies and fixing their bugs as well as reporting them. This will forever be the bane of open source and it's benefit... that everyone gets to see its flaws but at the same time, everyone gets to contribute to fix them.

Re:# of Vulnerabilities!=Acknowledged Vulnerabilit

[tb3 (313150)]

I'll go you one further; vulnerability != exploit. Show me a tally of exploits in the wild, or better yet, exploits that aren't proof-of-concept. I don't think you find a single one for Macs or Linux, while the number of dangerous exploits for Windows numbers in the tens, or even hundreds, of thousands.

Re:

[Foofoobar (318279)]

Show me the TOTAL number of vulnerabilities reported (not just the ones acknowledged) vs exploits reported and unreported. That's the problem. They do not like to report their vulnerabilities and large companies using their product do not like to report the vulnerabilities (bad for business you know). So these botnets just keep growing mysteriously for unknown reasons off the Microsoft backbone (Yes, 99% of Microsoft machines are being used for the botnets with Linux machines being used as the master nodes)

New Math

[huckamania (533052)]

So 1% of Microsoft machines are not being used for botnets?

I must be exceedingly lucky cause I have a few Windows boxes and they aren't part of any botnet. I did have one that got owned pretty bad this year, but it's now running Suse while I figure out if I want to fix the Windows partition (yeah, it was that bad).

Re:

[Foofoobar (318279)]

Yeah apparently. It's a pretty enormous figure. There is more than one botnet and Microsoft machines being ubiquitous on desktops around the world, you gotta figure that the their is probably more than 1% that researchers don't see behind firewalls and that the number is OBVIOUSLY an exaggeration but even with that it would mean that millions are still safe machines.

Still how many home users do you know that run as root? That run without updated antivirus? without ANY antiovirus? that open attachments? E

Re:

[huckamania (533052)]

If you are part of a botnet, they will notice something. 99% of botnet controlled computers may be Microsoft machines, but there is no way on this green earth that 99% of Microsoft machines are part of a botnet.

Corporations don't let botnets exist on their infrastructure for long, neither does the government and military. Even my ISP will deny you access if you have an infected machine.

Sorry, just don't buy the math.

Re:

[Foofoobar (318279)]

If you are part of a botnet, they will notice something

Yep... every single owner of a computer that is owned and part of a botnet knows that they are part of a botnet. They are 100% aware that they are part of a botnet. Now who is being naive?

Re:

[daviddennis (10926)]

Of all the Windows computers I have seen in the wild, the only ones that are not full of spyware and virii are those managed by professional computer people.

The problem is that most people think of their computer as an appliance and have no real understanding of how to use anti-virus/spyware software, even if they have it. The software itself seems to be better at trying to bludgeon people into buying upgrades than it is at actually doing anything to protect systems.

I do not think computer and operating sy

Fishing for vulnerabilities

[pongo000 (97357)]

Is Secunia presenting slanted information with the expectation it will be misused?

Here's one even better: We use GeSHi [qbnz.com] (Generic Syntax Highlighter) in WikkaWiki [wikkawiki.org]. We often scour the so-called "security vulnerability" databases because we've found many inaccuracies. In this specific case, Secunia issued this statement:

> we noticed the following entry in the changelog for GeSHi 1.0.7.18 and
> are about to issue an advisory based on this information.
>
> "Committed security fix for htmlspecialchars vulnerability. Also makes
> supporting multiple languages a lot easier"
> http://sourceforge.net/project/shownotes.php?release_id=489035 [sourceforge.net]
>
> To serve our mutual customers best we would appreciate to receive your
> comments on this issue before we publish our advisory.

WTF? This was a vulnerability in PHP's htmlspecialchars() function, NOT GeSHi. Yet, Secunia was planning on milking this vulnerability in order to boost its "vulnerability count" at the expense of a project that had absolutely NOTHING to do with the vulnerability.

You see, these so-called "vulnerability experts" try to wring out as many vulnerabilities as possible, because we all know that the most effective "vulnerability expert" will be the one with the most posted vulnerabilities. So they go on fishing expeditions to uncover vulnerabilities that really don't exist.

Or an even worse practice: "bottom-fishing" changelogs and bug trackers in order to discover vulnerabilities that have already been addressed. Here's another instance where Secunia was caught trying to boost its street cred through disingenuous reporting: They apparently scoured our bug tracking database and discovered an issue (already fixed!) and falsely implied in their report that the content of wiki pages marked private might be accessible via RSS. This was clearly false, as the original bug report indicated that the page name (not content) could be accessed. Secunia later corrected [secunia.com] the false report.

We've caught Secunia doing this on several occasions. My advice to anyone who is involved in an OSS project is to regularly scour the vulnerability databases and challenge each and every advisory that you believe is not accurate. You might be surprised at the amount of so-called "vulnerability intelligence" out there that is blatantly false, outdated, or inaccurate.

Re:

[Bill, Shooter of Bul (629286)]

Yeah, but if the htmlspecialchars was exploitable in geshi, then it was a vulnerability in geshi. You can't ignore vulnerabilities inherit in the language you use. If it was exploitable in geshi, then you in turn exposed the users of geshi to the vulnerability by incorporating the function into your implementation. I mean imagine microsoft claiming that buffer overflows were not its fault, as they were really vulnerabilities in C, not windows/explorer/office ect.

Re:

[pongo000 (97357)]

Then using this logic, it would be appropriate and fair for Secunia to list every project that is using PHP with the tainted function. Hundreds? Thousands? Tens of thousands? Where are those vulnerability reports?

Again, this goes back to my argument that Secunia simply cherry-picks its reports, penalizing those projects that are most open with their changelogs and issue tracking, often listing so-called "vulnerabilities" after said vulnerabilities have already been addressed (as in this case).

Re:

[Bill, Shooter of Bul (629286)]

Yes, I agree. Although, they should give more attention to more widely used products, than obscure ones. Ironically, that may lead to a cycle of people abandoning well known products and adopting the other lesser used one, if the only metric is listed vulnerabilities.

And warning of vulnerabilities that have already been patched is legitimate, IMHO, as many people will not always use the latest version and they would still be at risk.

Re:

[Martin Blank (154261)]

Or an even worse practice: "bottom-fishing" changelogs and bug trackers in order to discover vulnerabilities that have already been addressed.

I'm not sure that this is necessarily a bad thing, as people with far more time than I to look for how to make trouble for others are doing exactly the same thing.

If I'm running foo 1.3.2, I may miss that 1.3.3 came out, or may disregard it if I don't think it's imperative that I update, watching for 1.4 to come out. There are a lot of disparate systems that I have t

Said it a thousand times.

[catwh0re (540371)]

..well maybe not a thousand times, but maybe I should. Security of software isn't just a product of how many flaws found. Rather it's an equation of how many people looking for flaws, the nature of the flaw and the reluctance of the company to report it (rather than just silently patching it, or worse just removing the evident symptoms but not the flaw at all.) We all know who I'm talking about with each argument.. Open source, where all changes are viewable, listed (and so on) is much more trustworthy than completely private software where the public discretion comes about from a marketing department. Additionally where the seriousness of a flaw can be completely downgraded by sole discretion.

Again and again and again

[RAMMS+EIN (578166)]

We keep hearing this again and again and again.

It's very simple, really.

You can _never_ know the relative security of two systems. There simply isn't any way to measure it fairly.

Count disclosed vulnerabilities? What about the vulnerabilities that weren't disclosed?

Have teams search for vulnerabilities and compare the results? What does that tell you? Was one team equally good at finding vulnerabilities in one system as the other was at finding them in the other system? What if one system had many easy to find vulnerabilities, and the other had a couple of severe but harder to find vulnerabilities?

Count actual break-ins? Well, was that due to the system being vulnerable the way the vendor left it, or because of the administrator? What about break-ins you don't know about?

It's always a matter of what you don't know about. You don't know the vulnerabilities that weren't reported. You don't know the vulnerabilities that weren't found. You don't know the relative skills of the teams you used. You don't know if you tested for all possible classes of vulnerability.

And I haven't even mentioned the severity of vulnerabilities, the availability of exploit code, the way vulnerabilities are dealt with by the vendor, and a host of other issues.

The take home message is that you just _can't_ know. It's a hard pill to swallow, but you will just never know which system is more secure. All you have is flawed metrics and your gut feeling.

Re:

[RobBebop (947356)]

You can't know for sure, but you can get a pretty good idea fairly quickly using software testing before any release of the product. Your mileage will vary from application to application, but any self-respecting release team will have a set of regression tests to run through to at least give them confidence that they aren't releasing software that has previously discovered bugs in it.

And using a path-analysis software like GCov you can get a feeling that a large body of your code is actually being exerci

Where is the MacOS X malware?

[Zott and Brock (1204632)]

Windows has hundreds of thousands of known viruses and trojans, but the malware for MacOS X can be counted on your fingers. Just because Apple periodically publishes security updates doesn't mean that these vulnerabilites have ever been found outside of security labs and been exploited in the wild.

Re:

[eli pabst (948845)]

That's not really any better metric for comparing security either. OS X has had plenty of security vulnerabilities which have had the potential to be exploited by malware or worms. All that's really required is a remote vulnerability that allows an attacker to upload and execute code. Viruses and trojans are even worse of a means of comparison because most simply rely on tricking a user and don't need any kind of software vulnerability.

Nothing to see there, move along

[Aaron Isotton (958761)]

When I read the summary, I thought TFA could actually be interesting. But it's not any better than what it is criticizing.

Long story short:

ZDnet published an article comparing Secunia vulnerability counts in Mac OS X and Windows Vista/XP. They spun it the Microsoft way, so Mac OS X loses big time. A mac fanboy wrote a reply spinning it the Apple way.

TFA starts with a long-winded attack against the author of the ZDnet article without ever getting to the point. Let's just say that it talks about Zunes, XBoxes, train wrecks, ballet dancing and many more things.

Then it explains what Secunia does (in about two pages): they track software vulnerabilities which are - among others - reported by the vendors. So "honest" vendors get higher vulnerability counts. Who would have thought.

On it goes by saying that the "border" of an operating system is nowadays blurry; should the vulnerabilities in bundled applications be counted? Even if they are by another vendor?

Then he babbles about how most of the cited vulnerabilites in Mac OS X are related to what he calls "external software" - things such as python, java, perl, samba, tcpdump etc and that those same programs have the the same (or a similar) amount of vulnerabilities on other platforms. What he fails to point out is that Mac OS X *consists* of such "external software" for a big part, and that they are *part* of Mac OS X and cannot be removed easily.

Conclusion: a pointless (and extremely long-winded) article full of Microsoft bashing, as reply to an equally pointless article full of Apple bashing.

the only way to legitimately test this...

[buddyglass (925859)]

...is to construct a real-world test and repeat it fairly often, then tally up how each OS performs. Create a monthly or bimonthly hacking "tourney" with a money purse to properly motivate the contestants. Get "normal" IT staff (i.e. not experts hand-picked by MS or the OSS community) to "secure" the competing operating systems, then let the hackers loose.

Unfortunately this only gauges vulnerability to remote exploits, which probably aren't the most common means of penetration and which both systems prob

Vulnerability Counts: Humorous, Not Useful

[MattW (97290)]

Even if the information about vulnerability counts were pristine, it still wouldn't be useful, and anyone who has been involved in security knows it.

Over the years, there's nearly one flaw in the methodology for every one of these surveys ever released:

* Counting vulnerabilities in services installed by default the same as a service that is optional and not frequently enabled
* Subjective rating of impact (mild/severe)
* Treating remote code execution the same when on one system it is as uid nobody, and on the other, it is as administrator
* Ignoring the ease of use of tools that can actually verify a system's integrity (e.g., tripwire with signatures on RO media
and booted off CD)
* Ignoring what a user may have to do to trigger a vulnerability (ie, visit a web page with a malicious image, vs downloading a dmg file, running an install, and giving your password to elevate to root)
* Ignoring how an operating system enables or discourages user stupidity (ie, hordes of useless, "This program wants to do something, yes/no?" vs rare requests for a password)

And on and on and on. The average PC has over 25 different pieces of Malware installed. I know dozens of people with macs, and I don't know anyone who has had a single piece of malware, ever. I've been running linux for 12 years, desktop and server, and I've had two compromises ever, and both were via wu-ftpd.

Local security is good, but...

[argent (18001)]

Largely agree with you, but...

Treating remote code execution the same when on one system it is as uid nobody, and on the other, it is as administrator

Local security does need to be considered, but it shouldn't be depended on. A remote code execution vulnerability is still critical, whether it happens as LOCALYSTEM, root, Administrator, local user, nobody, or in a partial sandbox like a chrooted environment or Microsoft's new sandbox in Vista. Local privilege elevation attacks to exist, and even without priv

Re:

[BasharTeg (71923)]

What's funny to me is, Linux kiddies* and Mac fanboys have used # of vulnerabilities to claim how much more secure Linux and Macs are compared to Windows for years. Then when the empirical count of vulnerabilities no longer favors the point they're trying to prove, we get a thousand angry fanboys posting about how stupid the method is. It's blatent hypocrisy.

Linux fanboys* used to do the same thing with narrow performance benchmarks, showing how much faster Linux was than Windows. Once the benchmarks sta

Re:

[DECS (891519)]

You seem to be confusing actual live exploits that cause Windows to fall apart, hand control over to botnets, plague the world with spam, install spyware, etc, with vulnerability reports.

Windows does own the market for actual viruses, adware, botnet membership, spyware and other problems.

Secunia and George Ou are publishing numbers of vulnerabilities that suggest the opposite is true. But it's obviously not.

You can try to muddle those two ideas together, and you reveal your bias by describing my outlining o

Re:

[51mon (566265)]

It is a common trait to want to reduce everything down to a single number, or something easily compared, especially when most folks have only a very vague definition of the area being compared.

Everyone wants to validate their own prejudices (and some are paid to support other folks interests).

Security is a process, the goal of which is to protect something (usually your data - maybe your hardware - maybe availability or even user sanity!) and (usually at least) to minimize the resources it takes to do it. Y

How about George Ou sucks?

[GarfBond (565331)]

This is the same guy who (figuratively) fell in love with David Maynor and their associated AirPort exploit back when everyone else was telling them to show the goods. The guy isn't much more than an Apple troll - go through his archives (but don't actually - that gives him advertising hits) and it basically reads as "Apple sucks at this, Apple sucks at that, wah wah wah."

See here [cnet.com] for a brief recap of Ou's idiocy (not a word but still).

My experience with Secunia

[Aram Fingal (576822)]

At one point, I looked over all the Secunia advisories about OS X and came across one which said that OS X would send passwords in clear text without warning when logging into Appleshare volumes and that this vulnerability was "unpatched". I thought this was strange since I had, in fact, seen such warning dialog boxes in OS X. It was in an unusual case where I was connecting from OS X 10.2 to an old 68k Mac running MacOS 8.1. I also remembered seeing that there is an options button when you make an Appleshare connection. If you hit that options button, you get a screen with check boxes for allowing clear text passwords and warning when a clear text password is needed. The default is to allow with a warning. I sent email to Secunia asking for clarification about what circumstances would lead to sending a clear text password without notice. Do those check boxes not actually work? Are the defaults less secure in some cases? I never got a reply but the issue disappeared from the Secunia site. No explanation. Just gone. I wonder if enough other issues have just disappeared to affect the numerology.

never attribute to malace...

[v1 (525388)]

It'd be nice if some international body examined the issue of software security risks and established a guideline so we didn't have this ongoing problem of what to call a bug and what not to, and to finally put to bed the notion that notifying users of newly discovered vunlerabilities is bad for security.

I for one would like to see a rating scale that factors in not just the problem, the severity, and the scope, but also the availability of information on the problem. For example, you couldn't score anywhe

I've never read that site before

[spun (1352)]

And I never will again. He's as bad as the people he is criticizing, if not worse. He does exactly what he accuses the "Microsoft shills" of doing. From another article on the site: "I explained that he could just drag the application to the trash, and that in the Mac OS there are no DLL files to worry about."

Riiight. Mac OS doesn't have libraries. There are no possible library mismatch issues on Mac OS. Okay, buddy, whatever.

Re:

[Jeremy Erwin (2054)]

And I never will again. He's as bad as the people he is criticizing, if not worse. He does exactly what he accuses the "Microsoft shills" of doing. From another article on the site: "I explained that he could just drag the application to the trash, and that in the Mac OS there are no DLL files to worry about."

Riiight. Mac OS doesn't have libraries. There are no possible library mismatch issues on Mac OS. Okay, buddy, whatever."

his original post [roughlydrafted.com]

Look, have you ever used a Mac? Shared Libraries are versioned.

A

Re:

[spun (1352)]

Yes, I've used a mac. And libraries are versioned under all operating systems. That doesn't always help. Sometimes minor version numbers aren't minor. The interface changes (when it shouldn't for a minor number change). Old bugs are fixed, breaking things that depend on them. New bugs are introduced.

Bundles are great, but we are talking about shared libraries! If each package has its own version of the library, they aren't shared, are they? If each program loads it's own version and doesn't look at what is

Re:

[squiggleslash (241428)]

Firefox users might find TheRaven64's [slashdot.org] advice useful WRT to making it a little easier to identify RD articles.