Can You Trust Anti-Virus Rankings?

Slatterz writes "It seems nobody can agree on a universal set of tests for rating anti-virus software, with Eugene Kaspersky the latest to weigh in on the topic, criticizing the well-known Virus Bulletin 100. Kaspersky is one of several big anti-virus brands to fall foul of the VB100 tests, reportedly failing to pass a recent test of security software on Windows Server 2008, along with F-Secure and Computer Associates. At Kaspersky, bloggers have pointed out that they don't focus on detecting PoCs, calling it a 'dead end,' and saying their anti-virus database focuses on 'real threats and exploits.' 'I don't want to say it's rubbish,' Kaspersky told PC Authority. 'But the security experts don't pay attention to these tests. It doesn't reflect the real level of protection.'"

I'm with Kaspersky

[LibertineR]

I dont care about any tests, I care about what detects dangerous stuff on my network and what doesn't. Every client I have in on Kaspersky stuff, after Norton, McAfee, Trend and others FAILED to detect viruses that Kaspersky found straight away.

Game over.

Trust anti-virus ratings?

[olddotter]

I'd just like to be able to trust anti-virus software.

http://arstechnica.com/journals/apple.ars/2008/10/20/mac-malware-program-macguard-masquerades-as-antivirus-app [arstechnica.com]

I'm getting really paranoid about things. I find myself avoiding any web service that wants me to download a app or plug in I'm not very familiar with.

Re:No more....

[AceofSpades19]

Norton is an utter piece of crap, it would be advisable to get rid of it now

Re:Tests need to evaluate _something_

[thedonger]

In an unusual parallel, world famous rock climber Chris Sharma wanted to downgrade a rating on a climb - one of the hardest climbs of its type in the world. From what I gather, the reason was that you reach a point where the rating system becomes meaningless as higher and higher ratings are made, and you lose the context in which the previous ratings were assigned, and the foundation on which the rating system is based.

Re:No.

[A non-mouse Coward]

Anti-Virus is outsourcing the problem of deciding what is good to execute on your computer to a vendor who works backwards and blind.

It's "backwards", in that you don't tell them what is "good". They try to guess what would be on your "bad" list. As everyone here knows, it turns out that the "bad" list is much, much longer than the "good" list. In 2007 alone, F-Secure added more virus sigs to their products than the totality of sigs accumulated from the previous 20 years! And last I heard from them, 2008 was projected to double 2007. That sounds almost like quadratic growth to me ... and keeping up with that growth rate is not a game I'd want to play! My list of "good" software doesn't increase on a quadratic growth rate, does yours? If this were any other field of computation, the signature approach would have been laughed off the planet by now.

It's "blind" in that they aren't seeing what is actually running on your computer. For privacy (and performance) reasons, nobody provides metrics back to AV vendors about all of the executables that weren't labeled "bad", and rarely do the metrics about what is labeled "OK" actually go back to them. The AV vendors have to take a shot in the dark. They can simulate what they think your computing environment looks like, but it's just a guess. They cannot know if you have custom or proprietary software that matches one of their AV sigs unless they actually test that particular program against their sigs (and you don't let them do that, hence the "blind" remark).

Backwards and Blind is very problematic. Every once in awhile, we hear about fiascos like Symantec deciding an asian language DLL is a virus, killing all of their asian customers' windows installs for a day or two.

The question the benchmark is really trying to answer is: Which vendor's product is best tuned for the least amount of false positives and false negatives? When we should really be asking the question: Do I know what is good to run on my computers? And if the answer to that is "yes", then we should be asking the question: Why can't these vendors make a product that only allows my "good" programs to execute and nothing else?

Re:No more....

[Ngarrang]

Wow, solid, well supported argument right there.

Indeed, it is. Norton really is a load of crap. It is a resource hog of cpu, memory and hard drive. I believe the only reason it is found on anyone's PC is because Norton pays PC companies to install it by default. Because, frankly, you would have to literally know nothing about AV to choose Norton. As in, you did no research and picked the shiniest box off the shelf. At which point, I have lost sympathy for the user.

My company relies on SOPHOS. In 12 years of working with SOPHOS, never has a virus had a chance to spread...despite the users best efforts.

Re:I really could care less

[Anonymous Coward]

I really could care less

The fact that you could care less than you currently do suggests that you do in fact care. However, this conclusion doesn't quite fit with the general tone of your post. Could you clarify, do you or do you not in fact care?

Re:No.

[thePowerOfGrayskull]

Do I know what is good to run on my computers? And if the answer to that is "yes", then ...

The problem with that, of course, is that the answer is "no" for most people.

Re:No more....

[Ngarrang]

So Norton finally got their act together with the 2009 version? Good for them. But, they have a long road to travel to fix the perception that their product is bloated. Such a history is difficult to change overnight.

Re:No.

[jimicus]

And if the answer to that is "yes", then we should be asking the question: Why can't these vendors make a product that only allows my "good" programs to execute and nothing else?

Because such a product wouldn't need to be updated every year or require monthly subscriptions.

Re:My favorite AV software:

[hAckz0r]

That's very strange. Then someone should go tell VirusList.com that because when I do a query for "linux" I get 1156 hits. Ok, so maybe they are not all technically viruses because the first 306 are classified as backdoors, then came the denial of services, then... I didn't look at the rest because I just got tired of clicking the next page button.

Virus or not, there is plenty of malware out there so it is still prudent to be regularly check your system and be aware of these threats, even on Linux. [c|k]lamav, chkrootkit, and rkhunter are your friends and don't mind working late at night while you sleep. Setting up ipfilter to to default deny for outgoing services is also a good idea. I like firestarter because it lets you monitor what apps are connected to the net on what ports to catch some types of covert channels and back doors.

Re:No more....

[UberMorlock]

Sure you can. Just like a wife would recommend against trusting her husband just because he stopped cheating on her THIS YEAR, but had cheated on her in each of the last 6 years. Just because a change has been implemented does not mean that the change is permanent. Likely, this edition of Symantec is just a temporary reprieve from the all-consuming nature of Symantec products.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Don't use Windows....

[KozmoKramer]

Use Linux or purchase an Apple and your Virus troubles will go away.

Your brief review is possible shilling

[TravisO]

You do realize that's it's possible, albeit likely Norton encouraged them to write the review?

I believe this is tangent to the point of the /. article: not only are tests flawed, but you should inherently not trust any major news source to unbiasedly review a product.

- Why do they only compare it to Kaspersky?
- Why do they mention ram but not a speed comparison (I'd gladly give up 15mb of more ram just to have better performance in my AV, ram is dirt cheap)
- If NIS2009 is so "lite", why don't they mention the specs in comparison to older NIS (only Norton would want to cover up their old specs, which is a core issue that makes me suspect this is a shill article).

Not to mention I never trust any online news source, including tech sites, to have somebody savvy enough to know how to test an AV properly, which, as the /. article points out, not even the AV "experts" have figured that out, much less some tech site.

NOD32 *

[x102output]

NOD32 FTW!