Slashdot Log In
Can You Trust Anti-Virus Rankings?
Posted by
CmdrTaco
on Thu Oct 23, 2008 08:38 AM
from the of-course-not dept.
from the of-course-not dept.
Slatterz writes "It seems nobody can agree on a universal set of tests for rating anti-virus software, with Eugene Kaspersky the latest to weigh in on the topic, criticizing the well-known Virus Bulletin 100. Kaspersky is one of several big anti-virus brands to fall foul of the VB100 tests, reportedly failing to pass a recent test of security software on Windows Server 2008, along with F-Secure and Computer Associates. At Kaspersky, bloggers have pointed out that they don't focus on detecting PoCs, calling it a 'dead end,' and saying their anti-virus database focuses on 'real threats and exploits.' 'I don't want to say it's rubbish,' Kaspersky told PC Authority. 'But the security experts don't pay attention to these tests. It doesn't reflect the real level of protection.'"
Related Stories
Submission: Can you trust anti-virus rankings? by Anonymous Coward
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
No. (Score:2, Funny)
Re: (Score:3, Interesting)
Ok. Then what can we trust?
The guy himself pointed out the issue at the end of the interview:
"The problem is that in the industry there's no other complete tests," says Kaspersky.
Without some sort of test, however imperfect, how is the average home user supposed to choose?
Re:No. (Score:5, Insightful)
It's "backwards", in that you don't tell them what is "good". They try to guess what would be on your "bad" list. As everyone here knows, it turns out that the "bad" list is much, much longer than the "good" list. In 2007 alone, F-Secure added more virus sigs to their products than the totality of sigs accumulated from the previous 20 years! And last I heard from them, 2008 was projected to double 2007. That sounds almost like quadratic growth to me
It's "blind" in that they aren't seeing what is actually running on your computer. For privacy (and performance) reasons, nobody provides metrics back to AV vendors about all of the executables that weren't labeled "bad", and rarely do the metrics about what is labeled "OK" actually go back to them. The AV vendors have to take a shot in the dark. They can simulate what they think your computing environment looks like, but it's just a guess. They cannot know if you have custom or proprietary software that matches one of their AV sigs unless they actually test that particular program against their sigs (and you don't let them do that, hence the "blind" remark).
Backwards and Blind is very problematic. Every once in awhile, we hear about fiascos like Symantec deciding an asian language DLL is a virus, killing all of their asian customers' windows installs for a day or two.
The question the benchmark is really trying to answer is: Which vendor's product is best tuned for the least amount of false positives and false negatives? When we should really be asking the question: Do I know what is good to run on my computers? And if the answer to that is "yes", then we should be asking the question: Why can't these vendors make a product that only allows my "good" programs to execute and nothing else?
Parent
Re:No. (Score:5, Insightful)
Do I know what is good to run on my computers? And if the answer to that is "yes", then ...
The problem with that, of course, is that the answer is "no" for most people.
Parent
Re:No. (Score:4, Interesting)
Indeed; nor should we expect them to. The vast majority of computer users want to use the computer in the same way that they use any other appliance; and frankly, they /should/ be able to. Unfortunately, the only way to give them that experience is to a) line up all malware authors and shoot them; or b) provide them with locked-down machines that can only run Authorized Content in an Approved Manner.
The problem with that is we've just spent the last 20+ years going through massive innovation because there's no particular approval to how this tech is used. Bolting on Approval could have ugly effects. Unless, of course, that approval is from the end user. Which puts us in the same place we are now.
The other issue is that we're not dealing with a toaster. Nobody expects their toaster to also become a calculator, telephone, and TV on demand. We're dealing with a complex and powerful machine. A computer is not a toaster (or a truck - but I digress).
That doesn't mean we shouldn't be trying to simplify the tech. After all, an automobile is also a pretty advanced piece of machinery as well. But the key to this is making really intelligent and sufficiently paranoid choices on how to go about doing this so the end user doesn't have to. Part of the problem is that some aspects of the industry like to portray their products as toasters while making poor design choices; a customer base of monkeys with machineguns.
Parent
Re: (Score:3, Insightful)
And if the answer to that is "yes", then we should be asking the question: Why can't these vendors make a product that only allows my "good" programs to execute and nothing else?
Because such a product wouldn't need to be updated every year or require monthly subscriptions.
Re:No. (Score:4, Insightful)
Actually as someone who has been working in Win PC repair more years than I can count,I'd say the biggest problem would be a simple fix for MSFT,but for some reason they haven't. And that is that file extensions are all or nothing. What I mean is this: either they can see file extensions,in which case the user can fuck up EVERY single file they touch,because it lets them wipe the file extension when they go to rename the file. Or you can't see the file extensions,in which case the nontechnical user get bit by the "OMG watch Britteny suck teh titties!".avi.exe malware.
There should be a way to show file extensions but not change them unless you right click and explicitly choose "change file extension for this file" which would give the user a warning,like "This can cause the file not to open correctly. Are you SURE that you want to change the file extension?". If you did that,a whole damned lot of the infected machines that cross my desk weekly wouldn't be filled with malware. I don't suppose anyone knows of a freeware solution that does what I just described,do you?
Parent
I'm with Kaspersky (Score:5, Insightful)
Game over.
Re:I'm with Kaspersky (Score:4, Informative)
The real fun tho is when I run WAR it detects 'keylogger like behavior' from the software. Heheee.
Parent
Re:I'm with Kaspersky (Score:4, Funny)
Yeah, I've been pushing for Kapersky for a long time on my server, but The Dell Rep says that Symantec's is The Best AV Software out there. And he is clearly more knowledgeable about such things than a server jockey like me.
Parent
Re: (Score:2)
If it didn't have so many false positives I'd agree with you.
However Kaspersky seems far and away the most prone to them.
From random image false positives, to objecting to "hacking" tools, otherwise known as network discovery tools...
--Q
Re:I'm with Kaspersky (Score:5, Funny)
I don't care about tests either, I only care about anecdotal evidence in random /. posts. If Kaspersky worked for this one guy, it's good enough for me.
(Actually my only anti-virus protection is not using IE, and not running things that shouldn't be run. I've had no problems.)
Parent
No more.... (Score:4, Interesting)
than I can trust the hackers that write these damn viruses that keep infecting my PC! Yeah, standards in this industry would be a start in the right direction, but right now ANY virus protection software is better than none!
I use Norton Internet Security, and while it is passable, I find that it's a resource hog. I know there are other products out there that are less "intrusive", but I just don't want to take the chance (or time) with another product.
Re:No more.... (Score:5, Insightful)
Parent
Re: (Score:3, Informative)
Re:No more.... (Score:5, Insightful)
Wow, solid, well supported argument right there.
Indeed, it is. Norton really is a load of crap. It is a resource hog of cpu, memory and hard drive. I believe the only reason it is found on anyone's PC is because Norton pays PC companies to install it by default. Because, frankly, you would have to literally know nothing about AV to choose Norton. As in, you did no research and picked the shiniest box off the shelf. At which point, I have lost sympathy for the user.
My company relies on SOPHOS. In 12 years of working with SOPHOS, never has a virus had a chance to spread...despite the users best efforts.
Parent
Re:No more.... (Score:5, Informative)
Correction:
The reason Norton is on any PCs is because Norton pays PC companies to install it by default AND IT IS ALMOST IMPOSSIBLE TO REMOVE.
Cleaning viruses off by hand is easier than uninstalling Norton.
Parent
Re:No more.... (Score:5, Informative)
May I recommend the Norton Removal Tool [symantec.com]
It shouldn't need to exist in the first place, of course - the uninstall should work - but IME it works pretty well.
Parent
Re:No more.... (Score:4, Interesting)
Norton is ... ALMOST IMPOSSIBLE TO REMOVE.
Which I found especially hilarious/frustrating when I was required to upgrade the version of Norton on a bunch of lab computers. The upgrade wouldn't work, and told me I had to uninstall the previous version. Turns out uninstalling the previous version was unbelievably difficult. The auto-uninstall didn't work. The Norton removal tool didn't work. Finally I had to follow a series of manual step-by-step instructions about what files to delete and what registry keys to modify.
And after all this pain and suffering to remove Norton... I had to install a new version. (That I knew would be a pain to eventually uninstall or upgrade.)
Needless to say I now avoid Norton like the plague. Yet I would argue that Norton/Symantec is widespread not only because of default installs--but because they seem to do a good job marketing to the higher-ups. They win large-scale deployment contracts, where the software annoys end users and many admins, but looks good and secure on paper, I guess.
Parent
Re:No more.... (Score:5, Funny)
Oh come on who are you kidding? It is easy to remove:
1. Log in as administrator /s Symantec /s Symantec
2. Open command prompt
3. cd \Program Files\ and rmdir
4. CD Common Files and rmdir
5. Open the registry and go to the SERVICES key and delete all the Symantec services
6. Open the registry and go to the RUN key and delete all the Symantec entries
7. Reboot
8. Install and run ccleaner, run the registry tool and let it clean up the now-broken library registrations
9. Use the uninstaller tool in ccleaner to remove now-broken uninstallers (that don't really clean up Symantec's poop trail ANYHOW)
10. Now try removing the directories again (steps 3 & 4) to remove the remaining Symantec poop
There, now Symantec PoopWare is now completely uninstalled. Now, wasn't that easy?
Parent
Re: (Score:3, Informative)
>6. Open the registry and go to the RUN key and delete all the Symantec entries
>7. Reboot
Norton likes to hook into stuff like the ATAPI drivers. If you kill all of the Symantec registry entries, neither Windows XP nor vista will be able to start. Easy fix with Vista, but on XP you're just boned. I know this from personal experience.
Just use the Norton Removal Tool provided by Symantec. It works really well, assuming your Norton isntall isn't completely FUBAR. If it is, well, you were probably due for
Re:No more.... (Score:5, Insightful)
So Norton finally got their act together with the 2009 version? Good for them. But, they have a long road to travel to fix the perception that their product is bloated. Such a history is difficult to change overnight.
Parent
Your brief review is possible shilling (Score:4, Insightful)
You do realize that's it's possible, albeit likely Norton encouraged them to write the review?
I believe this is tangent to the point of the /. article: not only are tests flawed, but you should inherently not trust any major news source to unbiasedly review a product.
- Why do they only compare it to Kaspersky?
- Why do they mention ram but not a speed comparison (I'd gladly give up 15mb of more ram just to have better performance in my AV, ram is dirt cheap)
- If NIS2009 is so "lite", why don't they mention the specs in comparison to older NIS (only Norton would want to cover up their old specs, which is a core issue that makes me suspect this is a shill article).
Not to mention I never trust any online news source, including tech sites, to have somebody savvy enough to know how to test an AV properly, which, as the /. article points out, not even the AV "experts" have figured that out, much less some tech site.
Parent
Re: (Score:2)
[Your implicitly suggested alternative] is an utter piece of crap, it would be advisable to get rid of it now.
Citation required.
Re: (Score:3, Informative)
Common knowledge generally doesn't require a citation.
Re: (Score:3, Insightful)
Re:No more.... (Score:4, Informative)
Norton is itself a virus. It hogs resources, causes errors, and can't be removed without killing the host.
For what you pay, you should get something that is better than cheaper or free products available on the web...I usually replace Norton with AVG, and while I'm not a huge fan of AVG, I've never had anyone complain.
Parent
Re: (Score:3, Informative)
I've had a number of friends say this to me also, and I have been meaning to replace Norton with AVG (after my subscription runs out), but I haven't been able to get off my lazy ass and do it!
I've had a good experience with Norton over the years, but recently the quality of their product (read: quality sucks now!) has gone way down. For me, I first noticed it when they removed parental control from their antivirus product, and made it a free "add-on" that you had to install separately. WTF??? Why did you re
Re:No more.... (Score:5, Funny)
It doesn't spread, so it's not a virus. More like a cancer. Or a birth defect, if it comes pre-installed.
Parent
Re:No more.... (Score:5, Funny)
but right now ANY virus protection software is better than none!
That depends, do you walk around all day with a rubber on your weiner? No? Newsflash, niether does your computer, so stop putting it's dick everywhere.
Parent
Re: (Score:3, Interesting)
Would you consider using ZoneAlarm for your software firewall (or get a "hasbro" level appliance for home if you don't have one and don't bother with a software firewall if the PC isn't mobile), and then a F/OSS AntiVirus package that does AntiVirus and ONLY antivirus? If so, then check out Moon Secure AntiVirus. [moonsecure.com] I run it on my Vista installation (which exists for gaming).
On Linux, I don't worry about it. In fact, I submit bug reports to malware authors complaining that their crapware doesn't run on WINE
Tests need to evaluate _something_ (Score:5, Informative)
Take crash tests on new vehicles. Name me one that doesn't have a 5-star crash rating? The rating system is too easy, and needs to constantly be moved to achieve a new level of betterness. Not everybody should get A's. Once the majority of players reach a standard, the standard should be moved to motivate advancement in the field and show the better of the pack.
For example, the 5-star front-impact crash rating is par for the course now... but nobody seems to advertise the offset crashes, such as the right half of your bumper hitting the left half of your 'opponents' bumper. Why? Because it's sad in comparison. It's also not pretty to watch.
So all the power to making the standards hard to achieve. Yes this may not be the 'real world' threat, but it's a threat nonetheless. They're basically saying "Since England isn't going to declare war on the USA, any preparedness for receipt of an attack by the USA shouldn't be considered in overall military preparedness". That's of course rediculous. Protect only against the popular virus and the unpopular virus will begin to spread.
Re:Tests need to evaluate _something_ (Score:5, Insightful)
Parent
Understand first, then pick sides.... (Score:3, Interesting)
That's why I (Score:4, Interesting)
Bullet proof? Of course not.
So far with Avast, AVG, (mind you one virus product per computer only) ZoneAlarm, FireFox, and some basic sense I haven't been hit.
My only issues (sad enough) is when a windows update broke Zone Alarm and when AVG detected Zone Alarm as a virus (cause a new version came out) and shut it down.
Now that i really think of it all the products designed to protect me have been the ones giving me all the trouble. HAHAHA (as I cry)
Re: (Score:3, Informative)
Re: (Score:3, Interesting)
The new version of Kaspersky and couple of other vendors who spends money to development instead of animated ads tries to go with "white list" approach.
For example, while it does very suspicious things (due to its function), Zonealarm is very known to the AV solution and once it is surely the ZA it trusts, it won't bother with it too much UNLESS it starts doing things which it isn't known to do. It adds lot to the performance and Kaspersky is the last vendor to blame about heuristics since its early version
Trust anti-virus ratings? (Score:3, Insightful)
I'd just like to be able to trust anti-virus software.
http://arstechnica.com/journals/apple.ars/2008/10/20/mac-malware-program-macguard-masquerades-as-antivirus-app [arstechnica.com]
I'm getting really paranoid about things. I find myself avoiding any web service that wants me to download a app or plug in I'm not very familiar with.
Not a fan (Score:2, Informative)
PoCs (Score:2)
but couldn't resolve "PoC" without RTFAing.
WTF is this, some kind of trick to make us read TFA?
Re: (Score:3, Informative)
Proof of Concept; sad, but in Securityville this is actually used often enough that it would be considered a "normal" acronym. The debate usually revolves around the fact that a lot of PoC's are completely esoteric and can't be made into actual workable mass-market exploits.
Re: (Score:3, Funny)
Is there an acronym for "woooosh"?
IMHO: no. YMMV.
HTH HAND
8^)
Open Source and Free (Score:2)
I've had good luck with a combination of Firefox with the No Script addon and Clamwin, and maybe just a little common sense.
industry created whole (Score:3, Informative)
Proof of concepts are tangible vectors to infection. By not including and rigerously detecting such methods, they AV companies will allow more viral products into the market. This is a very self-serving stance.
I actually see problem of trust emerging. Once upon a time KAV was a brilliant peice of software that ran in DOS well enough to remove the plague of Win95 Marburg infections that hit the UK gaming community after a bad cover CD. That was a time when viruses existed, and you had to stop them infecting you. The prospect of new and novel viruses infecting you wasn't really an issue as home Internet penetration was small. As such, AV software wasn't marketed as the only thing you needed to stop all viruses forever, but as a tool that will detect more than its competitor more reliably. The money you paid was for a good huristics engine that was fast, efficient and more importantly, updated reguarly.
Now I see AV products as nothing more than 'ineffective-ware'. If AV programs claim to prevent the infection of known viruses, and reduce to risk of infection from emerging viruses, I'd probably have more faith in the industry. But they don't... in subscribing the "we can protect you from everything" marketing hype, almost every AV company has asked us to put faith in their product to stop "unknown" viruses... and we expect them to.
They don't. It's a computational nightmare.
KAV are in a past mindset. They have to change. They have to consider that what people really want is reliability - they want software guarantees. If any peice of AV software is going to help the market rather than hinder it, it is going to be reliable. What is the most reliable part of an infection? The vector, not the virus itself.
The truth is really in the pudding. Viruses have changed. Almost all now are polymorphic and highly reentrant. A few lines of code will change a signature making it undetectable. Fnfection is detectable at the point of entry. If the research is put into proof of concept code in making a system vulnerable, then the AV response should be to track and thwart that success.
Matt
Process - Not Product (Score:4, Informative)
Security for me begins with sensible configuration of the router and the PC's on the network, then it moves to access rights and regular patching of said computers.
This includes regular checkups and glancing at logs every three days or so to look for obviously suspicious traffic. Finally, after all of these steps, I use Kaspersky (since I had heard good things about it) together with rootkit detector. (Oh, and Firefox with NoScript)
All of this prevents pretty much all the scriptkiddies from getting in (I hope), but then again, the best thing you can do is to not download anything you don't know what it is.
TLAs (Score:3, Funny)
My guess was that it's a politer version of PoS.
Re: (Score:2)
I had all sorts of troubles trying to install AVG on a heavily infected system. But once I installed it on a fresh system it was fine. If the virus scanner is blowing up, something is probably attacking it.
And yes, running Linux is a lot less of a hassle. And you don't have to buy a new $40-80 AV license every year or so. Also you can install ClamAV on Linux, it's pretty handy if you're allowing Windows users to upload/share files with your Linux computer. Also helpful if you're one of those poor saps that
Re: (Score:2)
That's what happens when you stupify data, you loose data. Anyway Kaspersky don't give a rats ass about any tests, if it was them up there at the top of the list they would have nodded their heads and opened their pockets wide. And I wouldn't be surprised if someone fiddled with the software to the advantage of others, or even worse, fiddled with the logic. The anti-virus industry is ironicly equal to the medicine industry, same overadvertising unnecessary medication using scare tactics. It's simple folks, keep your fucking shit together, don't put your dick wherever it fits and then complain when it falls off because you eat 30 vitamines every day.
Well, that about speaks for itself . . ..
Re: (Score:3, Insightful)
Virus or not, there is plenty of malware out there so it is still prudent to be regularly check your system and be aware of these threats, even on Linux. [c|k
Re: (Score:3, Interesting)
You are correct; as I just told another guy, a trojan will work on any platform, and the only unhackable computer is a broken computer.
Backdoors, trojans, and DoSes are not "technically" viruses any more than a window is not "technically" a door and a screw is not "technically" a nail. And I doubt very seriously that Linux has 300 back doors; I'd be surprise dit it had one. If your source calls a trojan a "backdoor" your source is ignorant.
And yes, it's prudent to be vigilant. But with Windows, vigilance is