Consumer Reports Creates Viruses to Test Software

Maximum Prophet writes to mention an MSNBC article about a Consumer Reports plan to test anti-virus software by creating viruses. Security companies are objecting, on the grounds that it's a generally accepted practice not to create viruses for any reason. From the article: "Consumer Reports didn't create thousands of new viruses from scratch. Rather, it took a handful of existing viruses and created hundreds of slight variants, changing the malicious programs just enough to evade detection by an antivirus program with a list of known threats. That's a common trick in the virus writing world; it's standard for a successful virus to inspire dozens of variants. "

Of course they are...

[Theaetetus]

Security companies are objecting, on the grounds that it's a generally accepted practice not to create viruses for any reason.

Well, yeah. Plus, you'll expose all the weaknesses in their software. Testing security only emboldens the terrorists!

Re:

[Guysmiley777]

Testing security only emboldens the terrorists!

I wish I still had mod points, that is the funniest thing I've read today!

Re:

[Forge]

So for those who actualy get to read consumer report.

Who won ?

Re:Of course they are...

[Anonymous Coward]

Who won ?

The viruses.

Re:Of course they are...

[Anonymous Coward]

Testing security only emboldens the terrorists!

Why does Consumer Reports hate America?

Re:Of course they are...

[Lulu of the Lotus-Ea]

Plus the fact that the anti-virus companies don't like the competition from Consumer Reports; after all, it's those companies that themselves create most of the "proof-of-concept" viruses to scare potential buyers (especially to create scares of vulnerability on OSX, Linux, BSD, etc... where no real vulnerability exists).

Re:Of course they are...

[Schraegstrichpunkt]

(especially to create scares of vulnerability on OSX, Linux, BSD, etc... where no real vulnerability exists).

The vulnerabilities do exist; they're just not being exploited nearly as much. Of course, run-of-the-mill signature-based antivirus software is equally flawed, as Consumer Reports has shown and security geeks have already known.

Re:Of course they are...

[Bastian]

Of course, this isn't really why they are objecting. Whatever McAfee and Symantec say, writing proof-of-concept exploits seems like standard practise to me. My best guess is that their fear is that this might cut into their profits because Consumer Reports is going to make the non-geek public more aware of the limitations of antivirus software. This could make them decide, "Well, if it can't protect me from all the viruses, especially not the new ones, than maybe it's not worth the money."

Of course, Consumer Reports is almost certainly responsible enough to address this issue and point out to people that it's really a reason why they need to be updating their virus definitions as frequently as is practical.

The real thing is

[Sycraft-fu]

AV software WILL protect you from new viruses... Just not McAfee and Symantec's crap. Well I suppose I should rephrase: Their software can protect you, but not very well, not as well as others. Bitdefender appears to do the best job at finding viruses that it doesn't have in it's DB. AVG also seems to do a pretty good job.

That's what they are afraid of. Not that it will be revealed their software does nothing, it does work, just that there is cheaper software that works better.

Re:The real thing is

[Intron]

If their software defended you from new viruses, why would you subscribe to get database updates? Do you expect them to only sell their software to you once?

Because it's not 100%

[Sycraft-fu]

Bitdefender doesn't catch all new viruses, updates are still important, it's just very good at finding new variants. That's what CR is testing here. Say a virus comes out that your software knows about but a variant comes along that it doesn't yet: Can it catch that? For some (like Sophos) the answer is no never, they check against a database and if it's not there you are SOL. For some like Bitdefender the answer is usually. They have a heuristic checking that works pretty well.

There's no magic bullet, there's no "buy this once and be secure forever" kind of solution, but there are better and worse ones out there. Bitdefender and AVG (probably others those are just the two I know) are reasonably good at stopping new, unknown variants. Synametc, well not so good.

Re:

[Penguin Programmer]

Bitdefender appears to do the best job at finding viruses that it doesn't have in it's DB.

Which could be why it was one of the top-ranked AVs in the CR tests (if my memory serves me correctly; I have the magazine at home).

Re:

[pete6677]

I've seen more machines screwed up by Norton than by any official "virus". You don't need Consumer Reports to tell you that. Why people keep paying money for this crap is beyond me.

Let's call a spade a spade here

[rs79]

All this crap only applies to Windows XP. While is is true that MS-DOS, 3.1 and 98 can be infected by explicitly running an infected program (rare), XP is the only thing you can install, hook up to the net and expect it to be infected withing hours if not minutes.

For the one machine I have at home that has to use winbloze I use 98 and have since, well, 98. Although it has in typical MS fashion shit itself a few times it has NEVER become infected. Not once.

Other than an ill fated XP experiment here briefly t

Re:

[Bastian]

You are well outside the bell curve in your experience with Win98 and viruses.

Re:

[drinkypoo]

XP is the only thing you can install, hook up to the net and expect it to be infected withing hours if not minutes.

Not even close to true, although it is the only current operating system with those characteristics and frankly, if you're installing XPSP2, that's not true either, because you're firewalled by default. Still, I've actually seen it happen to Win2k...

You have made a sp2 slipstream CD, yes?

For the one machine I have at home that has to use winbloze I use 98 and have since, well, 98. Alt

Re:

[paranode]

Testing security only emboldens the terrorists!

And think of all the furry kittens that would die!

Re:

[jc42]

        Testing security only emboldens the terrorists!

And think of all the furry kittens that would die!


Yeah, but think of all the hairy software that's dying out there every day!

Re:Of course they are...

[Hoi Polloi]

I hear the Yale company is still furious over the time Consumer Reports tried a bunch of random combinations on their locks.

Re:

[Hoi Polloi]

Next time I'll include the tags.

Re:Of course they are...

[Hoi Polloi]

Wow, ./ cut off my dummy satire on/off tags.

Re:Of course they are...

[telbij]

Security companies are objecting, on the grounds that it's a generally accepted practice not to create viruses for any reason.

I also had to quote this sentence because it's so silly. It's generally accepted practice by people who don't create viruses. Obviously a lot of people are creating viruses whether blackhat or whitehat or greyhat. Now where's my MAD magazine?

Re:Of course they are...

[vought]

that it's a generally accepted practice not to create viruses for any reason

It was generally accepted practice for 50 years not to crash perfectly good cars. Until we started learning that we could protect the occupants of said cars better by finding out where the weak points were...by crashing perfectly good cars.

What are Symantec. et al afraid of?

Re:Of course they are...

[Jesus_666]

Well, one of these new virii could leave the laboratory and get into the wild. With a bit of bad luck, that virus could be a dangerous mutation - I'm not talking Melissa dangerous, I'm talking H5N1 dangerous. Just one tiny mutation and the virus could jump over to humans, creating a worldwide pandemia as people's immune sytem collapse, unable of keeping up with polymorphic virii that inject their own code into the header of the genetic sequence so that they're uncleanable without working from known-clean marrow. And you know what could be even worse? Worms. If they add a self-propagation mechanism to their new killer virus it would infect random bystanders without the need for a regular infection vector! Those people aren't developing weapons of mass destruction, they're creating doomsday devices! Somebody must put an end to this before it's too late!

Re:

[geekoid]

the same thing the car companies where afraid of when CR started crashing cars, even though they paid for them.
That huge flaw will be found out and consumer will demand change.

Re:Of course they are...

[Anonymous Coward]

> IMHO this tic for tac will go on forever.

Yes, it's one of the French benefits.

Conspiracy!

[susano_otter]

Clearly this is all just a cover. The Templars are using Consumer Reports as a cover to train a stable of elite Black Hat hackers, with which to take over the world. They're in a race against Communist China, the Russian Mob, and the NSA.

Re:

[ScentCone]

The Templars are using Consumer Reports as a cover to train a stable of elite Black Hat hackers, with which to take over the world.

Well, it is a conspiracy, but not the one you think. This is actually about the Masons, who are secretly behind the publishing deal for Dan Brown's upcoming book. I mean, what world-dominating secret society wouldn't want a piece of that action? Once their Masonware attack is launched, all web traffic will go through a link that tacks their affiliate code onto inbound Amazon

Re:

[peacefinder]

Well, technically he was on the right track since the Templars are just one expression of the larger Masonic movement. (Of which Jeff Bezos is, of course, a high-ranking member.)

Re:Conspiracy!

[Just Some Guy]

Even though I'm a Mason (complete with secret decoder ring and everything), I haven't been let in on the conspiracy yet.

On the other hand, ever notice the hypnotic patterns made by the Shriners in their little cars? Did you really think that NO CARRIER

Re:Conspiracy!

[darkitecture]

Clearly this is all just a cover. The Templars are using Consumer Reports as a cover to train a stable of elite Black Hat hackers, with which to take over the world. They're in a race against Communist China, the Russian Mob, and the NSA.

They're also in a race against Dom DeLuise, Jamie Farr dressed as The Sheik, Jackie Chan in a Mitsubishi supercar that can go underwater and some babes in a Countach. Wait, I might have that mixed up.

Anyway, in a post-9/11 world, at least we know they're definitely in a race against terror. Or is that a war against terror? No, that's a war against drugs. Oh I can never remember these things. I should turn on Fox News and let them tell me what we're fighting for again.

Re:

[the_rev_matt]

No no, you're thinking of Spencer Tracy and Buddy Hackett and Sid Caesar and Jonathan Winters in a race to find buried treasure in southern California.

Re:

[Hoi Polloi]

Hey, didn't I see you on a grassy knoll?

Re:

[adavies42]

I see you have a more recent copy of Illuminati than I do.

1st comment?!

[dave562]

And I'm not even a subscriber?!

You know you're in trouble when Consumer Reports is pointing out that your software is worthless. As just about every /.er knows, pattern / signature based detection is all too easily circumvented. Unfortunately it's pretty much all we have. It has been my experience that enabling Heuristic based detection (in Symantec Corporate AV) at any level other than the default just leads to too many false positives.

Re:

[drinkypoo]

As just about every /.er knows, pattern / signature based detection is all too easily circumvented. Unfortunately it's pretty much all we have.

No it isn't, we also have "capabilities" or whatever they're called, where your application is granted by the OS only those rights/abilities it actually needs. This approach has the potential to protect your system from anything short of a kernel-level exploit (like the wifi thing going on now.)

It is their property

[Anonymous Coward]

Consumer Reports destructively tests many things. Why should it matter what they do to their own computers? As long as they don't release these viruses into the wild, there is no problem.

Re:

[El Torico]

But think of all of the 1337 Hax0rs that Consumer Reports is depriving of employment?!

Oh wait a minute, maybe that is who they hired. Never mind.

Crying Wolf?

[bbernard]

FTA: "'Those viruses exist right now only on a CD in a sealed container in a locked cabinet in our computer lab,' Beckford said."

Seriously, it's not like these will ever exist outside of a lab, right? And if they do, the AV companies won't have any problem finding the source code, will they?

Isn't that kind of like telling the insurence institue that they can't change their car crash tests because car makers designed their cars only for specific crash tests? Gee, better not create anything that a car might

Re:

[eln]

Those viruses exist right now only on a CD in a sealed container in a locked cabinet in our computer lab

Sure, but the lock on the cabinet was made by the same people who made the locks on those FEMA trailers.

Isn't that what Morris claimed about the worm?

[Ungrounded Lightning]

FTA: "'Those viruses exist right now only on a CD in a sealed container in a locked cabinet in our computer lab,' Beckford said."

Seriously, it's not like these will ever exist outside of a lab, right?

And as I recall that's what Morris claimed about the internet mail worm, too: That he was experimenting with it on a set of local computers and it got out accidentally due to a connection he wasn't aware of or hadn't properly shut off.

(The timing of when it got out (when most of the relevant people for fightin

Corporate Honesty

[recordMyRides]

Security companies are objecting, on the grounds that they do not want the gaping holes in their software revealed to the public by Consumer Reports.

Re:Corporate Honesty

[jc42]

Heh, funny. But Consumer Reports does have a bit of a history of being sued by companies after serious problems with products were published by CR. CR also has a history of easily winning the few cases that actually go to court. Actually, the companies usually drop charges, after CR makes it clear that they'd be happy to demonstrate the problems in court. CR also often publishes their communications with such companies, which is not really good for sales.

It could be fun to watch an anti-virus software company face CR in court. It would be at least as entertaining as the SCO soap opera. Maybe /. readers should be contacting the companies and encouraging them to sue CR. Think of all the /. articles that this could generate.

If the accept liability

[Shivetya]

for one of their viruses getting out then by all means I think Consumer Reports should be allowed to continue.

Catching them after they are out is easy. The consumer really has so very little to go on from a "trusted source" in regards to virus scanning that the obscurity benefits the AVG companies. With a little more light on the subject we all benefit, all except the AVG companies. Guarantee that whomever CR picks is going to parade that around regardless of their stance before testing occurs.

Again, if CR is willing to accept liability for one of their tests getting out into the wild then I say go for it! Perhaps they should register their "new toys" with someone for backup? Of course that makes for another hole too.

Speaking as one who has been burned...

[Space cowboy]

(See my Journal entry [slashdot.org] for the gory details) ... I would sincerely recommend they don't play with fire. There are too many ways that self-replicating programs can go wrong... or too-right, as in my case :-(

If they can guarantee containment, of course, a virus is completely harmless to the rest of the world. The problem comes when containment is breached because of something you didn't think of - and the problem with things you didn't think of, is that you didn't think of them [grin].

Simon (now a thoroughly-reformed character, honest guv)

Re:Speaking as one who has been burned...

[Guysmiley777]

If they can guarantee containment

How hard is it to unplug a network cable in your world? Don't use a machine with a WiFi card. Low level wipe the drives from a bootable CD when you're done. Not really rocket science.

Re:

[phasm42]

If you've written a virus, then you know all the attack vectors and hooks. On a set of isolated machines, it's very easy to prevent it from spreading. It's not like a biological virus where it could randomly mutate and escape, and you should know this if you've written a virus. Releasing a virus onto an open network was just bad judgement if you want to contain it.

Re:

[TheOldSchooler]

"It's not like a biological virus where it could randomly mutate and escape." Apparently he's a fan of intelligently designed viruses.

Re:Speaking as one who has been burned...

[Space cowboy]

We weren't trying to contain it, in our case - we *wanted* to see if it would work as well as we thought it would. The problem came because we *didn't* think about the consequences of someone using a floppy - we were focussed on the network aspects.

So, we had a general routine to write a !boot (an autoexec-on-read-the-media) file, and hadn't considered the sequence of events of:

• someone writing the virus to a floppy
  
• Us wanting to get rid of the virus
  
• That person bringing the floppy back into the lab and re-infecting the network.
  
• Oh sh*t!
  

So, even though we knew exactly what it was capable of, we hadn't considered the actions of one of those infected, and *that* caused us problems. It's not the capabilities that changed, it's the environment. You don't tend to find that out until you've hit the problem, or you would have dealt with it in the source code - that's all I'm saying...

Oh, and I'm sure they'll take a more-responsible attitude than we had, we *were* 1st-year students...

Simon.

Re:

[cavemanf16]

Well then you will appreciate the fact that you *were* a class-action f___-tard at that stage of life. (I'm at work - love those internet tube filters here!) Anyways, the point is not that I'm hating on you, but that I seriously doubt that Consumer Reports or any other real-world test lab would be that stupid. Here's the solution to your dilemma in the real world:

1. Put five computers without CDRW, DVDRW, floppy or USB drives in small room. (And physically crush, mangle, destroy, or clog with superglue any

Re:

[SatanicPuppy]

You should never work on that sort of thing on anything but a 'clean' machine which has only the environment you're trying to test against on it and nothing else. Under no circumstances should it be connected to a network (besides an isolated test network), and it should never be connected to the internets.

It's just like working with an RL virus. You've got to take precautions unless you want to catch it yourself.

Re:

[mdielmann]

How can we safely develop/test viruses? Simple. Two networks with no physical link to each other or any other network, one for development, one for testing. Absolutely no wireless networks! Transfers by physical media only, on specially marked media, and only from the dev network to the test network. The biggest risk to the rest of the world is in handling the physical media (making sure it isn't loaded into outside machines) and dismantling the computers for a different use after testing. It might be

Re:

[Lord Ender]

Testing viruses should be done on an air-gap network with no removable drives. How hard could that be?

Hey, if it's good for AV products...

[TripMaster Monkey]

Be sure to read our other Consumer Reports articles, where we:

• Test the efficacy of burglar alarms by attempting to break into consumers' homes,
  
• Test the efficacy of the 'morning after' pill by creating unwanted pregnancies,
    - and -
  
• Test the skill of your local emergency room doctor by randomly stabbing people outside the hospital.
  

Thanks, Consumer Reports. Thanks bunches.

Re:Hey, if it's good for AV products...

[krell]

"Test the efficacy of the 'morning after' pill by creating unwanted pregnancies"

Hey, there has to be something out there that security penetration testers can moonlight in, right?

Steve Martin Philanthropy

[pipingguy]

"I help out unwed mothers... hey, I just help them get their start!"

Re:Hey, if it's good for AV products...

[ifrag]

I'll take a stab at that first example of attempting to break into [a] home, since that's the only one that's comparable to what it seems they are doing. If CR wants to setup a test home in which to practice breaking in that's fine, it's their property and they can do with it what they want. It's a test scenario... saying they'd go out and break into consumer homes is not a good parallel. Consumer Reports is (hopefully) not going to create any public security risk in their process if it really is self contained. As long as it stays within their little "sandbox" I don't see what the problem is. The second two examples deal with people instead of objects so it obviously doesn't make for an easy expendable test case.

Re:

[StarManta.Mini]

However, if a female CR editor wanted to test the morning after pill by getting herself pregnant, that would be fine!

Creepy, but fine.

Re:

[mdielmann]

The second two examples deal with people instead of objects so it obviously doesn't make for an easy expendable test case.

This is easily resolved. Don't test in a country with a constitution similar to the U.S. Or use terrorists, since they're obviously not the people mentioned in the U.S. Constitution, as defined by the current administration. Either way.

Re:

[rbochan]

Thanks creating useless strawman arguements.
Thanks bunches.

Re:

[soft_guy]

Test the skill of your local emergency room doctor by randomly stabbing people outside the hospital.

Now I want a job at consumer reports!

Claims shouldn't be verified

[Hoi Polloi]

Soon they'll propose testing car safety by doing test crashes! Or testing fire retardants by trying to set them on fire. Damn those Consumer Reports fools!

Nitpick

[ElleyKitten]

Test the efficacy of the 'morning after' pill by creating unwanted pregnancies

That wouldn't work, because once there's a pregnancy it's too late to test the morning after pill. The morning after pill is contraception, not abortion.

Re:

[d_54321]

Etc? No, really, please go on to the second one. I wanna hear how you'd correct the morning after pill analogy. Would it involve trying to impregnate a blow-up doll?

That's rich...

[advocate_one]

coming from them... I was under the distinct impression the vast majority of viruses only existed in their labs...

There's no good reason to object to this

[cagle_.25]

1) Virus writers will write exactly the same code, unless the boys at Consumer Reports are dedicated enough to come up with truly innovative virus variations. So there's no fear that someone out there will "get ideas."

2) Why not vet your software against somebody else's test suite? If CR wants to function as an extension of Symantec's R&D, let 'em. It's a win-win.

The horror! Real world testing without pr spin!

[dtolman]

No wonder the AV companies are up in arms - its a standard industry requirement to make sure that there is a PR rep assigned to each engineer to "interpret" results, whenever doing tests that shows how well the software actually works!

Symantec et al. are stupid

[Evro]

Security companies are objecting, on the grounds that it's a generally accepted practice not to create viruses for any reason.

You mean they aren't already doing this internally? If not... what the hell are they doing all day? If they're just being reactive without testing their software against possible variants then their software isn't really useful. Though frankly I find antivirus software to be a cure worse than the disease. A 1/100 chance I'll get a virus that does bad things to my computer, or a 100% chance that my computer will run like crap due to NAV.

Solution? Backup all my documents (mostly pics) to a dvd monthly and trust my Linux box firewall/router/proxy to keep the bad bits out.

Re:

[JordanL]

Backup all my documents (mostly pics) to a dvd monthly

Perhaps it's your "pics" which cause the viruses. :O

Re:

[JimBobJoe]

You mean they aren't already doing this internally? If not... what the hell are they doing all day?

According to the article...

Universally, companies say they won't hire former virus writers, and they follow gentleman's agreements to share discovery of dangerous programs with each other

Which in my mind means that they are basically self-flaggelating each other. No particular surprise there, companies in other security industries have similar issues of arrogance regarding what they do, their processes and pro

eicar already has a test file

[Anonymous Coward]

You can use these files to test if your AV program is working

http://www.eicar.org/anti_virus_test_file.htm [eicar.org]

Eicar file is of limited use

[bbernard]

The eicar test-virus file is a great way to see how your computer/av-suite will react to a virus. However, it's not an effective test to see how the heuristics systems and such react. It's non-destructive, and every AV vendor makes sure that they can "catch" it. That's nice for making sure that your AV is running, or that your AV on some workstation reports back to the management computer that it caught a virus, but not for testing the ability of AV software to find new viruses that don't necessarily have definitions written for them yet.

Not a big deal

[guruevi]

That is exactly what virusscanner sellers do. They create new virusses, mutate them and test them out. Of course they don't do that in a internet or network-connected environment. In all cases this should be in a lab environment completely closed off from the exterior world.

What's the big deal here? A bunch of Windows computer with antivirus software running in a closed off network as to benchmark some programs. Happens with games, office software etc... nothing to see here, please move along.

Of course this way you also get stories (hoax, urban legends) like the one about Symantec releasing virusses to sell their software...

Re:

[mblase]

Of course this way you also get stories (hoax, urban legends) like the one about Symantec releasing virusses to sell their software...

Aren't you thinking of "V for Vendetta"?

Re:

[geekoid]

That rumor was started by the uncanny coincedence that new virus alerts seem to appear as their stock drops.

Good Idea

[Apocalypse111]

This is a very good idea, IMO. I mean, for years the major security companies have been using fear tactics to push their software. For an almost equal amount of time, security-concious geeks have been critical of this software. Having a trusted, disinterested third-party like Consumer Reports put it to the test sounds like the perfect solution to this situation.
Its been a long time since someone outside of Norton has talked about how good a Norton product is, but they've been in the game for such a long time that they are trusted by the general public to do their job. I wonder how many would uninstall if Consumer Reports said that their product was utter crap? Or rather, how many would try to uninstall only to find that the uninstaller is broken too?

How well did they do it?

[frankie]

As a CR subscriber, I am utterly amazed that they even had the IDEA to construct a test like that, much less actually find capable programmers and do it. Perhaps that security company cold-called them and suggested it?

CR's technology reviews are often wrong in ways that would be laughable if they weren't so influential. Off the top of my head:

• monitor reviews with photo display tests, where it was obvious to me that no one involved had ever heard of the phrase "gamma correction"
• claim that a two-digit percentage of Macs were infected with spyware
• a seemingly uncanny ability to review hardware obsoleted by newer versions in the interim between testing and publication

Has anyone here heard of this "Independent Security Evaluators" biz? I wonder how many of the viruses were still functional (not just infectious) after twiddling.

Re:

[Hoi Polloi]

"a seemingly uncanny ability to review hardware obsoleted by newer versions in the interim between testing and publication "

In CR's defense this is a problem for virtually every print magazine. The internet has made it possible to publish reviews of hardware before it even reaches the store. Between the testing time and the lag time up to printing and distribution months may have passed.

Outdated hardware

[DragonWriter]

CR's model which provides its independence also means it doesn't tend to have the chummy, early access relationship many other outlets have with manufacturers. Them actually doing really substantial tests also means that they tend to take longer than some other outlets. OTOH, I've rarely been led astray by a CR review on anything, computer related or not, so I'm pretty happy with them despite their limitations.

Re:

[adavies42]

claim that a two-digit percentage of Macs were infected with spyware

At least it wasn't a three-digit percentage....

Mac Viruses & Spyware

[waldoj]

Even in the latest issue (September 2006), they persist in assessing the rate of Mac OS X spyware and virus infections by conducting a survey, an annual gaffe on their part. Rather than checking around and discovering that no such malware exists in the wild, they assume that computer users are able to judge for themselves the cause of computer difficulties.

This would be like studying the mechanisms of natural selection by way of a survey. Hey, whaddyaknow, turns out there's no such thing as evolution, a s

Re:Mac Viruses & Spyware

[Jah-Wren Ryel]

Even in the latest issue (September 2006), they persist in assessing the rate of Mac OS X spyware and virus infections by conducting a survey, an annual gaffe on their part. Rather than checking around and discovering that no such malware exists in the wild, they assume that computer users are able to judge for themselves the cause of computer difficulties.

Over on GardenHoseDot they are saying exactly the same thing - CR's survey of garden hoses makes the mistake of confusing kinks with twisted loops. They

Re:

[Phat_Tony]

Yeah, in last month's computer security issue where they rated Virus software, they threw Symantec Antivirus for Mac in with the PC ratings. I didn't read from front to back, but I couldn't find any place where they mentioned that Macs don't have viruses. I couldn't find how they did their ratings, and suspiciously, Smantec for Mac got the exact same rating for PC. Since they were checking PC anti-virus software to see what percent of viruses it caught when throwing hundreds at it, I find it interesting the

What Virus Detection Co's dont want you to know

[stevew]

Folks,

This one is REALLY obvious. Consumer reports is going to prove that these security products can't detect things they haven't seen before and the Virus detection companies don't want you to know their dirty little secret, i.e. this stuff only works after the cow is out of the barn, i.e. a virus has already been seen in the wild, measured, and characterized.

Anti-virus doesn't work

[kirun]

If this helps wake people up to the fact that anti-virus programs simply don't work, all the better. For example, at one time or another, nearly every antivirus package has declared applications with NSIS [sourceforge.net] installers as malware. I remember having a McAfee trial on my computer, that would regularly make up infections. Yet, when a slightly updated version of a worm comes out, you're unprotected.

Re:

[Valdrax]

I remember having a McAfee trial on my computer, that would regularly make up infections. Yet, when a slightly updated version of a worm comes out, you're unprotected.

That's weird. Is it just the demo version? I've never had any anti-virus software ever detect ANY viruses (false positive or not) on ANY machine I've ever worked on since the old days of floppy viruses.

What kinds of things were you doing to trigger it?

Real Engineering

[Anonymous Coward]

This is what real engineering is all about. It takes real software engineers, not code monkeys, to expost the vulnerability of a product, and report it to the consumers.

It's the duty of every engineer (those that can rightfully call themseleves engineers) to protect the public.

Clearly, classical antivirus software is not protecting us. Kudos to these folks for pointing out what should be the painfully obvious.

How else do you test against new virus varients?

[stefanlasiewski]

"Creating new viruses for the purpose of testing and education is generally not considered a good idea," wrote Igor Muttik of McAfee's antivirus lab on a public company blog this week. "Viruses can leak and cause real trouble."

All these years I've assumed that AV Companies created hundreds of virus varients in a closed lab somewhere so that they could proactively test their product against against new probable varients? How does McAfee anticipate new threats? Do they wait for a new virus to be released into

Results

[alexo]

Here are the scores:

BitDefender Standard - 87
Zone Labs ZoneAlarm Antivirus - 85
Kaspersky Labs Anti-Virus Personal - 82
Norton Antivirus - 80
Norton Antivirus for Macintosh - 80
McAfee ViruScan - 77
Trend Micro PC-cillin Internet Security - 75
Alwil Avast! Antivirus - 68
F-Secure Anti-Virus - 66
Panda Software Titanium AV - 64
CA/eTrust EZ Antivirus - 57
PC Tools AntiVirus - 41

However, I don't have a lot of faith in CR's ability to rank high tech items.

Re:

[geekoid]

AVG does have a corporate version that is paid for.

Bravo, Consumer Reports

[osgeek]

I casually perused CR here and there, but I'd never really known much about them until a relative gifted me with a subscription. Here are a few things I like about them:

1. They pay their own way. They purchase *all* of the products that they test and destroy, since cozying up to get sample products would tarnish their credibility.
2. They don't accept any advertising dollars within their magazine, since that might bias their reporting and tarnish their credibility.
3. They take a strong stand on protecting consumers beyond just good product recommendations. They do editorials and special reports on subjects that /.ers care about, like RFID and general privacy protection; taking strong pro-consumer stances that you don't see in other national publications.

When my gift subscription runs out, I plan on purchasing my own. Not only because I find the product articles useful and interesting; but because the Consumer's Union does other good things with my money.

Re:

[Just Some Guy]

Here's what I don't like about them:

1. They come up with the most bizarre ranking criteria available and stick with them until the bitter end.

Seriously, I don't care if my stereo's power cord comes wrapped in its own plastic bag.

I generally like CR, but it seems like every time they review something I personally know about, they screw it up. It's possible that my area of interest, technology, is the only glaring hole in their testing ability, but that seems somewhat unlikely.

Scare Marketing

[Treeluvinhippy]

As long as new software is being written, there will be some black hat trying to find the buffer overrun of the week. Everyone knows this. It is simply life.

Consumer Reports knows this also. This article is probably going to be geared to cause controversy. They can splash on their cover "What you don't know about your AV software!" and scare the crap out of every AOL Mom with a $499 Dell Desktop. This is just a type of marketing and it sells magazines. If I was a major AV producer I would be calling foul to

It's not "plan to", CR already did it.

[djan]

The /. summary says that "plan to test anti-virus software by creating viruses."

TFA says "Consumer Reports recently conducted one of the most thorough tests ever of antivirus programs. But to really put these security programs through the paces, the magazine hired a firm to create 5,500 new viruses, using them to test the antivirus software products for their ability to detect unexpected threats."

By the way: "In the results, McAfee scored in the middle of the pack. BitDefender and Zone Labs scored at the top, in part for the two program's abilities to detect new viruses."

Consumers Union and Linux

[rotenberry]

I have subscribed to "Consumer Reports" for over twenty years and have never seen a serious discussion of Linux.

You would think that the advantages of Linux and BSD would make it a natural choice for an organization that tries to help the consumer to get the best deal available. All I have seen are discussions about whether a PC or a Mac is best. It is as if the Consumers Union is in the bizarro universe.

brilliant

[Elwood P Dowd]

Security companies are objecting

WTG. Now there's an argument. And and MSN article about it.

I want to read the report now. If they really didn't want this report publicised, the correct response is "whatever".

Let me get this straight.

[Dekortage]

From the article: "I understand .. if you want to test a car's performance, you test the car put on road with lots of bumps on it," Marcus said. "But when you are talking about malicious code, there's a threat to public. There are professionals who know how to handle viruses. It should be left to them." (emphasis added)

Well, that's why Consumer Reports hired computer security professionals [securityevaluators.com] to work with on this. Maybe they're just mad that CR didn't ask them to be the security consultants... oh wait, that might be a conflict of interest for the product review. Tough.

Not planning.

[kahrytan]

Consumers Reports is the most trusted amoung consumers. They put products through their paces and ensure they work well. With that said, yes Consumer Reports create viruses. They already have done so for testing lastest virus programs. Consumer Reports September 2006 issue has said this. They have rated Bit Defender as the best. The issue specifically said they created new viruses to test how well they did against new viruses not already in the signature lists.

People like Igor Muttik are just scared their crappy anti-virus software sucks. Mcafee ranked #6 in the Sept 2006 issue. And even if a CR virus got loose, CR can release the viruses details to venders immediately. The virus wouldn't last more than couple days.

Exposes shortcomings in AV software

[dcam]

The problem is that AV software at the moment scans for signatures of known malware. Essentially they are reactive.

What they should be doing more heuristic scanning, identifying malware by characteristics rather than looking for particular malware signatures.

This is a fundimental weakness in most existing AV software. Certainly this is harder to because legitimate software can do similar things to malware. That doesn't change the fact that AV companies should be concentrating more on this. This is particularly true as most "successful" worms get modified and re-released. As a result it should be possible for the AV companies to detect the altered worms.

Consumer reports is doing us all a service here by exposing this weakness. Provided they ensure the worms don't get out I'm all for it. This is a perfectly valid way of testing the malware. In addition FTA they are doing what most malware writers do anyway: altering the worm just enough so that it is likely to get past the signature based scanning software.

Shame on you McAfee.

Re:

[Todd Knarr]

CR isn't identifying new exploits. All they're doing is the old virus-writer trick of tweaking a virus by shuffling the order of routines around or changing strings (banners or other displayed text) to change the virus enough to make existing signatures not match anymore. In short, CR's testing the ability of AV software to actually detect viruses, not just do a simple grep for a known byte sequence and return a yes/no based on whether it was found or not.

We won't get into stealthed polymorphic viruses tha