Forgot your password?
typodupeerror
Security Software Worms IT

Many Antivirus Tools Fail in LinuxWorld Test 234

Posted by CowboyNeal
from the survival-of-the-fittest dept.
talkinsecurity writes "In a public, side-by-side test conducted last night at LinuxWorld, ten antivirus products were confronted with 25 known viruses. The results were surprisingly disparate. Only three of the products caught all of the viruses; three only caught 61 percent, and one caught an abysmal 6 percent. The test, which wasn't particularly complicated, proves that there still are wide differences in the effectiveness of AV tools. A lot of people think all AV tools are the same — they're not!"
This discussion has been archived. No new comments can be posted.

Many Antivirus Tools Fail in LinuxWorld Test

Comments Filter:
  • The winners: (Score:5, Informative)

    by RichPowers (998637) on Thursday August 09, 2007 @08:48PM (#20177373)
    From TFA:

    Kaspersky, Symantec, and Clam AV: 100% caught

    FProt and Sophos: 94%

    McAfee: 89%

    GlobalHauri, Fortinet, and SonicWall: 61%

    WatchGuard's Linux AV: 6%

    And a graph of the results plus links to some of the test viruses: http://virus.untangle.com/ [untangle.com]
    • Re:The winners: (Score:5, Interesting)

      by alx5000 (896642) * <alx5000&alx5000,net> on Thursday August 09, 2007 @08:51PM (#20177397) Homepage
      What's even funnier:

      WatchGuard disputes the test results, stating that it uses ClamAV -- one of the products that caught all of the viruses -- in its own product. "We don't see how the results could be valid -- our product uses ClamAV," a spokesman says.
    • Re:The winners: (Score:5, Insightful)

      by Anonymous Coward on Thursday August 09, 2007 @08:52PM (#20177405)
      I must have missed something. How, with 25 different viruses can one catch 6%? My math skillz tell me that it should be divisible by 4.
      • by Anonymous Coward on Thursday August 09, 2007 @08:57PM (#20177429)
        Duh, it detected a virus and a half! Do I have to explain everything to you??
      • by quadra23 (786171) on Thursday August 09, 2007 @11:51PM (#20178607) Journal
        One product, WatchGuard's Linux AV tool, caught fewer than 6 percent of the viruses sent to it. "We're not exactly sure what the problem with WatchGuard is," says Morris. "The test was set up the same way for all of the vendors."

        This number quoted by the original poster missed the section in bold, it was technically < 6%, which could mean either 0 or 1 virus (funny how everything always works out to binary in some way or another :). My question would be which is it? Either way, my system would be compromised by either 24 or 25 viruses -- neither of which is a good scenario especially in regards to well-known viruses (according to the article no 0-day exploits were accepted).
        • Re: (Score:3, Funny)

          by rts008 (812749)
          "Either way, my system would be compromised by either 24 or 25 viruses..."

          24 or 25 out of 25?

          Hmmm....

          Does mean that *nix is finally ready for the desktop?..Just like Windows?

          Uhmm..w00t!?!?

          Disclaimer: coming to you from a Feisty Kubuntu PC that is running ClamAV.
        • Ugh, not binary (Score:2, Informative)

          by gerf (532474)

          I couldn't ignore the anal-retentive troll inside of me.

          which could mean either 0 or 1 virus (funny how everything always works out to binary in some way or another :).

          That is not binary, but rather only could be binary, but could be any m-ary. True, it could be binary, if you assume two viruses would be represented by 01, three by 11, four by 001, and so forth. As it is, it's ambiguous, as are all numbers. 234 viruses could be decimal, hexadecimal, or a higher base, just as X amount of something does

      • Re: (Score:2, Insightful)

        by iminplaya (723125)
        You must be one of those old timers that didn't have to suffer the new math from the 60s. Hint: It's all about self esteem now.
      • Re: (Score:2, Informative)

        If you read the website with the original results [untangle.com], it says that there were actually only 18 viruses in the first test, and Watchguard only caught one, which is 5.6%. You can download a nice spreadsheet with detailed information about which viruses every solution caught, too.
      • Re: (Score:3, Informative)

        by sbryant (93075)

        How, with 25 different viruses can one catch 6%?

        Because the test set was 18, and not 25 as reported. 100/18=5.555. Have a look at the test results [untangle.com].

        -- Steve

    • AVG (Score:4, Informative)

      by DigiShaman (671371) on Thursday August 09, 2007 @08:54PM (#20177411) Homepage
      What about AVG? I really love it. I've installed on both my workstations and a server (Windows). It uses minimal resources, it's fast, and it's managed to catch more stuff then Trend Micro, Symantec and McAfee.

      Also, Bitdefender and Nod32 are also good for the Windows enviroment. I'm curious to how all these ranked in the Linux world.
      • Re:AVG (Score:5, Informative)

        by Southpaw018 (793465) * on Thursday August 09, 2007 @09:16PM (#20177583) Journal
        They left out Eset NOD32 as well. Symantec and McAffee are the AV old guard: still strong, but also bloated, slow, and weakening. And they have the occasional health problems.

        Kaspersky and Eset seem to be the two main up and comers, and they left one out!
        • Re: (Score:2, Funny)

          by cp.tar (871488)

          Kaspersky and Eset seem to be the two main up and comers, and they left one out!

          Well, I haven't noticed a NOD32 for Linux... have you?

          • Re: (Score:3, Informative)

            by schwaang (667808)

            NOD32 Antivirus for File Servers runs seamlessly on all mainstream Linux distributions (RedHat, Mandrake, SuSE, Debian and others) and FreeBSD. The small footprint and fast performance makes NOD32 optimally suited for real-time or on-demand protection of your Unix File System Servers.


            http://www.eset.com/products/linux.php [eset.com]
            • Re: (Score:3, Funny)

              by cp.tar (871488)

              Well, my bad...

              In that case, I have two things ro wonder about:
              1. Why wasn't it included in the test? and
              2. WTF was my original post moderated Funny for?

      • Re:AVG (Score:4, Informative)

        by Kymermosst (33885) on Thursday August 09, 2007 @09:25PM (#20177659) Journal
        What about AVG? I really love it. I've installed on both my workstations and a server (Windows). It uses minimal resources, it's fast, and it's managed to catch more stuff then Trend Micro, Symantec and McAfee.

        Also, Bitdefender and Nod32 are also good for the Windows enviroment. I'm curious to how all these ranked in the Linux world.


        Test them yourself. The virus samples they used are found here [untangle.com].
      • Re:AVG (Score:4, Informative)

        by omeomi (675045) on Thursday August 09, 2007 @09:49PM (#20177855) Homepage
        I've had good experiences with AVG. Unfortunately, on the rare occasions that I have had to deal with a virus, I've had to go through just about every single virus scanner that I can find before I'm able to completely eliminate the virus. Last time around, AVG was the one that correctly identified the virus, allowing me to find some special utility that somebody had written specifically to delete that particular virus. I think it was still a fairly new virus, which might explain why the major brands weren't able to clean my system, but I've been somewhat surprised in the past that it's so difficult to remove a virus/worm with commercial virus scanners.
        • Re:AVG (Score:4, Informative)

          by Feyr (449684) on Thursday August 09, 2007 @11:23PM (#20178417) Journal
          my experience mirrors yours. based on many dozens of PCs running AVG: it's excellent at detection but once a virus does get past it you're fucked
          • What platform?
          • Once a virus is in the machine it can do whatever it likes, including hiding itself from your antivirus. I've personally disinfected dozens of machines which have Norton+a virus.

            The answer is usually to reboot in safe mode and scan from there.

            PS: I use AVG. Norton is just too intrusive, bloated and causes too many problems with normal system operation.

            • The answer is usually to reboot in safe mode and scan from there.
              How safe is safe mode? I reboot into Linux and scan the Windows partition from there.
        • by Sycraft-fu (314770)
          Is viruses can be a bitch to remove when the system is online, since the virus can do things to fight the scanner. I see a scanner running on a lice system as preemption, not recovery. You run it to stop the virus before it can cause harm. AVG seems good at that, it seems to notice viruses right away.

          If you want to use a tool like that for recovery, they way to do it is on an offline system. Either take the disk to another computer and set it up as a non-system disk, or build yourself a PE boot disc and cle
        • Re: (Score:3, Funny)

          by macdaddy (38372)
          AVG did the same for me about a month ago. Vundo got on my laptop and it took forever to get rid of the damn thing. It always makes me nervous when the instructions for doing something in Windows point out that "your machine will blue screen after this step but don't worry; that's normal."
      • by it0 (567968)
        I have bitdefender running on my system
        it0@home:/tmp/virus$ bdc all/*
        BDC/Linux-Console v7.1 (build 2559) (i386) (Jul 6 2005 16:28:53)
        Copyright (C) 1996-2004 SOFTWIN SRL. All rights reserved.

        /tmp/virus/all/000_eicar.com infected: EICAR-Test-File (not a virus)
        /tmp/virus/all/001_eicarcom2.zip=>eicar_com.zip=> e icar.com infected: EICAR-Test-File (not a virus)
        /tmp/virus/all/002_eicar_com.zip=>eicar.com infected: EICAR-Test-File (not a virus)
        /tmp/virus/all/004_eicar.zip.bad_extension=>ei
    • There were 25 viruses. How does something catch six percent? Eight or four, sure. But six?
    • by hazem (472289)
      I'm no math genius, but if there were only 25 viruses, how can any of the tools catch a percentage of them that is not a multiple of 4%?
  • by pddo (969282) on Thursday August 09, 2007 @08:51PM (#20177389)
    are viruses on linux a overflow from WINE?
  • Not much here.

    The story could have shown a list of the tested viruses verses the AV software being tested. A simple table would have conveyed a great deal more information than the druel the fellow wrote. Yes I RTFA and as I said - it is not very informative.
    • by shystershep (643874) * <bdshepherd@NosPAM.gmail.com> on Thursday August 09, 2007 @09:22PM (#20177639) Homepage Journal

      druel

      Is that a cross between drivel and drool? Maybe some gruel thrown in for flavor?

    • Re: (Score:3, Informative)

      by Kymermosst (33885)
      The story could have shown a list of the tested viruses verses the AV software being tested. A simple table would have conveyed a great deal more information than the druel the fellow wrote. Yes I RTFA and as I said - it is not very informative.

      You RTFA and then sadly don't do any research. Why would they bother to list the tested viruses when provide the actual viruses [untangle.com] (see "Test Set")?

      • Can you open the zip and tell me what they are?
        • by compro01 (777531)
          why open it? most any competant antivirus program can scan within a ZIP file.
          • But the above asked for a list.
          • by mspohr (589790)
            I just scanned the Virus samples zip file with my (presumably up to date) corporate Symantec Antivirus and it came up perfectly clean!??`!!!

            I then upzipped the file and Symantec still let 14 of the viruses through!!

    • by JackieBrown (987087) <dbroome@gmail.com> on Thursday August 09, 2007 @11:24PM (#20178423)
      000_eicar.com
      001_eicarcom2.zip
      002_eicar_com.zip
      003_eicar.rar
      004_eicar.zip.bad_extension
      005_eicar_big.zip
      010_18_04_2005.exe
      011_abuselist.zip
      012_fullstory.exe
      013_image.jpg.exe
      014_message.pif
      015_mntrup.exe
      016_patch-6143.zip
      017_photo.pif
      018_q347558.exe
      019_scan_check.jpg.exe
      020_test.zip
      021_The_taxation.zip
      100_8.zip
      101_scan.jpg
      102_Syndony.zip
      103_Update-KB8136
      104_Attachement.scr
      105_image.jpg.exe
      106_Info.exe
      107_Please-confirm-pay
      108_virus_87
      109_virus_88
      110_vvzh.scr
      111_xxx.com
      112_untangle1.zip
      113_untangle21.zip
      114_untangle22.zip
      115_untangle3.zip
      116_untangle4.zip
  • math question (Score:2, Interesting)

    by jeebee (229681)
    How does i/25 not equal 4*i%? Were some of the 25 viruses half-caught, or one-quarter caught?
    • Re: (Score:3, Insightful)

      by seriesrover (867969)
      thats exactly what I was thinking...how can you have 25 viruses and get anything other than 4%, 8%, 12% etc. The article refers to 6%, 61% and 89%...bizarre - I can only reason that they weighted the severity of each virus.
      • Re:math question (Score:4, Insightful)

        by VirusEqualsVeryYes (981719) on Thursday August 09, 2007 @10:06PM (#20177971)
        Additionally, they could have calculated the type of virus (by entry method, severity (as you mentioned), spread method, mode of attack, age, etc.) and weighed their percentages in the wild. It's also possible that the programs perhaps prevented some of the damage of some of the virusus, thus meriting partial credit.

        It's also possible I'm wrong, but either way, the article is omitting some information we're supposed to know.
    • by mhall119 (1035984)
      From the article (emphasis mine):

      One product, WatchGuard's Linux AV tool, caught fewer than 6 percent of the viruses sent to it.
      Obviously WatchGuard only caught 4% (or maybe 0%), and they were just trying not to embarrass them too much, you insensitive clod.
      • by Spikeles (972972)
        Actually if you read the spreadsheet, the only thing Watch guard picked up was the EICAR test pattern.

        Which is fairly easy to test since it's just a string of characters that make a fully workable DOS program..
        X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIR US-TEST-FILE!$H+H*
    • Re:math question (Score:5, Informative)

      by Bibz (849958) <seb2004.hotmail@com> on Thursday August 09, 2007 @10:21PM (#20178051)
      Because the summary isn't right.

      They used 18 test cases, Watchguard got only one : 1/18 = 5.55%, rounded = 6%

      All from the spreadsheet available at http://virus.untangle.com/ [untangle.com]
    • by AlanS2002 (580378)
      Multiple runs of the tests, perhaps.
  • Odd numbers. (Score:5, Interesting)

    by DerekLyons (302214) <fairwater AT gmail DOT com> on Thursday August 09, 2007 @08:56PM (#20177421) Homepage
    Something seems a little strange here. With 25 test cases, and a binary outcome (either the virus was detected or it was not), the %caught should proceed in even step of 4%. There's some number massaging going on somewhere.
     
    Hmm... the Fight Club Website [untangle.com] lists 35 test cases, not 25. It's not clear if there is any overlap between the various test cases. In fact, there's not any discussion of the testing methodology (let alone what precisely was tested) at all. Just "here's our numbers - believe them or infect your own machine and find out for yourself".
     
    Now, while I admire the 'do it yourself' hacker ethos as much as the next guy - this is taking it a bit too far.
  • by eddy (18759) on Thursday August 09, 2007 @08:56PM (#20177423) Homepage Journal

    For fun I downloaded an application where I suspected the "keygen" was trojanized. I was correct; the real keygen had been bundled with some, as it would turn out, Off The Shelf trojan. However, I didn't know what trojan so I scanned with F-Secure's online-engine, which didn't detect anything (neither did my active AVG installation). So I sent in the exectuable as a sample, explained what little I had to say; where I found the file, that it was pecompact2'ed, that their online scan didn't detect it. The process of submitting a file req. you to attach the scanner log.

    Got the reply that "The file you submitted was found to be malicious, and is already detected as Trojan-Downloader.Win32.Delf.asz using the latest virus definitions." and "Please update your virus definition databases to properly detect the file".

    Remember, I had scanned it using their latest online scanner and provided the log where the trojan was NOT detected.

    So, maybe an extra warning for online scanning engines.

    PS.
    Shortly after I had submitted the file to f-prot, AVG started detecting it.

    • Re: (Score:2, Funny)

      by ianare (1132971)

      "The file you submitted was found to be malicious, and is already detected as Trojan-Downloader.Win32.Delf.asz using the latest virus definitions Please update your virus definition databases to properly detect the file".
      Translation :
      "Thanks for your submission, we analyzed the file and it's a new variant of Trojan-Downloader.Win32.Delf.asz that we hadn't seen before. Do an update to verify it's being detected properly by the client."
    • I've had excellent results myself with submitting unknown suspicious files to McAfee. Sure, their software isn't what it used to be, but they've been very fast at getting back to me with virus definition "extra.dat" files to detect the virus/trojan in the field.
    • A while ago, I purposely downloaded the Bagle virus from one of my old yahoo accounts. That's when I found out the media was messing up every time they refered to it as the Beagle virus. How did I find out it was really Bagel? Because I opened it in vi, vi went into hex mode, and I found a bunch of registry strings containing Bagle instead of Beagle. In order to download it (because the online filters caught it as a virus,) I had to supply the direct URL that bypassed Yahoo's antivirus. It wasn't hard,
      • Re: (Score:3, Funny)

        by Spikeles (972972)

        I purposely downloaded the Bagle virus

        How did I find out it was really Bagel?

        containing Bagle instead of Beagle
        I'm sorry, which is it again?
  • by blind biker (1066130) on Thursday August 09, 2007 @08:59PM (#20177451) Journal
    Nice to see opensource programs perform so well, so consistently. I only wish the author(s) maintained the ports and packages himself. The Win32 port seems a bit of an afterthought. Anyway, still a brilliant antivirus program.

    (My other OS favourites include Audacity, CDex, The GIMP and OpenSolaris (you didn't expect that one coming, did you)).
    • by jnf (846084)
      It's a horrible AV program, unless you don't count the fact it will get you owned [google.com]
  • We use Sophos on our Linux mail relays and Trend on the desktops, servers and web proxy. We've only had one small virus outbreak in 15 months. I guess Trend isn't covered since there is no Linux client, but it is in the top bracket on every shootout I have seen in the last couple years.
    • by xrayspx (13127)
      I guess Trend isn't covered since there is no Linux client

      That's not really true. Trend sells IMSS [trendmicro.com] for linux relays. I notice you said "client", but still, I would think IMSS should have been included.
  • Not surprising... (Score:4, Informative)

    by SuperBanana (662181) on Thursday August 09, 2007 @09:14PM (#20177579)

    ...considering that most of the antivirus programs were tricked when a new "variant" of one of the worms back around '99 or so. So kids- just insert random whitespace into your worms!

    The change? The line endings in the VBS script changed. It probably wasn't even intentional- some broken mail server probably modified CR's into CRLF's. It sailed right past Trend Micro's email scanner and infected several dozen systems.

    I was the first person to notice why it slipped by, and brought it to the attention of a big-name "security expert" who ran a mailing list which shall go unnamed. He thanked us for the research, passed along my findings to the list, and then promptly went around doing interviews with the press using the first person voice. "I discovered that...", blah blah was what I read the next day.

  • by BearRanger (945122) on Thursday August 09, 2007 @09:21PM (#20177627)
    Let me preface this by saying that I work in a Windows free environment. I understand that not everyone has this luxury.

    Am I a bad citizen because I don't scan for Windows viruses on my Linux systems? It's almost like another Microsoft tax--you're expected to degrade your performance to prevent their victims, uh, customers (yeah, that's it) from infecting each other. Those folks need to be responsible for their own safety and not expect the rest of us to do it for them. They could start by holding Microsoft accountable and making other choices at purchasing time. To me, Windows isn't worth the hassle.
    • Re: (Score:3, Insightful)

      by n0dna (939092)
      Ever consider that every virus infection stopped by anyone, target or not, could cut down on the bandwidth sucked away from all of us by the ever increasing botnets?

      What about infected files that don't originate on your systems but are passed through it? If you send out an infected file, the recipient won't care where you think you got it, or how much you feel that it isn't your problem, you're the one who infected them.

      You can piss and moan about trash on the sidewalk or you can just pick it up.
    • Yeah... same idea as "My fucking legs work. Is it my fault that yours don't? Am I expected to forgo the luxury of an escalator because you are in a wheel chair?"
    • Re: (Score:2, Interesting)

      by tech10171968 (955149)
      I, too, work in a completely Windows-free enviroment at our company (in fact, I'm the one who spec'd everything, from our database server to the workstations). But I still insist on everyones' machine running ClamAV because, while we don't have many/any worries about being compromised by malware, we do exchange web traffic with our customers (like, say, most any business using at least one computer with an internet connection). I'd hate like hell to think that we may have inadvertently passed a virus- or tr
    • by dbIII (701233)

      Am I a bad citizen because I don't scan for Windows viruses on my Linux systems?

      It only makes sense on mail gateways and possibly web proxies that have Microsoft machines behind them.

    • On an enterprise level, if one of your workers sends me an infected file, I don't care that all your systems are linux and that it doesn't affect it. I'm permanently putting your company on my block list
  • Interesting that SonicWALL only caught 61% compared to McAfee catching 89%. The virus protection on our SonicWALL at work is powered by McAfee.
  • Rainbow Fonts (Score:2, Interesting)

    by Tablizer (95088)
    The charts used those damned ClearType sub-pixelation fonts in the image, which is not going to work right with many monitors since they have to be tuned per user. When I see that rainbowy tinge, at first I check to make sure I haven't drank too much c c c coffee again.
    • by Anpheus (908711)
      The image itself won't contain any subpixel data, it can't. Except for SVG, and no browsers support doing anything like this yet, there is no format that renders to a higher resolution than a device pixel. I suppose SVG -could- use subpixel rendering, but I'm fairly certain that anti-aliasing is the extent of what is done there.
  • by RootWind (993172) on Thursday August 09, 2007 @09:33PM (#20177729)
    Not to knock Clam but there is something odd about these results (Besides the absurdly low testbed). TFA says Clam won two years ago (which meant Untangle would use it), and again now. However, just last May the results from AV-Test.org (a real trusted legitimate source) against a comprehensive testbed put ClamAV near the bottom of the heap: http://www.pcmag.com/article2/0,1895,2135053,00.as p [pcmag.com]
    I can't help but think that Untangle is trying to justify their own choice, rather than have a real test. With a testbed of only 25-35, it is possible to pick a group of malware that can put any AV on top. Even the user submitted malware is suspect, especially when that testset is also so low. ClamAV is great against virus outbreaks, with one of the fastest signature responses, but it has pretty atrocious trojan and zoo detection, since there is not enough man-power to collect and create signatures for less prevalent and non-replicating malware.
  • For the Excel-averse, I have uploaded the Excel Results of the test to the Zoho Viewer website. So you needn't install Excel or OO. http://viewer.zoho.com/docs/edblaI [zoho.com]
  • by Anonymous Coward
    All of them depend on guessing whether a file is good or bad.
    All of them will have false negatives as well as false positives, most likely skewed to have fewer false positives to reduce the annoyance factor at the expense of missing real viruses - false negatives.
    There are substantially better and computationally cheaper ways to protect your system than an anti-virus.
  • I think their counting frame has a cracked bead...
  • Detected, not Caught (Score:2, Interesting)

    by Riquez (917372)

    Only three of the products caught all of the viruses
    Does this not strike anyone as a really stupid way to word the detection of a virus?
    If you "catch a virus", you're infected.

    "where's geoff today?",
    "oh, he caught the flu"
    "he caught it! nice one geoff, you managed to destroy that pesky flu & not get infected - so he's out celebrating right?"
    "erm... fk off weirdo"
  • Given any set of 25 viruses, each virus represents 4 percent. So one antivirus caught a virus and a half?
  • I use Watchguard all the time and nothing has ever gone wr&,;*..}..Get 3 months of Viagra free with our low mortgage rate offer now now now!
           

"Any excuse will serve a tyrant." -- Aesop

Working...