Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
×
Security Microsoft

Microsoft Explains Why Windows Defender Isn't Ranked Higher in New Antivirus Tests (zdnet.com) 85

In its most recent reports, AV-Test had very few flattering things to say about Windows Defender. Microsoft's security suite was rated as the seventh best antivirus product in the independent test. In total, 15 AV products were tested. Microsoft, however, has now disputed AV-Test's methodology and conclusion. For some context, the top AV products rated by AV-Test on Windows 10 were Trend Micro, Vipre, AhnLab, Avira, Bitdefender, Kaspersky, and McAfee.

Windows Defender was able to detect 100 percent of new and old malware, but it lost few points for performance (which, AV-Test measures on the basis of how a security suite slows applications and websites on the test computer); and usability (which counts false-positives or instances where AV wrongly identifies a file as malicious.) From a report: Windows Defender's performance rating was dragged down because it slowed the installation of frequently used applications more than the industry average, and wrongly detected 16 pieces of legitimate software compared with the industry average of four. But Microsoft wants enterprise customers to know that Windows Defender is only half the picture, given the option for customers to also deploy Windows Defender Advanced Threat Protection's (ATP) "stack components" including Smartscreen, Application Guard, and Application Control.

In the January and February test Windows Defender also scored 100 percent on protection. However it did miss two samples. Since then it's retrained its machine-learning classifiers to detect them. But Microsoft notes in a new paper that Defender ATP did catch them, which isn't reflected in AV-Test's or other testing firms' result. Microsoft hopes to change this so that testers include so-called stack components available in ATP. "As threats become more sophisticated, Microsoft and other security platform vendors continue evolving their product capabilities to detect threats across different attack stages," Microsoft's Windows Defender Research team writes. "We hope to see independent testers evolve their methodologies as well. Our customers need greater transparency and optics into what an end-to-end solution can accomplish in terms of total preventive protection, including the quality of individual components like antivirus."

This discussion has been archived. No new comments can be posted.

Microsoft Explains Why Windows Defender Isn't Ranked Higher in New Antivirus Tests

Comments Filter:
  • Attack surface (Score:5, Insightful)

    by sinij ( 911942 ) on Friday May 25, 2018 @10:04AM (#56672698)
    MS Defender has one very clear advantage over competition - it doesn't create an additional attack surface and installs yet another vendor's application with deep kernel hooks, network connectivity, and an equivalent of root privileges.
    • Re: (Score:3, Insightful)

      by Anonymous Coward

      but it lost few points for performance (which, AV-Test measures on the basis of how a security suite slows applications and websites on the test computer);

      I would like to know which non-Microsoft AV is this polite. Long, long ago, McAffee was a minimal AV option, but then it joined Norton and all the other "security suites" as a bloated and unwieldy mass of advertising other McAffee products and panicing over 1st party software patches.

      • by bobby ( 109046 )

        I've never been much of a fan of any AV product, but sometime in the last 2 years I discovered McAfee "Real Protect". It seems to work as described. Basically instead of scanning everything you do, it just watches critical system files and disk areas and flags suspicious activity. I've had it alert me a few times, and it was spot-on. Quite happy with it. (Windows Defender still running too, with no detections that I remember).

      • That's what I was wondering too:

        it lost few points for performance (which, AV-Test measures on the basis of how a security suite slows applications and websites on the test computer)

        given that the scale for the metric "Fucks up your computer's performance" is rated from 1 to McAfee.

    • Along similar lines, AV hooks are one of the common causes of system instability, usually blamed on something else, like browsers or Windows itself.

      • I had a customer last week where every time they switch user or log off, the entire graphics subsystem shuts down and the monitor goes to sleep instead of showing the login screen. Turns out it was caused by Avira antivirus.

      • Along similar lines, AV hooks are one of the common causes of system instability, usually blamed on something else, like browsers or Windows itself.

        The design of windows itself is a more common cause. And now Chrome has antivirus capabilities, so it's both a browser AND an antivirus and can fuck up your computer in both ways

    • Re:Attack surface (Score:5, Informative)

      by phantomfive ( 622387 ) on Friday May 25, 2018 @10:16AM (#56672766) Journal

      it doesn't create an additional attack surface

      Unfortunately, yes it does [arstechnica.com].

      • by sinij ( 911942 )

        Key to understanding my post is "additional". Defender isn't categorically better than other AVs, but you are not giving additional access to a third-party into your system. That is, MS already has that level of access. Plus, since they wrote OS, Defender will play nice with it.

        • Re:Attack surface (Score:4, Informative)

          by ( 4475953 ) on Friday May 25, 2018 @10:38AM (#56672910)

          An additional attack surface is one that exists if you install and run the software but doesn't exist when you don't install or run the software. Microsoft Defender adds an additional attack surface like any other antivirus software.

          • by epine ( 68316 )

            An additional attack surface is one that exists if you install and run the software but doesn't exist when you don't install or run the software. Microsoft Defender adds an additional attack surface like any other antivirus software.

            He/she meant an additional attack surface beyond the necessary entailment of the category itself. In language, "additional" can be deployed anywhere along the semantic chain, so long as the situation can get worse, or worser, or worstest.

            However, depending on how Microsoft manag

        • Re:Attack surface (Score:4, Insightful)

          by Riceballsan ( 816702 ) on Friday May 25, 2018 @10:57AM (#56673028)
          so by surface, you mean company? Windows defender is an attack surface, in the sense that it is a piece of software with admin access that rests in addition to the OS as a whole and can in some situations be tricked into doing bad things. If you install bitdefender or something else they generally disable windows defender, which closes down those possible attack vectors, and replace them with whatever the other protection's vectors are. No matter what protection you are using, you've got the same number of attack surfaces, it's just that all attack surfaces are owned by the same company, instead of by 2 companies.
    • Re:Attack surface (Score:5, Interesting)

      by danbert8 ( 1024253 ) on Friday May 25, 2018 @10:24AM (#56672820)

      I use Windows Defender because it's the only AV that isn't worse than the viruses it is supposed to be protecting against...

    • Found the microsoft paid damage control thread!

  • Optics (Score:2, Funny)

    by phantomfive ( 622387 )

    Our customers need greater transparency and optics

    Oh, they are laying fiber now?

    • Our customers need greater transparency and optics

      Oh, they are laying fiber now?

      Is that a euphemism for a healthy poop?

  • I have Malwarebytes Anti-Malware Scanner and Windows Defender installed on my Windows systems at home. I haven't had any issues since the Windows XP era.
    • I have uBlock Origin, SandboxIE, and virtualization. This has kept bad things at bay since the early 2000s. An ad blocker does more for security than most AV programs (which usually are good enough to catch older stuff, so better than nothing.) Of course, virtualization and sandboxing ensures that stuff that gets out is well contained.

    • +1 for Malwarebytes if you *have* to use Windows

  • by Anonymous Coward

    In the January and February test Windows Defender also scored 100 percent on protection. However it did miss two samples

    So which is it? 100% or missed two samples? Because I can tell you my kid doesn't get 100% if she misses 2 questions on an exam.

    • by Pascoea ( 968200 )

      I can tell you my kid doesn't get 100% if she misses 2 questions on an exam

      What if they are grading on a curve? Granted, It's like saying "A condom works 100% of the time, except when it doesn't."

    • by Rhipf ( 525263 )

      I don't know how many items were used in the test but for an antivirus test it could be thousands. If Defender missed two of two thousand it is close enough to 100% to just use that figure. They could have said it was 99.9% in that example but as the number of viruses tested increases the easier it is to just say the test was 100%.

      • It may be easier to say... but it kills a lot of the meaning. phrases like "over 99%", or 99.99% or something like that carry a HUGELY different connotation than 100%. Perfect is a very specific thought in peoples mind, IE they couldn't find something it couldn't handle. There's a reason even Lysol doesn't advertise as killing ALL germs, even though from what I understand the .01% that it misses are litterally just germs it doesn't touch.
      • by Anonymous Coward

        I worked for a company that did this sort of thing for a couple of years.
        The had over 32K signatures in their database.

  • it lost few points for performance (which, AV-Test measures on the basis of how a security suite slows applications and websites on the test computer); and usability (which counts false-positives or instances where AV wrongly identifies a file as malicious.) . . . . . . But Microsoft wants enterprise customers to know that Windows Defender is only half the picture, given the option for customers to also deploy Windows Defender Advanced Threat Protection's (ATP) "stack components"

    I have a hard time believin

  • by Anonymous Coward on Friday May 25, 2018 @10:31AM (#56672860)

    Anyone should understand that Relative rankings are mostly worthless. If all the products in the top 10 are excellent, but one product has slightly less points than the top 9, does it really matter than it ranked 10th?

    The main advantage of Windows Defender is it's free. For most people that trumps all the other rankings. It's free, it protected against everything the competition did, it's nearly as usable, and slightly slower. That's good enough to not buy something else.

    The AV vendors should be quaking in their boots. Why would you buy another product when what MS puts out is generally fine? My guess is they'll improve the usability a bit, and they'll rank in the top 3. Then start saying goodbye to several of the other AV vendors.

    • by ranton ( 36917 ) on Friday May 25, 2018 @11:03AM (#56673066)

      The AV vendors should be quaking in their boots. Why would you buy another product when what MS puts out is generally fine?

      One reason is because many users have learned they should pick an anti-virus software suite every time they go to Dell and order a new computer. Retailers have an incentive to only offer paid versions because they will get their cut. So many users will keep on choosing either McAfee or Norton just because those are options they are given.

      I'm not sure how many users this describes, but my guess is a lot of them. Then again any significant loss is sales should have them quaking in their boots.

  • I also filter out 100% of the spam mails. I am even close to 160% [statista.com] depending on how many spam I get. I just point everything to /dev/null in .procmailrc .

  • by Anonymous Coward

    Showing GDPR acceptance plea in barely readable greyed out text in a box that covers whole screen is a new low for this site which posts articles about security and privacy.

    How ironic!

    KTHXBYE!

  • Couldn't keep Windows off my machine.

  • I am not defending MS here - but who wants to be compared to industry _averages_ when it comes to security. The people adjusting the ranking because it does not compare well to an average are what I like to call stuupid (it is not a typo). You should want perfect security - to hell with averages.
  • Virus scanners are judged on how well they completely cripple a target system. Windows Defender doesn't do that so it just isn't any good.

    Oh and First post. Or at least it would have been if I wasn't running McAfee.

  • by Anonymous Coward on Friday May 25, 2018 @11:11AM (#56673134)

    Ok, direct experience here, and I am absolutely no fanboy of ms software. But, as part of a offensive security cert a few months back, I got heavily into writing and compiling windows exploit code, and one of the course exercises walk through testing a piece of malware by the virus total site.
    So as part of my studies and self learning I wrote a non self propagating malicious exploit, but it did elevate privileges from the user to admin and get access to things and start calc as a admin user to prove it was exploiting. I took a common windows POC exploit and modified it heavily in ways I will not discuss to a wider audience (because teaching people av evasion techniques is best left to offsec and their ilk, to the right people) and compiled it.
    Out of sheer curiosity I submitted the original POC code, one encoded by a old common packer & my heavily modified "malware" to virus total, and the original and encoded packed version was picked up by about 45/47 av's straight off. The *ONLY* av that managed to detect my custom payload was.... Windows Defender. It must have opened the executable and saw where it hooked when it shouldn't, and the competition seem to rely on pattern matching instead.
    So yeah, sign me up for free windows defender. When the subject comes up with lay people who ask me what to use, its what I would recommend them. From first hand testing.

    Anon, because even with all the above, I'm basically admitting to authoring a custom exploit, and while I'm employed in this field, I could do without the extra attention.

  • by QuietLagoon ( 813062 ) on Friday May 25, 2018 @11:14AM (#56673150)
    (as regular readers here may note)... but... so what. Windows Defender was ranked 7th seems to be the big takeaway in the summary. What if the top 10 are all good to use, does being 7th really matter? I've been using Windows Defender for a couple of years (when Avast started their annoying desktop pop-up adverts that I could not disable, I switched to Windows Defender).

    .
    Additionally, Windows Defender does not seem to install all manner of additional software that digs deep into the Windows kernel in order to do its job. For my needs, Windows Defender is a simple, effective a/v solution that works well. Why should I care if it ranks 7 or 3 of even 1?

  • Things have improved tremendously since the Windows XP era in terms of Windows and app security. Also people tend to use adblockers and flash isn't on by default on newer systems.

    Adobe now has sandboxing and Windows gets new security updates each month. IT departments now update software regularly and people use ancient IE almost never outside of a Citrix or vdi environment.

    The use of AV software to protect idiots who click on everything is unheard of as people know better now than in 2000.

  • Defender proves this. So why doesn't Microsoft just sell a hardened Windows ? Why sell an insecure product and then addon security ?
    • Defender is built in, it isn't an addon. To use the other vendor products you are removing Microsoft's hardening and adding another vendors.
  • nice info tkss bro https://ordertrungquoc.com.vn/ [ordertrungquoc.com.vn]
  • If you keep your HOSTS file updated regularly!
    Must be true, I read it here!!!!!

    *ducks*

  • If anything has complete keys to your PC kingdom, it would be anti-virus software.

    With everything going on in the news today, how can anybody truly trust any of these solutions? If you think that there isn't a cold war going on in the internet, you are uninformed. I see it for myself in logs and honeypot activity all the time. I have to chuckle every time I hear somebody swear by an antivirus program because it doesn't cause trouble and they think they aren't getting infected. I do however expect more of
  • what kind of testing methods did the 'independent' AV-test use, as my own experience with Trend Micro and Kaspersky is they are CRAP and have a very big impact on the performance of your computer. Trend Micro is really rubbish, if an application deletes multiple files after each other (using simple API calls) it immediatly removes it without warning, even though there is not a single virus/malware signature in it.

"If it's not loud, it doesn't work!" -- Blank Reg, from "Max Headroom"

Working...