Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Microsoft Sponsors Antiphishing Bakeoff

Posted by kdawson on Thu Sep 28, 2006 03:24 PM
from the here-phishie-phishie dept.
Security Microsoft
uniquebydegrees writes, "InfoWorld is blogging about the (predictable) results of a Microsoft-sponsored antiphishing technology bakeoff. From the TechWatch blog: 'Microsoft's Phishing Filter (MPF) in IE 7 Beta 3 received the highest "composite score" at 172, followed closely by NetCraft's toolbar with a composite score of 168. But when you dig into the numbers, another story emerges... IE's MPF antiphishing toolbar doesn't top out any of the individual tests that make up the composite score... So how did MPF end up on top?... Microsoft didn't do the best job of spotting phish sites, but it did do the best job of blocking the ones it did spot, and blocking was what garnered the most points... Blocking a phishing Web site earned you twice as many points as just warning about it in this test, but is blocking really twice as effective as just warning users?'"
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Microsoft Sponsors Antiphishing Bakeoff 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.

  • What a silly question. (Score:5, Funny)

    by Saint Aardvark (159009) * on Thursday September 28 2006, @03:26PM (#16236169) Homepage Journal

    ...but is blocking really twice as effective as just warning users?

    No, of course not. That's why I tape the root password for the file server to users' monitors, but warn them strongly not to use it.

    • Err..

      The real questions:

      Provided that
      1) If you block and earn twice the point, and
      2) If you warn and earn as many as half the points you earn by blocking

      how many point you would earn if

      a) You warn 34 sites and block half as many as you have warned
      b) You block twice as much sites as you have warned

      Also, which one is more effective?

      Remember, you will earn as much as twice the score for answering first quetion right, but as much as half the score for answering the second question wrong.
    • Of course the rules have been twisted to get the MS offering on top. It 2x had not worked, then it would have been 3x or 10x or whatever mgic multiplier would have got the MS device on top.
    • Block should be worth one point, a false positive should cost one point, and warnings should be worth nothing. As Bruce Schneier once said:

      You're surfing the Web and you see a button on the Web site saying,
      "Click here to see the dancing pigs." And you click on the Web site
      and then this window comes up saying, "Warning: this is an untrusted
      Java applet. It might damage your system. Do you want to continue?
      Yes/No." Well, the average computer user is going to pick dancing
      pigs over security any day. And we

      • The difference is of course that if you're stationg that you are paypl or a bnk or any other site that site that handles money and credit-cards people should be more careful with the warnings. I am not saying that they would be, but surely they should be. If you walk in the steet fanning your self with several 100$ bills don't you think sooner or latter someone will mug you? I believe that people can learn.

        The "dancing pig" is another thing. Browser should block every kind of executable of being run directl

  • What do most users do when they get a warning box? (Score:4, Insightful)

    by chroot_james (833654) on Thursday September 28 2006, @03:27PM (#16236189) Homepage
    "What is this window doing here?! I just want to get to paypal already..." *clicks ok* "There. Now I can finish this ssn and cc verification..."
  • Is it just hype or is this still an effective tool?
    • Lots of people still do. Because Some people will believe anything. The biggest problem has been and always will be the user. As I keep telling the associates in the center I support. All these computers were working perfectly before you got here.

    • Re: (Score:3, Informative)

      by xenoarch (817676)
      untill December of last year i was a sysadmin for a large ISP, and when i left we still had 30+ phsing scams caught per day. Phishing is a social hack, and those are always more effective then just plain tech hacks. And yes blocking is more effective then warning.

    • Re:Do a lot of people still get phished? (Score:4, Interesting)

      by porcupine8 (816071) on Thursday September 28 2006, @03:36PM (#16236387) Journal
      Just a few months ago, someone "broke into" my sister's PayPal account, and from there her bank account.

      A couple of months after the fact, my mom let slip that not only was this actually because she fell for phishing, but my mom had fallen for the same email - luckily, they didn't get to her bank account. (Mainly b/c when my sister discovered what had happened, my mom ran to cover her ass.)

      I wanted to whack them both upside the head. But trust me, they are far more representative of the average user than you or I.

      • Re: (Score:2, Interesting)

        by kers (847541)
        I am curious, how did they "get to her bank account"? A lot of my international friends have this scare of people getting to know their bank account number and I can't understand why. Is it really that easy to pull money from an account that *belong to another person*? Over here I need a valid ID or a PIN-secured cryptographic device (that look like a simple pocket calculator) just to move money between my own accounts. Is bank security really that terrible?

    • Re: (Score:3, Informative)

      by jd142 (129673)
      I get calls from people asking about emails from banks that they don't even do business with!

      Them: I got a message from XYZ bank that my account is frozen. Do you think it is a scam?
      Me: Do you have an account with XYZ?
      Them: No, I've never done any business with them.
      Me: Then you can be very sure it's a scam.

      • Re:Do a lot of people still get phished? (Score:5, Interesting)

        by merreborn (853723) on Thursday September 28 2006, @04:29PM (#16237307) Homepage Journal
        My fiance just started as a teller at a Wells Fargo. She says that people come in with questions exactly like that every single day, along with "I need a cashiers check to send to this nice man in Nigeria", and

        "I just got an email saying I won the Canadian Lottery, and I need a cashiers check for $4,000 to cover the taxes"
        "Did you ever _enter_ the Canadian lottery?"
        "No."
        "I hate to tell you this ma'am, but it's a scam."

        Every god damn day.

        • A fool and his money are soon parted.

          ---Thomas Tusser


        • Maybe if you make a sign that describes these most common scams, have it printed on a nice board with a very official looking Wells Fargo logo, and put it on the counter, these people will recognize their situation and believe you when you hear their story and point to it. Then maybe this board would be seen by a district manager, your fiance gets a raise for a great idea that protects the customer and fosters faith in the company, and similar fancy signs go to every Wells Fargo in the country just like th
    • For non-tech users ... this is a very effective method. We know to look for an SSL lock and to make certain the the URL matches the site, but this is total jibberish to non-techs who have no idea what a URL is supposed to look like, much less how SSL works. My advice is usually, if it comes in an email ... do not click on it. Whomever invented HTML email should be shot, it's almost as if it was purposely invented for this purpose.
    • This actually is an effective tool. You would be suprised how many people I have here at my work that getting phishing attempts. Not through there work email, but from checking ther web mail accounts. Here at my non profit organization I have a RedHat server setup with Dan's Guardian/ClamAV/Squid Proxy. We don't filter for content, but do filter for viruses using ClamAV. It works great and reduces the number of downloaded viruses in our organization. An added bonus to this configuration that I discove

  • I hate slashdot so much (Score:5, Insightful)

    by anotherone (132088) on Thursday September 28 2006, @03:30PM (#16236249)
    If anything, blocking a site should be worth more than double, since most people I know seem to just ignore warning dialogs.
    • If blocking a non-phishing site doesn't cost points, I'm sure I can come up with a filter that performs even better!

    • Re:I hate slashdot so much (Score:4, Insightful)

      by jrumney (197329) on Thursday September 28 2006, @04:08PM (#16236925) Homepage

      If anything, blocking a site should be worth more than double, since most people I know seem to just ignore warning dialogs.

      My first thought was that the false positive rate is probably going to be about the same as WGA, blocking far too many sites, but you're right. The ideal solution would be to have it configurable and default to blocking, since the users who click through without reading are probably not going to go anywhere near the Options dialog.

  • BS composite scores didn't make a huge difference. (Score:4, Informative)

    by dtfinch (661405) * on Thursday September 28 2006, @03:30PM (#16236261) Journal
    Disregarding their arbitrary scoring BS, and only looking at detection percentages, IE7 still did a good job, as expected from a Microsoft commissioned study.
    GeoTrust TrustWatch caught 99%, but had a 32% false positive rate.
    IE7 - 89%
    Netcraft Toolbar - 84%
    EarthLink ScamBlocker - 64%
    Firefox/Google - 53%
    eBay Toolbar - 46%
    Netscape 8.1 - 28%
    McAfee Site Advisor - 3%

    How they came out with only 89% when they selected the sites themselves is anyone's guess.
    • "How they came out with only 89% when they selected the sites themselves is anyone's guess."

      Perhaps they thought nobody would actually believe the 100% figure they had originally planned to report - after all, 89% of statistics are made up on the spot by a caucasian male under the age of 35...
      • 89% of statistics are made up on the spot by a caucasian male under the age of 35...

        ... as shown in the research done by Professor Togashi Raichu, a professor of Statistical Analysis at Tokyo University.

        Statistics are much more credible when backed by reliable sources.

    • Don't you think 99% or 100% would have been a little "phishy"?
    • GeoTrust TrustWatch caught 99%, but had a 32% false positive rate.

      I'd be interested to know about these false positives. I'd bet that some legitimate sites use designs that are are hard to distinguish from phishing sites. I would argue this is bad.

      Perhaps GeoTrust is right and the false positive sites are wrong.
      • I can't vouch for false-positives for websites, but Thunderbird routinely thinks that the monthly Sun Developer Network Program newsletter is a scam, and quite often labels developer mails from Microsoft as scams too. Ignoring the obvious jokes, it's irritating, especially as there seems to be no way to configure it (other than turning it off) and it completely fails to catch most of the real scam mails I get...
      • Maybe they also wanted to google/firefox to perform as badly as possible on the same dataset.

        Ding! Ding! We have a winner!

        Microsoft-sponsored benchmarks are almost always about making the other guy look bad, while inflating their own performance. Think of the 'Get the FUD^WFacts' campaign or the tests that pit Windows 2K3 Server against Samba, where the Red Hat box was tuned -- on purpose -- to the worst possible setting.

        You only have to look better than your next biggest competitor in Microsoft's playboo

  • Stupid questions (Score:3, Insightful)

    by Solkre (787360) on Thursday September 28 2006, @03:31PM (#16236271)
    Why do all article descriptions end with a stupid question?

    And for those who disagree, there ARE stupid questions.
  • Blocking a phishing Web site earned you twice as many points as just warning about it in this test, but is blocking really twice as effective as just warning users?

    In fact, blocking is pi times as effective as warning, so this result is even better for IE than it appears. (Yeesh, even by Obligatory Stupid Question standards, that one was pretty stupid.)

  • Actually.... (Score:4, Insightful)

    by zappepcs (820751) on Thursday September 28 2006, @03:32PM (#16236317) Journal
    It is the blocking part without user interaction that provokes that 'just click ok' reflex all the time. When the OS (or any machine, service, etc.) coddles the user to the point that they don't know what they are doing, or having the computer do, it breeds ignorance. No, I'm not dumb enough to think that all computer users must be sysadmins, but software that deepens their ignorance is not good software. Intelligent software should tell user's what is happening, why(if possible), and what the software can do about it, and/or what the user should do about it. I know that clippy was pretty annoying, but a less annoying and more intelligent approach like clippy would help user's to make better security decisions in the future. Just two cents worth.

    • Re: (Score:3, Insightful)

      by merreborn (853723)
      As much as I'd love to agree with you, your average user doesn't *care* what the computer's doing, or what their options are. They just want their email. They don't *want* to know any more than they absolutely have to.

      That, bundled with way too many dialogs asking them questions they don't know the answers to, has resulted in the "Just click yes" reflex.

      By way of example -- the first time you submit a form in any browser, you get that "You're about to send unsecured information over the internet!" dialog.

      • People don't care about what their car's doing or what all those road signs mean or why they should be looking ahead of them while driving, all they want to do is go places in their cars. But we still force them to prove they do know all those things they don't care about, on pain of not being allowed to drive, because their not knowing would endanger others. I fail to see why the same shouldn't hold for computers.

        • I don't think anyone's been killed by a computer-ignorant octogenarian checking his email.

          People have been killed by those not fit to drive. (And it's worth noting that the system hasn't proven too good at keeping those people off the road, by the way)

          Ignorant computer users pose a minimal risk to life and property.

          Theoretically, in the 'land of the free', we don't legislate activities that pose little risk to otherwise uninvolved parties. Of course, there are numerous examples of this not actually happen

  • Sadly, yes (Score:3, Insightful)

    by Angst Badger (8636) on Thursday September 28 2006, @03:35PM (#16236361)
    [...] but is blocking really twice as effective as just warning users?

    While I am loath to say anything positive about Microsoft, I'd have to agree with the scoring. Most end-users, especially the developmentally challenged ones that are prone to phishing scams, simply do not read warnings. If someone is drooling, it does no good to tell them. Just wipe their chin.

  • Yes... (Score:3, Insightful)

    by loraksus (171574) on Thursday September 28 2006, @03:37PM (#16236407) Homepage
    Because your average user is stupid and will click away any phishing warning, especially if the email says "You may see a dialog like this, click yes/ignore (just like installing your printer, scanner, tv card, etc drivers)"

    I really don't want to advocate handholding, but some people really do need it..

  • Template for MS Slashdot Articles (Score:5, Insightful)

    by derrickh (157646) on Thursday September 28 2006, @03:38PM (#16236435) Homepage
    Microsoft did something right...but is that something actually not wrong?

    Microsoft performed well...but is performing well more important than performing badly?

    Microsoft isnt all bad...but is not being bad the same as being good?

    D
    • More like "Microsoft did something right, but thier marketing department is pushing it to be way better than everything else in technically questionable ways."

      I'd add "Again." to the end of that, myself.

      Soko
  • At first when I read the post title I thought Microsoft was going to have an actual baking competition. "Wow," I thought, "That would be an awsome way to spread the antiphishing message to the common Windows user." Alas, it was not to be. Maybe I was just overcome by the image of apple pie cooling by the monitor, fresh from the gentoo box. *sigh* Memories...

  • Blocking a phishing Web site earned you twice as many points as just warning about it in this test

    This reminds me of when the "Quarterback Rating" came out back in the 80's. Back then, there were people arguing that Joe Montana was the greatest QB in history. Around that time, a "Quarterback Rating" scheme emerged with some esoteric weighting of various performance stats (completion percentage, TD's per game, etc. etc). Although nobody seemed to understand the rationale for the particular weighting...

  • The results of the study as below:
    1. Internet Explorer 7 Beta 3 RC3 with Microsoft Phishing Filter with a score of 172 points
    2. Netcraft Toolbar with a score of 168
    3. Google Safe Browsing on Firefox with a score of 106
    4. eBay Toolbar with a score of 92
    5. Earthlink ScamBlocker with a score of 76
    6. GeoTrust TrustWatch with a score of 67
    7. Netscape 8.1 with score a of 56
    8. McAfee Site Advisor with a score of 3

    Check http://www.3sharp.com/projects/antiphishing/ [3sharp.com]
  • The real solution is an email system with end to end encryption and digital signatures. Basically an email doesn't pop up in your inbox unless it passes these tests. The same with e-commerce sites. You sign up to a provider who allocates you a PGP key which is then published to a number of online directories. Why we don't have such a solution is that the security services won't be able to monitor our online activities.
    1. As an ISP, offer your users the ability to alias their mail address for companies they do business online with.
    2. If the user receives mail from that company not to the alias they registered, it's obviously a phishing attempt or spam. Heck, the ISP could just drop it altogether based on the mail routing information.
    3. profit?

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.