Forgot your password?
typodupeerror
Spam Microsoft

Microsoft Researching Anti-Spam Technique 660

Posted by michael
from the penny-for-your-thoughts dept.
Tim C writes "Microsoft's Research group are working on a technique to combat spam. Dubbed the 'Penny Black project', it involves making email senders perform a computation taking around 10 seconds, which their recipients can then check for. This delay would limit bulk emailing speeds to around 8000 a day, meaning that to spam all of those 'fresh, guaranteed 25 million addresses' would take approximately 8.5 years." We've reported on this before.
This discussion has been archived. No new comments can be posted.

Microsoft Researching Anti-Spam Technique

Comments Filter:
  • Question... (Score:4, Insightful)

    by Xpilot (117961) on Friday December 26, 2003 @12:17PM (#7812614) Homepage
    How do you "make" senders do anything?


    • by notque (636838)
      How do you "make" senders do anything?

      With large pointy sticks....
    • Re:Question... (Score:2, Insightful)

      by Sc00ter (99550)
      you don't understand, once the sender does this there will be some type of key. If the client doesn't see this key in the headers or wherever then it will be seen as spam by the reciving client.
      • So I would have to give up on linux to send email?
        • Re:Question... (Score:5, Informative)

          by the_mad_poster (640772) <shattoc@adelphia.com> on Friday December 26, 2003 @12:47PM (#7812861) Homepage Journal

          Calm down, killer. Microsoft's not THAT smart.

          It Is Not A Big Secret [weizmann.ac.il]

          At worst, I suppose Microsoft could make it's own scheme and try to push other people out, but I doubt that there are enough Microsoft MTAs out there to make that sort of system survive. If they implemented it for Microsoft-only, they'd almost have to give the option to revert to a traditional white-list when the sender can't play Microsoft's Holy Encryption Puzzle. After all. If you send someone an e-mail and outlook Express won't give it to them, just tell them that - Outlook Express won't let you look at it. I sent it, sorry. The problem is clearly on your end, call support for help.

          Microsoft HATES support costs and one thing you don't do on Windows is screw with grandma's emails.

      • Re:Question... (Score:3, Interesting)

        by Lord Kano (13027)
        you don't understand, once the sender does this there will be some type of key. If the client doesn't see this key in the headers or wherever then it will be seen as spam by the reciving client.

        How do you know if the key is valid?

        Why can't a spammer just make up a false key? Does the client check it mathematically? How long does that take? Why not just delete the spam manually (like we all do now) if it's still going to take time to filter it out?

        LK
        • Re:Question... (Score:4, Insightful)

          by MegaHamsterX (635632) on Friday December 26, 2003 @01:19PM (#7813082)
          With that question, I thought of another one....

          If this is so computationally expensive, what would happen to the mailserver if I sent...oh half a million emails with bad keys in them.
    • Re:Question... (Score:3, Interesting)

      by tomstdenis (446163)
      By rejecting their emails otherwise. D'uh.

      You really want to email me [or get priority over other emails] you will do as I say.

      Of course you can get to the point where it's too much hassle. I think MSFT is seeking to have this built into OE [e.g. integrated]

      Tom
    • Re:Question... (Score:2, Informative)

      by asquared256 (637499)
      by automatically rejecting any emails where the computation's results aren't present, like using cryptographic signatures?
    • Re:Question... (Score:2, Insightful)

      by Kierthos (225954)
      Oh, they could roll it out as part of a "required" patch that fixes other security holes, it could be part of the next version of Outlook, and as part of MSN... there are ways.

      What concerns me is how this would affect people who use Eudora, or yahoo-mail, or any of the host of other systems that don't require the Lords of Redmond holding their hands to send e-mail.

      It seems that it would be a stop-gap measure for anyone using MS products or services to spam, but unless it was adopted by every major (and ma
    • by conner_bw (120497) on Friday December 26, 2003 @12:31PM (#7812729) Homepage Journal
      I haven't read the story in question (this is slashdot, after-all) but while on the toilet i came up with this:

      I use a service called Spam Interceptor [si20.com] which uses SpamAssassin [taint.org] as it's scoring filter to judge if it should send an authentication message to the original sender.

      How about instead of doing that on the recipient's end, it was done at the SMTP server?

      The email is sent and the server runs it through the scoring process. If the message scores more than 6/10 the server sends the sender an authentication message, asking to validate the email [si20.com].

      This would require spammers to manually intervene and waste tons of their time. if they forged the sender email, their email would go to someone else's email and they would just trash it (and complain to a service like spamcop)

      A winner? Or a loser?

      --
      Kill all spammers. [si20.com] Let the irony of this sig sort em' out.
      • While this seems useful at first glance (at least open relays would stop working), how does your technique address these issues:

        1. Clueless admins (of windows or *nix servers) who refuse to use SA or similar? These are the same who leave the mail servers as open relays in the first place.

        2. People who use their own SMTP server

        Sure, go ahead and say that you can add reverse domain lookups. But registering a domain is quite cheap these days ($4.95 a year) and point the NS to your machine, set up MX records, and you're on your way.

        Your solution is useful, but not comprehnsive. I doubt there is a comprehensive solution short of making the spammers incapable of accessing the internet.

        --
        Clueless People? Everywhere I look, I see them. And some of them, they WORK here!
      • by John Hasler (414242) on Friday December 26, 2003 @02:10PM (#7813441) Homepage
        > The email is sent and the server runs it through
        > the scoring process. If the message scores more
        > than 6/10 the server sends the sender an
        > authentication message, asking to validate the
        > email.

        So you are one of those resposible for bomabarding me with those damn things.

        > This would require spammers to manually
        > intervene and waste tons of their time. if they
        > forged the sender email...

        They always do. My domain is a favorite.

        > ...their email would go to someone else's
        > email...

        Yes. Mine.

        > ...and they would just trash it...

        Isn't that what the spammers say? "If you don't want it, just delete it. What's the big deal?"
        The big deal is that about a quarter of my email is bogus bounces and useless "confirmation" message from systems such as yours.

        _NEVER_ _REPLY_ _TO_ _SPAM_
    • Re:Question... (Score:2, Informative)

      By refusing connections or refusing to send e-mail unless they do. Kind of like how SMTP servers "make" the senders do a HELO before sending the message. Like:

      220 mail.example.com SMTP server ready
      HELO client.example.com
      250-Hello client.example.com, calculate
      250 1+2+3+4
      ANSR 10
      250 Answer correct, continue
      MAIL FROM:<foo@example.com>
      ...

      or

      ...
      250 Hello spammer.example.com, calculate
      250 1+2+3+4
      MAIL FROM:<user@example.com>
      503 You didn't answer my question, go away

      although the computation would be

  • by tomstdenis (446163) <tomstdenis@gmail.TIGERcom minus cat> on Friday December 26, 2003 @12:17PM (#7812616) Homepage
    Well actually yeah they did. At Crypto'03 a method for memory bound HC was presented.

    So while MSFT didn't invent the original HashCash concept MSFT did improve upon it. So before anyone gets the bright idea of flaming MSFT ignorantly.... know your facts!

    Tom
    • what's your point? (Score:4, Insightful)

      by penguin7of9 (697383) on Friday December 26, 2003 @12:33PM (#7812740)
      Microsoft Research is no different from other industrial research labs: IBM, Bell Labs, etc. They hire the same kinds of people and get the same kinds of inventions out of them. One can't expect any more or less from any big company with a lot of money to spend. However, so far, MSR has not had much positive impact when it comes to driving innovation into the marketplace.

      If Penny Black is all there is, it doesn't look like that's going to change. It will probably be decades before we know whether MSR will have had lasting impact. By that time, Microsoft will probably be a benign, lumbering giant, just like its monopolistic predecessors, AT&T and IBM.
      • by Frisky070802 (591229) * on Friday December 26, 2003 @12:47PM (#7812850) Journal
        I fully agree that MSR hasn't had a huge impact thus far, but I don't think it's fair to equate AT&T and IBM's research arms in this fashion. AT&T's research has declined considerably in recent years as its (pseudo-)monopoly in long distance has dried up, and IMHO the company has done only a so-so job in translating research into practice, and in particular revenue for the company. Yet even then, no one can deny AT&T's impact with such things as the transistor, UNIX, C++, etc.

        On the other hand, IBM Research has done pretty well, though it too has gone through hard times. Its contributions to open-source are substantial, and at the same time, it's much more in touch with the demands of the company.

        Now, if someone had beaten me to it and moderated my parent as flamebait perhaps I'd have kept quiet....

  • not a solution (Score:2, Insightful)

    by Quasar1999 (520073)
    This is not a solution... as *I* still have to check for something on my end, and then discard if that condition is not met... my bandwidth and time are still wasted.
    • Re:not a solution (Score:5, Insightful)

      by notque (636838) on Friday December 26, 2003 @12:20PM (#7812654) Homepage Journal
      This is not a solution... as *I* still have to check for something on my end, and then discard if that condition is not met... my bandwidth and time are still wasted.

      Whine!

      It may not be the end all be all solution, but obviously we haven't found that yet. This seems like a pretty good solution for the moment. There may be a better one that comes out, making this one null and void, but we are continuing to find ideas which are a little better than the last.

      How can that be a bad thing?
      • It may not be the end all be all solution, but obviously we haven't found that yet.

        Maybe because people keep misidentifying the problem.

        The problem isn't that email is easy to send. The problem is that there are people who want something for nothing, and don't care who they harrass or steal from in order to get it.

        Solve that problem, and spam will go away!
    • Re:not a solution (Score:2, Interesting)

      by tomstdenis (446163)
      Your server can do the calculations for you. That's the point. You pay for email right? [if you don't run your own server]. Then why not expect your ISP to actually provide service.

      The idea though is that you can automate the process. E.g. unless the email has a tag on it that's valid you delete/filter the message.

      Tom
      • I should expect the ISP to have one server per 8000 emails? Why should they spend 100 times as much when the can just put in spam filters?
        • Re:not a solution (Score:3, Insightful)

          by tomstdenis (446163)
          I'd think the server would verify and the users would generate.

          Recall that verification is trivial while generation is what takes the time.

          Or the server could put the burden on the users.

          The idea is not to stop spam it's to make it easier to filter out. Spammers won't take a 10,000x fold penalty increase to spam with valid tags...

          Tom
    • Re:not a solution (Score:3, Insightful)

      by dustman (34626)
      No, it *is* a solution...

      Some of your bandwidth and time is being wasted in the short term, because spam is still being circulated.

      But in the long term, spam ceases to be an effective business model.
      • Re:not a solution (Score:5, Insightful)

        by walt-sjc (145127) on Friday December 26, 2003 @12:30PM (#7812723)
        Um, maybe you don't realize what spammers have been doing lately. They use huge networks of compromized machines to spam FOR them (thank you MS and your wonderful security model). There is plenty of horsepower out there to handle any kind of HC type system. The bottom line is that spammers ALREADY have the resources to make a HC system useless.
      • Re:not a solution (Score:5, Insightful)

        by schon (31600) on Friday December 26, 2003 @01:30PM (#7813170)
        No, it *is* a solution...

        No, it isn't. Three years ago it might have been a solution, but right now, it's just a colossal waste of time.

        The problem with this is that it operates on the assumtion that spammers work within the same boundaries as everyone else. Anyone who has spent even a tiny fraction of their time fighting spam knows this is simply not true.

        The days of spammers sending spam from a single server are long gone - nowadays, they use thousands of trojaned machines to do their work. How many machines do spammers control? Enough to launch effective DDoS'es on some of the largest pipes out there.

        The effectiveness of this 'solution' would be marginal at best.

        Now compare the effect it would have on legitimate users - an individual sending mail wouldn't notice 10 seconds.. but email is not only used by individuals.

        Something to keep in mind when assessing any anti-spam 'solution' such as this is the following:

        From a receiver's standpoint, the only difference between a legitimate mailing list and a spammer is that the user asked to be part of a mailing list.

        Now think about how this would affect legitimate mailing lists: How many mail servers do most mailing lists have? One? Two? Six? Some large mailing lists might have a dozen.

        So how does this affect those mailing lists?

        It would shut them down, is how. They would cease to be useful, as it would take days for their mails to get through.

        So the 'obvious' solution to this problem would be to whitelist legitimate mailing lists, right? Wrong. That's not a solution either (and we'll ignore the point that any 'solution' that requires exceptions is probably not very well thought out.)

        I maintian a mail server for a few thousand people. I have no idea which mailing lists they would subscribe to. It would probably become a full-time job to keep such a whitelist up to date. (And most users wouldn't have any idea to notify me in the first place - so the end effect is that they would subscribe, and then bitch about how they're not getting the stuff they signed up for.)

        This 'solution' does not solve anything, and will create more and worse problems than it attempts to solve.
        • Re:not a solution (Score:5, Interesting)

          by p7 (245321) on Friday December 26, 2003 @03:31PM (#7813906)
          You are missing the point. Nobody is saying that this is going to be required for all machines. Essentially it is an extra header attached to emails so email recipients can filter messages that don't have this tag. As I see it this is how it would work for most end users.

          First setup a whitelist, make this your first spam check. On the whitelist? Email goes through never checking for any other spam criteria. (Mailing list should be accepted here).\
          For mail that doesn't pass the white list check we can check for the header created by the MS program. We verify that the computationally intense header is correct and maybe we can let that through if we want, maybe I let emails with this tag pass through my spam checker with a higher spam score.
          If we decided to accept mails with the header, we now check the remaining email with a very thorough spam checker and use a very low score.

          No matter how many computers they have, it will lower the number of emails that are able to be sent, if people filter on this criteria.
    • It depends, maybe it's the kind of problem that is really hard to solve, but simple to verify, something like a NP problem. So validating the email would be very quick on the receiving end.
    • Re:not a solution (Score:3, Insightful)

      by xigxag (167441)
      No, *you* don't have to check for anything. Your email client will check, and could easily be programmed to discard the email sight unseen if it doesn't contain the appropriate validation code.

    • It's also not a solution becuase there isn't an easy way to have widespread adoption (yet), which would be required for it to work. Also, it would just give birth to a new generation of email worms, only this time the zombie computer it infected would be used for DDoSing AND for computing hashes.
    • Re:not a solution (Score:3, Interesting)

      by ReadParse (38517)
      I have two points. First, I think you're wrong about that. They speak in terms of the sender and the recipient taking actions, but I think they're referring to software on the sender and recipient computers taking these actions, and not humans. The only action that was clearly intended to be taken by a human was the part about agressively whitelisting good recipients, which is definitely something that I anticipate users will need to be willing to do.

      The second point that I have is that the whining is i
  • Technique? (Score:3, Insightful)

    by conner_bw (120497) on Friday December 26, 2003 @12:18PM (#7812627) Homepage Journal
    This technique is only good for (closed source) Exchange servers. Also, what good is this research if Exchange is the recipient of spam, not the sender?!

    --
    Kill all spammers. [si20.com] Let the irony of this sig sort em' out.
  • by monadicIO (602882) on Friday December 26, 2003 @12:18PM (#7812629)
    Is it something that will require using Outlook on Windows to work? Alternatively, will I be force to use some MS software just to send mail to people who are using MS based web/mail/etc client/server programs?
  • by baseinfinity (18023) * on Friday December 26, 2003 @12:19PM (#7812636)
    We studied this in a computer security course I took. This technique has been proposed to TCP establishment as well. It involves the server calculating a hash of a particular nonce (random value). The server then provides the hash and a certain number of bits of the nonce. It becomes the clients job to complete the nonce such that the value hashes out correctly. The server can vary the number of bits it provides to vary the difficulty of the puzzle...
    • So how in the world does this work with a new email program sending mail to an old email program? Or vice versa?
      • It's transparent to that. All this has to do with is if you want to use a service of a server (sending mail). This strategy doesn't have to be global, you could tack it onto any authentication protocol and it would only be the senders job to get the required software. However the reciever authenticates is the buisiness of the server they recieve from.
  • Phew (Score:4, Funny)

    by Lord_Dweomer (648696) on Friday December 26, 2003 @12:19PM (#7812639) Homepage
    From the article:
    "The payment is not made in the currency of money, but in the memory and the computer power required to work out cryptographic puzzles. "

    Phew!!! For a second there I thought I was going to have to do a math problem for each email I was going to send. I woulda been fucked!

  • ...and I'm sure all the spammers in countries I've never heard of with .xyz top-level domains would be happy to use their $0.28 copies of the latest and greatest Microsoft OS to comply.
  • I know, I think microsoft should charge the customer for each and every message that is routed through a exchange server. Just think of the money they could make and help curb spam.
  • by UnderAttack (311872) on Friday December 26, 2003 @12:21PM (#7812666) Homepage
    Even today, the most annoying spammers are not using their own computers, but insteady they are bouncing e-mail off virus infected and trojaned PCs.

    So 8,000 emails / day is fine, if you have a couple thousands relays to pick from.

    • This is what I was thinking. I like the idea of making email expensive - it's a good idea in theory, but I am also thinking that spammers might be able to use trojan boxes not only to send their batches 8000 mails but to even do their calculations, like many distributed networks already function.

    • Damn straight. All the spam I get is from stupid people on campus who have insecure computers that spammers gain control over and send spam with.

      Let's say you leave your gun safe unlocked and someone comes in and takes your guns and kills somebody. You're going to get sued for big moneys. If you leave your computer "unlocked" and someone sends spam with it you should be held accountable in some way.

      Spam is an international problem and is very difficult to stop. But there are known spammers in the unite
    • So this would have the effect of making legitimate high-volume, high-subscribership mailing lists expensive to operate (unless subscribers configured their MTAs to accept "unstamped" messages from the list, which is annoying and error-prone -- and has an obvious "workaround" for the spammers).

      <tinfoilhat mode="on">Ha! Now we see Microsoft's *real* goal... to slow Linux development by shutting down the kernel mailing list!</tinfoilhat>

      Seriously, though, any attempt to make e-mail expensive hampers those who have a legitimate need to send lots of e-mail.

      Plus, there are obvious workarounds that will be developed in short order. A hardware stamp-generator could probably cut the stamp generation time to practically nothing, particularly since their approach somehow depends on memory/CPU latencies rather than processing time. You might be able to make a much faster stamp generator by running it on your graphics card, and custom-built hardware could certainly do it.

      • So this would have the effect of making legitimate high-volume, high-subscribership mailing lists expensive to operate

        Well, maybe. There still could be a white list for cases like this.

        I think that high volume mailing lists should probably actually be newsgroups anyway. But what it does do is put a crimp in people who host a lot of low volume mailing lists.

        • Well, maybe. There still could be a white list for cases like this.

          I think that high volume mailing lists should probably actually be newsgroups anyway. But what it does do is put a crimp in people who host a lot of low volume mailing lists.


          As somebody who hosts low-volume mailing lists, I have to agree.

          Whitelists are nifty (we use them extensively), but what worries me on that score is that if they become frequent, I suspect we'll just see spammers hijacking address books along with machines, and forgi
  • Qouted from the article:

    But, he said, for such a scheme to be all-encompassing, there would have to be some provision for open standards, so that it is not proprietary to Microsoft.

    Glad the guy from MessageLabs hit the nail on the head right away... what are the chances Microsoft will go along with THAT idea? They'll implement this as an Exchange/Outlook only feature, if they can get away with it...

    And, a poster above me states that Microsoft basically invented this, giving me reason to believe there i

  • by FreeUser (11483) on Friday December 26, 2003 @12:23PM (#7812670)
    Count on Microsoft's "cure" to be worse than the disease itself. You would think for $40 billion they could buy just a little more intelligence than that.

    SMTP needs to be redesigned. Not by Microsoft, who will use any change in the protocol to tighten their monopoly grip, locking in their customers (and locking out the non-Microsoft world), but by the IETF.

    Spammers having to do a computation before delivering email isn't going to limit them to 8000 pieces of mail a day, it simply means they're going to cluster all of those Windoze boxes their custom worms have infected, and let those millions of PCs do the work for them in parallel. SPAM won't decrease one bit, but the load and toll it places on those who use the net will go up significantly.

    The solution isn't to increase the cost of email (computationally, bandwidth-wise, or financial), the solution is to repair the design flaws in SMTP (and, for that matter, USENET, something that remains the most useful medium on the 'net despite its widespread abuse) that make SPAM a viable methodology.
    • Scrap SMTP? (Score:3, Insightful)

      by sethadam1 (530629) *
      Before you chuck the entire protocol, do you have a solution for a better one?

      Until you know how you're going to repair the problem, let's not get too excited about scrapping a protocol that still has a lot of flexibility. I've learned a lot about SMTP in the last few months, if there was universal agreeement as to WHAT to do, we could probably accomplish it in place.

      What are the options? Whitelists, blacklists, red lists, gray lists, hash cash, filters, etc. No one can agree HOW to combat the problem.
  • by bigberk (547360) <bigberk@users.pc9.org> on Friday December 26, 2003 @12:27PM (#7812702)
    How is my older hardware (or even pretty recent hardware on a huge ISP, with lots of SMTP activity) supposed to be able to handle this? Bah. It seems to me that adding computational difficulty is not such a great way to combat spam. Do you have any idea how effective IP blocklists [openrbl.org] and statistical filters [sourceforge.net] alone are? (Or, you could combine them as this project [pc9.org] is doings).
  • Okay.. (Score:5, Insightful)

    by NegativeK (547688) <tekarien@hot[ ]l.com ['mai' in gap]> on Friday December 26, 2003 @12:28PM (#7812704) Homepage
    If this works as stated, then I can see issues.. For instance, large mailing lists. Would they have to be white-listed? 3000 seconds of computation is a heavy tax on a community based program like the Linux Kernel Mailing List, which averages 300 messages to my inbox a day. Also, there's the issue of viral spammers.. Those that send out viruses to do the spamming for them. If you infect enough, 8000 mails per day per computer can still be quite a bit.

    Personally, my whole take on spam is that everything needs to be done on the user end. Laws have loopholes in every situation (foreign spammers being a large one,) server restrictions are either too restrictive on small servers, or can be defeated with distributed computing.. I say we stick with Bayesian filtering. It works _wonders_ for me, and I'd love to see more people use it.
    • Re:Okay.. (Score:3, Informative)

      by Sparr0 (451780)
      You already opt-in to mailing lists by subscribing to them, which takes anywhere from 10 seconds to 5 minutes depending on the list. Would it be so hard to add them to a client-side white-list, perhaps an additional 10 to 30 seconds, in addition to subscribing?
    • Re:Okay.. (Score:3, Informative)

      by Lord Kholdan (670731)
      If this works as stated, then I can see issues.. For instance, large mailing lists. Would they have to be white-listed? 3000 seconds of computation is a heavy tax on a community based program like the Linux Kernel Mailing List, which averages 300 messages to my inbox a day. Also, there's the issue of viral spammers.. Those that send out viruses to do the spamming for them. If you infect enough, 8000 mails per day per computer can still be quite a bit.

      Personally, my whole take on spam is that everything nee
    • This doesn't have to be a big problem for mailing lists.

      You know how mailing lists require you to confirm your membership? Well, this confirmation mail would have you add the mailing list to your whitelist. As a result, future mailings on that list would be let through without having to do the computation.

      The mailing list could simply refuse to deliver mail if you ask it to do the computation, or it might give you a one time warning that you have to add it to the whitelist, or similar.

      But all it takes

  • 1) Needs to work between MTAs. Your Exchange server might trust the Outlook client, but my exim server doesn't trust your Exchange server. Be prepared to pay again.
    2) No-one discovers a mathematical short cut for the hash.
    3) What are the calculation costs on the recipient?
    4) The Intel "Spammer Edition" Pentium 5 with a half gig of L1 cache. Memory bandwidth is no longer a bottleneck.
  • My group alone generates hundreds of e-mails to people outside our domain every day. I'm sure they whole company easily exceeds the 8000 mark mentioned here.
  • What about mailing lists? Until we recently upgraded, we were doing reasonably OK with a Axil 320(Sun Sparc clone. No, not an UltraSparc, a sparc. Yes, that slow) for about 3,000 subscribers. One of our lists was at least 30-40 messages a day.

    Ten seconds of P4 3ghz time is about....half a year for a 110mhz microsparc ;-)

    We've since upgraded- but I can tell you right now that anyone who tries to make us leap through these hoops will simply find themselves removed by Mailman for bouncing. Like those

  • I searched the article for Mozilla and Thunderbird, but Firebird reported the words were not found.

    Hummm...doesn't look like Microsoft is really serious.

    :)
  • This is just a fancy way of saying "Microsoft is trying to figure out how to turn off Hotmail"
  • This is an interesting idea -- I don't know how it works in a world where some people are running 133 Mhz computers and others are up at 3Ghz. But it's interesting.

    I think that any postage scheme should be hybridized with a white list to avoid imposing burdens on people you want to talk to. The postage (economic or computational) should only apply to people who you don't know.

    In other words, if I know you, you should be able to email me for free, but if I don't know you, it should cost something -- not
  • GPU's? (Score:2, Interesting)

    by Naksu (689429)
    The idea was originally formulated to use CPU memory cycles by team member Cynthia Dwork in 1992.
    But they soon realised it was better to use memory latency - the time it takes for the computer's processor to get information from its memory chip - than CPU power.


    Don't GPU's have a lot smaller memory latency?

    hmm, whats this?
    BrookGPU: General Purpose Programming on GPUs [slashdot.org] ;)
  • Uhm (Score:4, Insightful)

    by geeveees (690232) on Friday December 26, 2003 @12:38PM (#7812784) Homepage Journal
    If it takes a long time to send out bulk email, what about all the mailinglists people subscribe to? How would lkml or sourceforge lists continue to operate?
  • by dybdahl (80720) <info&dybdahl,dk> on Friday December 26, 2003 @12:41PM (#7812813) Homepage Journal
    This seems to be a "let's fix this by limiting what technology can do" case.

    Instead, they should focus on adding more functionality to the smtp protocol. For instance, they could add sender e-mail address verification. You can't check the actual e-mail address, but you can make a "dial-back" TCP connection to check, if the e-mail is known by the mail-server that belongs to the sender e-mail address.

    Combined with law enforcement, blacklists etc., this is extremely effective.
  • Why not just.... (Score:5, Interesting)

    by rongage (237813) on Friday December 26, 2003 @12:57PM (#7812934)
    Ok, I'll bite - why not just insert a "sleep (10);" line into the connection response of sendmail (or qmail, or whatever MTA you are using)? By making the sender wait 10 seconds before delivery can begin, you get the same effect as a tar-pit...
  • by Angst Badger (8636) on Friday December 26, 2003 @12:58PM (#7812945)
    The programmer who works next to me used to be a construction worker. Every so often, I come up for an idea for some kind of home project, explain it to him, and he tells me a way to accomplish it that is much simpler and more reliable.

    This MS solution is almost a caricature of one of my own over-done home improvement ideas. Why bother with some elaborate cryptographic system to delay inbound emails? Why not just have the receiving SMTP process call sleep(10) at the beginning of the SMTP session? You get the same desired slowdown, and all you have to change is the SMTP server software. There's no need to modify MTAs, promulgate new standards, or fit yourself more tightly into the MS monopoly noose.
  • by eaolson (153849) on Friday December 26, 2003 @01:05PM (#7812998)
    OK, I may be missing something here. The point of this method is to make the sending computer jump through some sort of computational hoop that takes about 10 seconds, so that it can't just send a huge amount of mail in a short time.

    So why bother with all the computation and hashing, and just refuse to accept connections from a given IP except every 10 seconds? So if an email was sent from AAA.BBB.CCC.DDD at 00:00.00, don't accept another from that IP until 00:00.10.

    This makes it happen entirely at the recipeient server side, so you're not breaking SMTP, and it's backwards compatible with everyone else.

    On the other hand, if it's 10/sec per email it doesn't sound like this would be feasable to implement:

    • Hotmail receives about 2,000,000,000 spams per day. Let's say the amount of legitimate email they handle is 10% of that.
    • legit emails: 200,000,000
    • emails/day at 10 s/email: 8640
    • necessary servers to handle this amount of email: 23,000
    OK, this is a bit of an oversimplification because it assumes that in that 10 s, no other server is trying to send mail to that machine, but it's a rough guess.
    • The idea is not to take longer sending one email. Spammers don't send spam one at a time and wait for the first one to be finished before sending the second one. The idea is to force the spammer to spend something, specifically in this case 10-20 seconds of CPU time, per message. If all you are doing is sleeping 10 seconds, the spammers can out multithread you and just wait, while making 10000 other SMTP connections in parallel doing the same thing. The rate of messages will ultimately be the same but i

  • by KC7GR (473279) on Friday December 26, 2003 @01:08PM (#7813015) Homepage Journal
    Something that the Redmond Empire conveniently neglects to mention is that an awful lot of the spam is due to virus-compromised systems running -- you guessed it -- Microsoft Windows! I've lost count of the number of broadband IP ranges, notably from Shaw Cable and Comcast, that I've had to dump into our domain's local 'Reject' list thanks to their endless attempts to propagate Swen, SoBig, or whatever the latest spammer-zombie trojan is.

    Perhaps, if Steve 'Uncle Fester' Ballmer and his cronies had paid more attention to basic security to begin with, or had taken the trouble to actually try and educate their customers about the most basic computing security steps, there wouldn't be such a huge problem now.

    This 'Penny Black' nonsense looks like nothing more than a means for them to make money off a mess that they created in the first place.

  • by clickster (669168) on Friday December 26, 2003 @01:15PM (#7813062)
    I actively subscribe to a lot of tech sites that have tens of thousands of subscribers. Slashdot is one of those sites. How many people have Slashdot e-mail their mail to them? How are legitimate bulk mailers (of their own content, not ads) supposed to send out newsletters, etc.)? If a retail outlet with a legitimate opt-in newsletter needs to send it to 50,000 or 100,000 people, what kind of hardware upgrades are they going to be looking at. I mean, I can add them to a trusted senders list on my side, but that doesn't tell them that they no longer have to run the computations. "If I don't know you, I have to prove to you that I have spent a little bit of time in resources to send you that e-mail. How do you know whether you "know" me or not? Does the user's mail client alert the sending server that it approves of mail from that SMTP server? Once senders have proved they have solved the required "puzzle", they can be added to a "safe list" of senders. Whose list? My personal list that is part of my mail client? My mail service's white list? Microsoft's special white list?
  • Email Fiefdoms (Score:3, Insightful)

    by rakeswell (538134) on Friday December 26, 2003 @01:35PM (#7813198) Homepage

    Having read the article, I was impressed by how clever their proposed solution was, though since I don't have a CS background, I don't understand how a mathematical computation can be essentially bottlenecked by memory latency -- I'd love it if someone could give an explanation of how that works.I'm guessing that some cryptographic hash needs to be held in memory, such that the nature of the data structure and physical access to it proves a bottleneck. This is probably way off.

    But having read the /. comments, it becomes clearer to me that this solution, and many other proposed solutions face problems insofar as they "break" the assumed contract under which email has worked for so many years. To me, this seems to boil down to a challenge / response system (allbeit one that increases the overhead of the transaction signifigantly). The problem with these systems is that for a time, email will be broken for certain people, or broken when trying to communicate with certain people depending on whether or not one has migrated to the proposed system. I'd worry that this would have the effect of segmenting email users into little fiefdoms determined by which email system they are using.

    I don't think a migration can happen unless there is some "benevolent dictator" who can force everyone to migrate to such-and-such a new email model and system, and frankly, I wouldn't want that forced on us.

    It seems that the challenge to any such spam-reduction system is that migration must be immediate and non-backwards-compatible, and universal, otherwise for a time email users will be segmented into little fiefdoms based on whether they've migrated, and solution to which they've migrated.

  • old and embraced (Score:3, Informative)

    by Tom (822) on Friday December 26, 2003 @02:10PM (#7813436) Homepage Journal
    The technology is fairly old, it's known as Hash Cash [hashcash.org].

    It has known shortcomings, but it is one of the best solutions out there.

    Its main problem, however, was not yet known when it was invented: That spammers would control huge zombie networks, as they do today.
    With 100k zombies (which is not uncommon), the spammers can still send out 10k mails per second, or those 25 mio. spams the topic speaks about in under one hour.

  • solution (Score:3, Insightful)

    by shokk (187512) <`ernieoporto' `at' `yahoo.com'> on Friday December 26, 2003 @03:09PM (#7813805) Homepage Journal
    So the solution is for spammers to set up compute farms of cheap old hardware with an open soure version of the mailer. Since memory latency matters, and not processor speed, the solution is to have access to more than one computer. A farm of 10 machines then sends out 80,000 messages a day. A real super computer farm funded by a spammer alliance could get back to shipping millions of spam messages a day. What was the cheapest supercomputer cluster mentioned on Slashdot, something like $30,000? Is that really all that much money when you consider that a group of spammers could split that and amortize over many years? Remember, age of the hardware is not a consideration, just CPUs with access to memory segments. How about a very large system with hundreds of virtual 386 processes running 128k memory segments?

    I think in the long run only something more expensive will deter most spam, but will not succeed completely. Case in point is all the junk mail we still get in our real mailbox. Someone out there is paying for postage to send that crap, yet they still ship it to me so that I can place it in my trash can.
  • My simple solution (Score:3, Insightful)

    by KalvinB (205500) on Friday December 26, 2003 @03:12PM (#7813819) Homepage
    Instead of hitting the delete button I started putting spam in a folder for later analysis. What I found is that spammers use affiliate programs. For example, I recently got a porn spam with an image from

    http://gallery7.withsex.com/

    All I do is block withsex.com with an expression filter and all spam that's afilitated with that site goes away. Spammers can't ofuscate an URL otherwise it won't work. The image linked from the same site is 28KB. If that spam was sent out to 25 million people and all of them looked at it once that cost the spammers 667GB of transfer. On a standard DSL line it would take about 6 months to transfer that. These companies need a dedicated host to allow them that kind of bandwidth. The company may have a number of domains for the site but spammers aren't going to be using random ones to advertise it like they use random from e-mail addresses. They also have to keep the domains functional or all that spam goes to waste.

    Not many hosts would allow that kind of bandwidth transfer without charging up the nose for it. Which limits the number of hosts that spammers will use for images. 2004Hosting.org/.net is a big one for the cable filter and "banned CD." 530000x.net is also affiliated with those spams.

    http ://www.silverstate.co.sy@click.com-click.com.ph/cl ick.php?id=sicosyl

    click-net and click-com are what spammers use to get paid. If you click on a spam link, most likely it goes through a common domain to log the referal to calculate how much the spammer gets paid. Block the referal site and all spam that uses that referer to get paid is gone.

    For example

    http://www.xswcde.biz/index.php?id=173&affid=561 &c ampid=
    342

    Is a big e-bay spammer site. I block xswcde.biz with an expression filter and all e-bay spam from that company goes away.

    It basically boils down to blocking the company and not the spammer. My spam count went from about a dozen a day to 1 or 2 and they also have obvious tells. If possible I also block the domain in the from address. Using a web-form cut down on spam quite a bit as well.

    Ben
  • Why not... (Score:3, Informative)

    by The Master Control P (655590) <[ejkeever] [at] [nerdshack.com]> on Friday December 26, 2003 @04:22PM (#7814176)
    Simply de-allocate the IP blocks of any ISP that continually harbors spammers, meaning it refuses to terminate them immediately? They can't spam if they can't connect to the internet!

    And to "strongly discourage" any ISP that would consider flaunting this rule, they get zero compensation for that netblock they paid for and are denied from buying any new netblock for a time (possibly a week).

    Because this would necissarly work on the level of ARIN and the root DNS servers, you can't avoid it, because those are known, reputable organizations who will have no choice to comply.

    Can anyone think of a way you *could* avoid this?
  • E-mail list killer (Score:4, Insightful)

    by Black Art (3335) on Friday December 26, 2003 @04:35PM (#7814237)
    I don't think this is a good idea.

    First, it would kill legitimate mailing lists. Imagine what the perl5-porters list or the Linux kernel list or any of the other high traffic mailing lists would have to do to keep operational. Large mailing lists already have problems with lag. This would just add to that.

    Also, there does not seem to be anything that would stop them from doing these operations in background and just contact multiple sites while working on the problem. They would just multi-thread the mail spammer or just hijack more machines to use as their slaves.

    This technique requires replacing every mail program out there to support the protocol. Of course, they will just make it a condition to connect to exchange. Might be a way of getting people away from having to talk to compromised Windows mail servers.

    This is a bad solution for a big problem.

    "Something must be done! This is something, therefore we must do it!"
  • research? microsoft? (Score:4, Interesting)

    by MoFoQ (584566) on Friday December 26, 2003 @04:43PM (#7814273)
    M$ should consider out-sourcing it since well....my hotmail account still gets spam even though I set it to exclusive (meaning only email from ppl in your address book will get through); spam with obvious fake addresses. And the spam that goes through this "exclusive filter" also seem to fly passed my custom filters that have the words that the spam has ("financial", "viagra", "herbal", etc.)

    Yahoo works better with regards to spam though I wish it would empty the bulk mail folder more often.

    And my pop3 acct has something called greylisting and that alone cuts 95% of spam. Plus black and white listing IPs and domains helps too (for instance, only allowing email from hotmail.com if it originates from one of hotmail's servers, etc.) and blocking known spam-haven Class C ranges (eg x.x.x.*).
  • This is just hashcash.

    Hashcash is wasteful... it just runs processes at full blast for tens of seconds to tens of minutes at a time, which is a small energy waste but overall a loss.

    Hashcash is impotent... any hashcash scheme cheap enough to let someone with an older computer send mail in less than minutes won't slow down a P4-3GHz at all.

    Hashcash is harmful, because it makes no distinction between solicited and unsolicited mail. How would you subscribe to Slashdot without whitelisting it?

    And once you're whitelisting senders, you might as well just whitelist everyone you get mail from, and now you only need to discourage unknown senders. And hashcash is still a silly solution there, how about real cash?

    Here's one way to do that. Whitelist not a sender, but a server. A server at a company that simply charges a few pennies to a few dollars to forward mail (you pick the level of unsolicited mail you want), or one that requires other hoops...

    Much simpler, doesn't require new proprietary Microsoft technology, and allows all kinds of alternatives...

An optimist believes we live in the best world possible; a pessimist fears this is true.

Working...