Yahoo! Develops Anti-Spam Architecture 283
prostoalex writes "Yahoo!, the owner of one of the largest e-mail systems in the world, is said to be developing a cryptographic product that will be offered freely to mail servers. 'Domain Keys,' according to the Reuters article, would require the message sender to authenticate in order for message to come across a trusted e-mail network. The idea has been around for ages, however, it required someone from the big league like Yahoo! to step in." While Yahoo! isn't the first name that comes to mind when I think of trusted email, it's still a step in the right direction.
Oh yeah it seems like a good idea right now.... (Score:3, Interesting)
Could you imagine this becoming really popular and then Yahoo! getting bought by someone like oh say Microsoft? (or any other big commercial interest)
Re:Oh yeah it seems like a good idea right now.... (Score:2, Insightful)
Of course, Microsoft will probably figure out a way to break it so that it only works with their products but that's a different story...
Re:Oh yeah it seems like a good idea right now.... (Score:5, Insightful)
Presumably a 'domain key' is some cryptographic element that authenticates that your domain is who it claims to be. To me this sounds an awful lot like SSL where a third party issues the keys, or acts as a clearinghouse for self-issued keys.
Either way, Yahoo could be the man in the middle acting as either issuer or clearinghouse. Think of it this way, OpenSSL is open sourced, but that doesn't keep the SSL issuers from having a lock on that market.
Lock-in isn't necessarily an issue (Score:4, Insightful)
I don't see how lock in will be an issue. Imagine the following scenario:
I can't see how this would neccesitate a clearinghouse.
Re:Lock-in isn't necessarily an issue (Score:2)
Re:Lock-in isn't necessarily an issue (Score:2)
Trusted email? (Score:2, Funny)
Re:Trusted email? (Score:5, Insightful)
They have a pretty good spam catching service.
It puts suspected spam in a "Bulk" folder. You can
review this folder or just like it get purged after 30 days. Nice. You can also click on the "its not spam" / "this is spam" buttons to help them tune.
They offer a SSL login and it was discuessed recently on Slashdot that they use the Javascriptcrypto library to calculate MD5's on the client side and send the digiest for seduvcity (maybe when you are not logging in with SSL).
You can check your POP3/IMAP mailboxes. The resources come back color-coded.
Good uptime. Always available.
It's free. You can enought resources for reseaonable use. But you can buy more if you want.
All this sounds exactly like a crypto-nerd and slashdotter would design a mail service. And this new thing is going to be opensourced!
Re:Trusted email? (Score:2)
Re: (Score:2)
Re:Why? (Score:2)
Re:Why? (Score:3, Informative)
My experience over-all has been excellent, with on minor exception:
Oh come on! (Score:3, Interesting)
Whereas the latter completely true, I think the weakness of the argument is a testament to the idea being an excellent one. CPU horsepower is very very cheap. If Yahoo think they can do it, then who exactly will have a problem ?
Just as long as I can incorporate it into my server, I'll be a happy bunny - all the other proposals put forward so far seem to limit the mail providers to the big boys
Simon.
This needs HYPE (Score:3, Interesting)
As soon as the certified network is considered a valid alternative to the current spam-ridden, scam-infested open email exchange s
Temporary (Score:3, Interesting)
In all seriousness, I think this is a good idea. But, sadly, it's going to be cracked. Domain keys can be forged, and that will be the first thing that these spam servers will be focussing on right now. They'll set up a Yahoo acct and monitor traffic to see what the domain keys look like. They will then duplicate the acks and be back in business. It's only a matter of time.
This is a good step, no doubt. It is just that we should be looking at ways of putting spammers out of business, too. Hit their wallets, not their tech. Tech can always be worked around, especially by dubious people.
Instead of domain keys, I had a different idea that might work a lot better.
What if nobody sent email over the Internet?
Today we have the ability to use web forms to pass messages back and forth to other users on the same service. With that option, the server admin would be able to flag spammers and ban them. If you wanted to message another user of another server, you could type in their location as USERNAME@DOMAIN, and that would queue to be sent in batch to the other server after authentication.
No outside contact. No spam. One message per customer. If you send more than a certain number of messages in a day, they are held as possible spam.
Privacy goes out the window, but hey... it's not like there is any privacy in non-encrypted email anyway.
Not necessarily (Score:5, Interesting)
Personally, I'd like to see two things.
1. The software Yahoo! is developing should be open-source, so nobody can monopolize it. At the very minimum, the protocols involved should be well documented so open-sourcers can make their own implementations if they have to.
2. Give this software a few months to propogate to a good chunk of the ISPs out there. Then, Yahoo! should announce that they will NOT accept any email that is not signed with this software. I'll guarantee that everyone will be using this new protocol in a matter of weeks, since no ISP wants customers screaming because they can't get mail through to Yahoo! accounts.
Re:Not necessarily (Score:3, Insightful)
Re:Not necessarily (Score:3, Insightful)
Seems like a big business case to me - last I heard business didn't like spam. (.. except the spam business I suppose)
Re:Not necessarily (Score:2, Insightful)
Re:Not necessarily (Score:3, Insightful)
1) Software needs to be based on open standards. RFC90210 or something like that... Others need to be able to make implementations.
2) Yahoo's implmentation should do ONE THING WELL. It shouldn't try to stick an advertisement on the bottom of my emails the way their groups tools do.
3) Give the software a few months to propagate to a few major ISPs.
4) On a given date, all email going through those servers that are not 'si
Must be missing something (Score:5, Insightful)
I'm assuming that what is sent out is an encypted token for which the public key can be used to decrpyt, so:
So, the token to be encoded will change from mail to mail, thus making replay techniques pretty much impossible, I think. At least, that's the way I'd do it, and I'm pretty sure I've seen it presented before as well...
On the other hand, I ain't a security expert, so there's probably a gaping hole in the above
Simon
Re:Must be missing something (Score:2)
> the private key is delivered as a message header!
> Hmm, not very private...
That just means that the reporter is ignorant and careless. In other words, the usual kind.
Re:Must be missing something (Score:2)
Oh well, to think about it, it'll be "domain keys", so it won't be so hard to cache, I'm not sure how many domains with SMTP servers running out there, but say there are a particular server receives most of its email from a thousand particular domains, when each has 1024 bit keys, it would only take
1000 * 1024/8 = 128000 bytes = 125 KB
to store the public keys of those domains (not counting overhead).
The real problem would be trying to decode all
Re:Must be missing something (Score:2)
Re:Must be missing something (Score:2)
I'm still not really convinced about the CPU argument. Anyone to whom it would apply is almost certainly an ISP or large company, and they can just throw slave machines at it, even as basic as using round-robin DNS for mx.mycompany.com.
That's assuming the receiving daemon does the check, of course (which would be best, I reckon). You could always devolve the processing to the mail c
Re:Not at all. (Score:2)
My point above is that it's only a matter of time before we have to scrap the email system and start over -- because of spam.
Re:Not at all. (Score:2)
No permanent damage can be done, if they use well known algorithms and do it right.
OS? (Score:3, Interesting)
Re:OS? (Score:4, Informative)
Trying 64.157.4.78...
Connected to mx1.mail.yahoo.com.
Escape character is '^]'.
220 YSmtp mta108.mail.sc5.yahoo.com ESMTP service ready
It looks like they run YSmtp, just like everyone else I know. In all seriousness, I'd imagine there isn't much of Yahoo's infrastructure that isn't highly optimized for Yahoo's own use. I think that Yahoo did a lot with FreeBSD at one time, but I'd presume whatever they have isn't just an out of the box app.
Re:OS? (Score:2, Informative)
http://www.qmail.org/top.html
and search for "Yahoo". I also know it from an independent source because I discovered a bug in qmail:
http://www.washington.edu/imap/IMAP-FAQs/index.
while tracking a bug report cocerning my MUA.
Re:OS? (Score:2)
So now... (Score:3, Funny)
Re:So now... (Score:5, Insightful)
step, by step, the spam problem can be solved. That doesn't mean that you should not take the first step simply because it doesn't provide a total cure.
Re:So now... (Score:2)
Re:So now... (Score:2)
AT least I know a company advertising with Yahoo is supporting something I care about. (a lot of mailing lists on groups.google.com...) So long as Yahoo doesn't get gready. In radio they only sell so many comercials, because they know eventially listeners change stations and when listenership goes down advertising dollars goes down. Maximizing profit is the goal, and they have figgured out how to do that. (though personally I think they all have way too many comercials)
When I get a spam for enlargeme
Open standards? (Score:5, Insightful)
Re:Open standards? (Score:3, Informative)
Re:Open standards? (Score:3, Interesting)
Why's that? If Yahoo doesn't accept email from anyone except the biggest 50 companies in the world who could afford to take part, you can place a bet that there won't be many people using their email service anymore.
Re:Open standards? (Score:2)
Free? (Score:2)
if they do not share the technology completely, this is a bad thing as yahoo will have some kind of power over all email servers. the article doesnt go into those kind of details... does anyone know any more, and like to share the knowledge?
What do they mean... sends a private key? (Score:2, Redundant)
Is this a mistake, or is there some other terminology this is following?
Re:What do they mean... sends a private key? (Score:2)
They mean something is encrypted with the sender's private key, not that the key itself is sent.
Re:What do they mean... sends a private key? (Score:2)
Read the article (Score:2)
I'll be interested to see how the details of how they attempt to protect the system from key forgery.
You just can't win with the /. crowd (Score:5, Funny)
Re:You just can't win with the /. crowd (Score:5, Funny)
No, Xor is the operation most often used in cryptographic functions...
Broken already? (Score:5, Interesting)
Re:Broken already? (Score:3, Informative)
Maybe I don't understand the problem. I thought Yahoo's new scheme was designed to authenticate the mail server that originated a transaction with a Yahoo mail server, not to authenticate the domain in the "From:" line.
Re:Broken already? (Score:2)
Re:Broken already? (Score:2)
1. HELO domain. Used in "Received:" headers.
2. Envelope FROM. Used by MTA.
3. "From:" header. Used by end-user's mail s/w.
Re:Broken already? (Score:3, Interesting)
That is correct. Yahoo's scheme is to provide authentication for the Received: headers, not the From: header. Currently, the Received: headers frequently get forged, so it is hard to tell where spam is coming from. A real person can usually tell fairly easily, but you can't reliably tell a computer how to do it. It would be
Re:Broken already? (Score:2)
Yahoo's software won't work because most mail servers won't be set up with it.
That's basically what the author of that article says. He gives two examples of mail servers that won't be containing the required software: his own domain, and a forwarding service on another domain. So what? So use a server that's set up with that software, and it works. Anyone not using that software will still
Re:Broken already? (Score:2)
Summary of his argument: You couldn't send mail as foo@mydomain.net thorugh the mailservers at operated by your ISP.
That really doesn't need to be an issue. If you legitimately own mydomain.net, you need to generate a public key/private key pair for that domain. Then configure mydomain.net to offer that public key. When you send mail through any server with a "from" of mydomain.net, you need to also use the mydomain.net private key. Your mail client should be able to do this easy enough. The mail se
Double take (Score:2)
Even if the ISP is for some reason taking over the whole job of sending the e-mail, it's very trivial to set up
Not sure if I understand it right (Score:3, Interesting)
If the spammer...or anyone for that matter is spoofing a header anyway, it shouldn't be difficult to find out the encrypted private key, since it is sent out with every message originating from the domain.
I could, presumably send an email from my secure email address to a non-existent email address of the domain whose encrypted private key I wish to find out: eg bounce@email.com. The bounced message should have it in the header.
Re:Not sure if I understand it right (Score:5, Interesting)
The authentication token would likely be some sort of hash of the message contents. In that way, a token is only valid for that particular message. The sender would generate a checksum of the message, encrypt it with a private key, then transmit the encrypted checksum as the token. The receiver would generate the same hash of the message contents, and decrypt the token with the public key. If the decrypted checksum equals the generated checksum, then one can be confident that the message came from the server it said it came from.
Re:Not sure if I understand it right (Score:2)
So what about a teergrube? (Score:3, Interesting)
Re:So what about a teergrube? (Score:2)
Re:So what about a teergrube? (Score:3, Interesting)
One is to set up a Teergrube/Tarpit (it's easy using the Linux ipchains TARPIT target) on a machine that shouldn't receive any mail by SMTP. You can tarpit everything, and nothing will get lost. (I think this is something everyone should do; it'd be neat if this sort of functionality was built into those little Linksys/Dlink firewall boxes..
One solution (Score:5, Insightful)
1) Who will issue the keys?
2) Is anonymous mail possible if the receiver allows it?
Furthermore spamming is a social problem emerging from our commercial world and technical solutions can never be 100%. What if:
a) I send spam from a "secure" domain?
b) forge certificates?
c) the certificates are too expensive? (like SSL, I think it should be included with a domain)
I like the "Bayes" spam filters best. You get 99.5% spam protection and keep anonymous mail.
We all see the need for authenticated senders (biz communication, etc.), but we should be careful
Re:One solution (Score:2, Informative)
When a mail from that domain goes via the mail server, the mail server will calculate the hash of the message and encrypt with the private key and ad
Re:One solution (Score:2)
> generate the keypair.
Why would you need special tools? What's wrong with Gnupg and PGP?
Re:One solution (Score:4, Insightful)
We should expect something like this to come from the IETF, but big corps do good things all the time. What makes you uncomfortable about it? The privacy issue? If it's on the net and you want privacy, encrypt the content. But if you want to hit my network w/ SMTP, much less an ICMP package, I want to at least know who you are.
Are you worrying who will govern the entire thing? Who do you trust? Some
romancing the stone (Score:3, Interesting)
Me personally, if spam makes it through my filter, I ban off the offending address working my way up towards the class c - b - a. All attempts at a port 25 connection is drop point blank, http, https, etal are kept open. I also have dontspam#somefreemailaccount.com's to use for form shit. Once in a while when registering for say an upper-crust website account, I'll use something like msndoesntspam@mydomain.com to see who exactly is sharing my addresses, then null the account if I see anything odd coming in to that account, and never trust the site again. Procmail works the most wonders though.
User account verification (Score:5, Insightful)
This enables SMTP callbacks to stop spam being spoofed "from yahoo", just like everyone else does.
good to hear (Score:2, Interesting)
Thanks! Again! Yahoo!</elRegStyle>
-bZj
So where's the info? (Score:4, Interesting)
Re:So where's the info? (Score:2, Funny)
Are cycles that cheap? (Score:4, Interesting)
Bala Krishnamurthy at AT&T Labs has given a number of talks recently, including to the IETF, on a spam disincentive program he calls SHRED [att.com]. My understanding is that it uses offline cryptographic computation to amortize this overhead and distribute it to parties willing and able to devote the computational resources.
In any case, the tag line for this article had it right, standardizing this will be hard and heavy-hitters like Yahoo will need to take the lead. But a key problem is getting the new system to interoperate with the old.
Re:Are cycles that cheap? (Score:2)
The advantages are
Re:Are cycles that cheap? (Score:2)
I'm glad you're humble about it -- let me be the first to try :)
I think the idea of propagating abuse notifications is great, in theory. I'm wondering how it works in practice. There's nothing that says a sending host has to route mail via various intermediaries -- the spammer can simply inject a message via a willing ISP that aids and abets the spammers. You send a notification to that ISP, and it drops it on the floo
Re:Are cycles that cheap? (Score:2)
That's right, and we don't care. The alert goes back as close to the originator as it can. They still have to process the message to find out it's an alert. For spammers sending lots of messages, that generates lots of alerts for everyone nearby, and stops the flow at the first non-spam server.
The key is viewing spam as an error condition, and trying to notify the source of the error.
One thing
Re:Are cycles that cheap? (Score:2)
If server spammer.com sends mail to user@yahoo.com that is spam, it can potentially go straight from spammer to yahoo, as far as the mail headers go. Sure, lots of routers see the mail at the IP level, but that's not known to yahoo when it gets the mail. All it does is send a complaint to spammer, which drops it on the floor.
So in this case, yahoo is the first non-
Are cycles that cheap? Yes, in comparison. (Score:2)
As to interoperability: during the adoption period, one would have to accept both signed and unsigned mail, but as soon as it becomes obvious that mail-signing is a way to get legitimate mail past ever-stricter automatic mail filte
Only for GPL players? (Score:3, Insightful)
From the article: Yahoo said its "Domain Keys" software, which it hopes to launch in 2004, will be made available freely to the developers of the Web's major open-source e-mail software and systems.
But later: Garlinghouse also argued that Yahoo's proposal should be attractive to other e-mail providers because it is free and comes with no special restrictions. Is the GPL considered a "special restriction"? Will it not actually be GPL, just available to open systems?
I'm guessing that you'll need to be a GPL mail server to both require the private key for receipt, and to be able to use the system to give the email the private key for sending. So, what will this do to non-open mail systems?
Is Yahoo trying to break MicroSoft's mail service? Will this work? What's MSFT's option--reverse this and include it in their system anyways? Switch to an open system for a mail server, like, say, something based on a BSD license? Or ignore it, in an attempt to deprive it of critical mass?
Indeed, this might all be moot; Yahoo might make it free and available to everyone, either on a free system or a non-free system; the article isn't clear as it says both. It could also be that MSFT already uses an OSS mailserver in IIS for all I know about MSFT product. But I suspect this is a power-grab, like everything else these days. And, I have to say, if it is I wish Yahoo the best of luck--this would be another demonstration of the power of OSS; it allows the community to change together on a dime and play well together. Whereas makers of proprietary systems each have to modify their own systems with their own coders.
Re:Only for GPL players? (Score:2)
what about fowarding services? (Score:2)
Consider t
Re:what about fowarding services? (Score:2)
The obvious extension of this idea is if you have th
identity based antispam is censorship tool (Score:5, Interesting)
This is why I have put my efforts into sender-pay systems and specifically the camram project. We invite you to please come and join us in the effort to build a decentralized, user-friendly, freedom-of-speech supporting antispam system and hit spammers in the pocketbook.
camram antique documentation [camram.org] (too busy writing code to write new documentation)
Re:identity based antispam is censorship tool (Score:2)
BEWARE THE BIG RED Y! (Score:3, Insightful)
*waves hands ominously*
Yahoo beats eariler proposals? I hope not. (Score:4, Interesting)
Yahoo's size doesn't give that much weight to their proposal. Yahoo's email is not used in business to business communication (do not count hot dog stands as businesses), so businesses can just aswell block everything that originates from *@yahoo.com if it is not directed to their consumer service department.
Also, reverse mx [mikerubel.org] records provide much of the same benefits with minimal alterations needed to current email infrastructure. One DNS record added and small change in MTA software.
If Yahoo would really like to do a service to the internet community, they should rather consider looking AMTP and reverse mx records.
Too resource intensive, and broken anyway (Score:3, Insightful)
Under Yahoo's new architecture, a system sending an e-mail message would embed a secure, private key in a message header. The receiving system would check the Internet's Domain Name System for the public key registered to the sending domain.
If the public key is able to decrypt the private key embedded in the message, then the e-mail is considered authentic and can be delivered. If not, then the message is assumed not to be an authentic one from the sender and is blocked.
For every message, I have to check and unpack the header, go out to some PK server, and validate the keys, before I decide to accept/reject? That introduces a big latency into SMTP.
Also, this doesn't do anything to stop 'legitimate email marketers'. There's a death penalty (blacklist) for a site or particular sender's key, but nothing to stop a spammer from changing keys and starting over.
Or will everyone have to get their own key pair? Who's going to validate them, and at what cost per key pair?
This won't do a thing to stop spam, and imposes too big a burden on the infrastructure and on the 99% of us who don't spam.
Not for me (Score:3, Insightful)
Read: trusted network == commercial network
Why do you think this is in the "Money & Investing" department (see the linked article). No, this isn't for me. Businesses may well choose to use something like this for their communications, but they will not have the pleasure of communicating with me. While SMTP has its flaws, it still allows any IP host to send mail to any other IP host and that is a good thing.
To gain insight into what's going to happen with email and Internet communications in general over the next couple of years, you have to adopt a business mindset to see it from their eyes. There is a big problem (spam) hence a potential to make money. Various companies are going to try and cash in on this situation by offering a solution that might very well decrease spam -- some sort of commercially controlled communication network -- but this is definitely not in the best interest of the Internet. Of course, it's in the best interest of the company that's peddling the solution (duh!)
The Internet isn't Compuserve, or AOL. It's a network of IP hosts, and those are the entities which should have a facility for sending communications back and forth. There is no need for a central carrier for communications
Why does no one seem to get it? (Score:2, Informative)
All mail servers will have a public/private key of some type. The public key will be stored in the DNS system as extra data.
When an SMTP server connects to another SMTP server, the sending server will encrypt something (likely a checksum) with the private key for the domain the mail is from (likley the envelope from, not the From: header) and place it in a header.
The receiving server will then grab the public key f
Re:Why does no one seem to get it? (Score:3, Insightful)
I get it, because it sounds like an idea I've been bouncing around for a while (e.g. See previous [slashdot.org] comments [slashdot.org] of mine)
The mail server could add a header saying if the domain was verified and spamassassin could then adjust the spam rating of the message appropriately. Eventually servers would be configured to refuse mail from unverified domains.
Exactly, and the main advantage of this is the network effect - if yah
Yahoo as the big SPAM fighter .... not so !!! (Score:2)
When I asked Yahoo why they didn't do any SPAM filtering and even not deleted those very well known virusses, their reply was that I should upgrade to one of their small business offerings, which prov
Glad to see it coming from Yahoo. (Score:2)
It's in Yahoo's best interest for this to become an open standard. And I'd much rather see an open standard than something like Palladium become a de-facto standard that
Leading to a standard (Score:4, Insightful)
This process was used to create the internet today, including all of the network protocols and services that run on top of it. Even SMTP was an RFC first.
This is a large, stinking pile of bullshit. (Score:3, Insightful)
Public key spam control - technical implications (Score:3, Informative)
The basic idea, as I understand it, is that the DNS for a domain holds a public key, and mail sent with a "from" address in that domain must be signed with that public key. That's an old idea, and not all that bad. You create your own public/private key pair; you don't have to buy a "certificate" from somebody. (I think.) If you control a domain's DNS info, you can send mail from anywhere with that domain listed as the sender, as long as you know the private key.
For the free-mail services, it's fine. All their mail is authored via web applications and sent from their own servers. Only the service has the private key. Only the outgoing SMTP servers need to know the private key. That's the Yahoo Mail case.
If you own a domain, you should have full control over your own public and private keys. But adding additional info to a DNS record is not well supported by most hosting services. If you're not running DNS yourself, you may have problems setting your public key. Hosting services have to support this.
Signing can occur either in the original user agent (the SMTP sender) or in a mail forwarder. It's easier to implement this in mail forwarders, but if you want to send using a return address other than the one of the mail forwarder you're using, your user agent has to know how to sign mail.
If you're downstream from an ISP and don't control a domain, the ISP owns the key for the domain and can control what they sign. That has implications. They might force you to use web mail, for example. Or run their client software on your machine.
Spammers can still register domains, run their own DNS, sign their mail, and spam. It doesn't really stop spam.
Your public key is now valuable, and a target for spyware and viruses. Expect to see viruses that steal public keys from (inevitably) Outlook and send them to spammers. Or just send spam from the attacked machine.
What this really does is provide a clear way to identify joe-jobs using addresses from major mail services like Yahoo Mail. That helps Yahoo more than anybody else.
Re:Yahoo! Mail & me (Score:2)
Re:Yahoo! Mail & me (Score:2)
Of course, I prefer the web-interface. Much less bandwith usage when you get on the order of hundreds of pieces of spam a day, like I do. Most spam-fighting clients download the whole email and then filt
Re:Private key misinterpretation (Score:2)
Re:I've looked into Yahoo's plan (Score:2)
So, care to enlighten us about what exactly is wrong with their system?
Re:I've looked into Yahoo's plan (Score:2)
This is basically SSL tagging for emails. Have a mail server and domain, have your own private key, sign every of your email and they'll be distributed across the certified network. Abuse the system and your key is revoked/refused by the rest of the network. Don't have a key and domain, or forge the header to abuse your ISP's mail service ? No cookie for you.
Re:Sorry people, who I moderated but... (Score:2)
Re:who watches watchers? (Score:2)
> control in the hands of an international body.
Why does anyone need to be put "in control"?
Re:Cut your cat. 5 or see this in you inbox! (Score:2)
Between procmail and spamassassin, I don't see that stuff anymore. It is sent directly to /dev/null without ever having to first pass by my eyes.
Just produce a whitelist so that any idiot friend/family member can still send you those great viagra or erection jokes.