Novel Method for Universal Email Authentication 212
MKaplan writes "Most spam is sent using spoofed domains. Email authentication schemes such as SPF attempt to foil spoofing by having domain administrators publish a list of their approved outgoing mail servers. SPF is sharply limited by incomplete domain participation and failure to authenticate forwarded email. A paper describes a novel method to rapidly generate a near-perfect global SPF database independent of the participation of domain administrators. A single email from an unauthenticated domain is bounced and then resent — this previously unauthenticated domain and the server listed in the return path of the resent bounce are entered into a globally accessible database. All future emails sent from this domain via this server will be authenticated after checking this new database. Mechanisms to authenticate forwarded email and to nullify subversion of this anti-spam system are also described."
Greylisting? (Score:2, Insightful)
Fails to account for SMTP farms... (Score:5, Insightful)
This also requires users to install software to use effectively, and features CAPTCHAs which are a usability nightmare and not nearly as impregnable as the author thinks.
All that effort instead of just adding a TXT record to their domains.
FUSSP (Score:4, Insightful)
The author may be an anti-spam kook [rhyolite.com] but the paper is so badly written I can't be bothered identifying which.
Not exactly. I think. (Score:5, Insightful)
Then he talks about having people install software:
Yeah, installing new software is a great solution.
Major flaw in methodology (Score:5, Insightful)
The proposed scheme ignores one thing: the majority of bounce messages today are false bounces caused by spammer joe-jobs, therefore they themselves get flagged as spam and deleted/ignored. In addition, it also increases the annoyance of greylist authentication schemes, since a spammer forging my address in the From field will cause every host participating in this scheme to send me a verification e-mail for a message I didn't send which I'll have to deal with. The proposed scheme makes a very fundamental mistake: assuming that you can trust the sender's address in a message to be the true sender's address. You can do that only after you've determined the message is authentic and not spam, at which point you don't need this scheme anymore.
Re:Not exactly. I think. (Score:1, Insightful)
Now if you could bounce the message, it would just go back to the original IP, so I don't see why that would help either though.
Re:Fails to account for SMTP farms... (Score:4, Insightful)
And OKing the receipt of any address at a domain from such an infrastructure seems less than ideal. I mean, if I send out all my email for "me@mydomain.com" from Hotmail's SMTP servers, I'm not sure I want that to automatically give the go-ahead so that anyone can send spam from "Need-Viagra@mydomain.com" and "refinance-your-house@mydomain.com", etc..., from those domains.
SPF, as I understand it, has some contexts in which it works well. But it doesn't cut with fine-enough a blade as far as I'm concerned. Automating the process so that I (if I haven't set up SPF records) could allow spammers to use my domain with more authority by responding to an automated message just doesn't sound like a good idea. I think this opens up the door for a lot more spam if people believe in it.
If it went a step further and tried to authenticate each time a unique USER@DOMAIN pair sent an email via a particular host, I could see that being useful. The protocol could be extended such that even the SMTP farms could conceivably use something to say, "if authorized at one of my servers, an email should be authorized at all of my servers". But it's a lot of work to get there, and the size of such a universal database would be ridiculous, and it seems that for there to be a single-source host for such a thing, there would have to be a lot of cooperation between some major corp^H^H^H^H sources of funding.
Re:Fails to account for SMTP farms... (Score:3, Insightful)
The long and the short of this is that during a very large-scale distributed dictionary attack, having a server attempting to verify return paths, as this "novel" idea suggests would be nuts. Just getting your mail servers to cut the connection is going to be enough work. Why in hell would you want to multiply the traffic that a goddamn attempted spam is already taking up. I guess for that lucky bastard who never has to pay per gigabyte or whatever could use this.
Re:No, I didn't RTFA.. (Score:5, Insightful)
Re:No, I didn't RTFA.. (Score:2, Insightful)
[ ] "My approach is immune from all criticisms"
[X] "Doing SOMETHING is better than nothing!"
[ ] Willfull ignorance of founded criticism.
Yes, it's a worn out joke (and yes, the form is a JOKE, it applies to ALL current antispam approaches). Yes, moderators are stupid. You must be new here.
Re:That's the problem. (Score:3, Insightful)
This is exactly why greylisting is effective. It pushes the cost of spamming back on the spammers. Now they have to have a semi-legitimate mail relay, vs. fire and forget. If everyone greylisted, then the spammer's mail queues would be huge.
Of course, all bets are off with zombies that start using legitimate SMTP servers, but there are solutions to that already in place:
The only place this fails is if the spammers as part of their owning of zombie hosts begin to check for the proper SMTP server to relay through and configure accordingly. Admittedly, this is not too difficult to do, but they aren't doing it yet.
Re:Still barking up the wrong f'ing tree... (Score:3, Insightful)
I receive approximately one spam email every 45 seconds. Constantly. Without spam filtering, I would go to bed with an empty inbox and wake up to 500 spam emails. Spam filtering, far from being futile, is the only thing that makes email usable for me. Without spam filtering, I would simply have to give up on email.
Can it stop all spam? No. Do filters have to adapt? Yes. But that hardly means that filtering is futile, it just means that it's not as easy as we'd all like it to be.
Re:Still barking up the wrong f'ing tree... (Score:4, Insightful)
Nope. We need a solution involving cruise missiles though bedroom windows late at night.
We need Spam Assasin Ninjas clad in impregable black carbon-fibre capes with the knives of cutting edge technology and the deadly intent of artificial intelligence enhanced mania.
We need mountains of spammer bodies piled high on the forefront of technological .
We need chain gangs of spammers publicly televised chanting "The Only Good Spammer is a dead Spammer" to the sound of hammers hitting rocks.
IN Summary: Cruel and inhuman tortue is not enough for these guys
Re:That's the problem. (Score:3, Insightful)
Huh? When I take a look at how many mails are bounced on all my domains, thanks to greylisting, each day, and hold it against how much spam actually enters my mailbox, i'd say they haven't adapted at all.
When you are sending millions of mails, retrying is far, far more expensive than just ignoring it.
Re:Greylisting? (Score:4, Insightful)
I don't know, I didn't get that far. The article and the concept is bullshit.
The 'From' field is the keystone of their identification process. Well, I got news for you if you bothered to read the RFC. 'From' does not have to represent the real sender. I can forge it up all I want into anything I want and you can't tell. I didn't get past section 3 where this is before I determined the rest isn't worth reading.
Once again we have another company trying to come up the next Big Thing and they don't know what the hell they are talking about. SPF is cute -- but relies too much on people setting it up and correctly. I suppose you could pay a service to act as a third party validator, but that's turning into a boondoggle too.
I don't think bouncing email at valid senders is going to win any friends.
Perhaps there is a way to do it successfully and with great accuracy. I would love to say I'm working on it. But quite frankly, if I do figure it out I probably won't mention to anyone since I really don't want the legal hassle of trying to defend my idea against someone else's billions. I can block spam. I can block spam to the tune of 99+%. The rest is trivial. I was even surprised to hear them say 94% was the average. Perhaps people would be better off if they stopped using SpamAssassin.
Sorry, my opinion is that statistical filtering is more than sufficient if it's managed well. I think few people are willing to do the work required of them to make them spam free. Kind of like locking the door to keep out the crooks.
Re:Greylisting? (Score:5, Insightful)
At the end of the day, you're right. Statistical filtering, with the careful use of all of the above solutions (though I think whitelists/blacklists are as bad as the problem they attempt to solve) is the only way to reliably filter spam. You're never going to catch it all, but the ISP I worked at was catching, by my estimate, about 90% to 95%, which meant that a guy getting about fifty spam a day was down to three or four, and in many cases less than that. It does mean work, there's no solution that doesn't require monitoring, management and tweaking, because the spammers are smart bastards who learn the tricks as fast we can come up with them.
So many reasons it doesn't work (Score:3, Insightful)
- It causes backscatter
- It doesn't work with mail from mailing lists
- It's not accessible
Additionally:
- It doesn't work well with sites that have many MTAs (requires one bounce/CAPTCHA per MTA)
- It doesn't work well with an SMTP server that sends for many domains (requires one bounce per MTA per outgoing domain)
- It merely confirms that "this server can send mail for domain X". If you've got a spambot and can determine your user's domain name (e.g. comcast.com), this won't stop anything at all.
The author brushes off concerns with bold (well, italic now) statements like:
Resend software is a simple onetime update for webmail systems, email clients, and local mail servers...Universal Distribution of Auto-Resend Software is a Surprisingly Simple Thing to Achieve
Hah! A simple one-time update for all servers and clients everywhere! Granted, RIA doesn't depend on that update happening, but it's clear even the author thinks it'd be a pain without auto-resend.
There is little disincentive to implement Auto-Resend software as it is a one-time upgrade that remains dormant until needed.
There is a huge disincentive; looking up a user's mailbox to see if he did, indeed, send the message you claim he sent is a ridiculously expensive operation, if it's even possible at the server level. It could also lead to a privacy leak if done wrong; people could forge RIA bounces to probe outgoing mail flows.
At best, it potentially doubles the volume of outgoing mail, which deepens queues, requires more disk space, etc. etc.
I'm guessing the author is unfamiliar with high-volume mail sites - the very ones he wants to implement this scheme first.
Suspicious Domains Will Be Neutralized By CAPTCHA Encoded Sub-addresses
Great. So now e-mail that's "suspicious" requires intervention from a sighted human, and all his "auto-resend" silver bullets are used up. He does imagine yet another client change that will "nicely reformat" a CAPTCHA. Yeah, right. Oh, and now he's e-mailing me graphics on my Blackberry.
In general, he seems to imagine that he personally runs the One True RIA list, and we all trust his determinations of what is and isn't "suspicious", with reputation scores, rate limiting, etc. That is, of course, ridiculous; the original MAPS RBL has splintered and grown to the point where there are over 200 DNSBLs available.
He talks about automatically e-mailing users that he has "detected" are running zombies. Right, because that's a good idea and isn't spam.
Domains commonly associated with phishing (e.g. Paypal.com, Citibank.com)
As if there's a way to create a comprehensive, or even useful, list of "domains commonly associated with phishing".
with the passage of time it will become difficult for spammers to purport that all of their spam is sent via increasingly obsolete or esoteric brands of software.
Of course it won't. I still get spam from "The Bat!". Before, he forgot about the big guys; now he's forgetting about the long tail. Spammers can make up any number of X-Mailer names.