Earthlink Deploying Challenge-Response Anti-Spam System 520
deliasee writes "The Washington Post reports that Earthlink is preparing to offer new spam filter technology that requires sender authentication. AOL is still concerned that such technologies will put too much burden on consumers." The day after it's deployed, every legitimate mailing list on the planet will get challenges from all the Earthlink subscribers...
Correction (Score:5, Informative)
every legitimate mailing list on the planet will get challenges from all the Earthlink subscribers
Not exactly right. It happens only for the first time to detect whether the sender is legitimate or not. Quote the article:
The system automatically recognizes future e-mails from the same sender, so the verification needs only to be performed once.
The problem with this system is that the spammer can still spam using legitimate e-mail accounts as a camouflage (or expired e-mail accounts). Once the legitimate e-mail address is procured, the spam still goes on. It is futile, IMHO.
Warning: Infinite loop detected (Score:2, Informative)
Re:Now the spammers get address validation for fre (Score:3, Informative)
Good idea, bad idea. (Score:5, Informative)
Squirrel Mail [squirrelmail.org]
SpamAssassin Config for Squirrel Mail [squirrelmail.org] <- Register Globals must be turned on in php.ini to use this.
Now, that being said, I run an ISP in St. Louis, and spam is a problem, but for the precise reason mentioned on the submission, I can't use a challenge-response system. The reason is that our support staff equals myself plus 1. If I want to answer phone calls all day from people complaining about not being able to get mail from their daily spamming of mailing lists, I best allow all. The problem is that these same people complain about all the spam they get...ugh. The above solution is elegant and leaves the ability to control the filter to the end user via webmail. If they don't like it, set the threshold high and it's 'off'. Been using this for months without a complaint.
Now if you don't use lists, and it's for your own mail server...go for it. That has to be the most effective method available, but not appropriate for wide scale use.
Re:Intrusive and Easily Fooled (Score:5, Informative)
Then, I give the address to all my fellow spammers and we use it until it dies. Then we make a new one.
You missed the point. You would have to do this _per user_ you wanted to spam. Which would get a little tedious to say the least. The point of challenge/response is that most of the reply-to:'s are fake email addresses. Hence, the challenge bounces and the message doesn't get put in the users inbox.
Re:How do two people with C/R communicate? (Score:4, Informative)
From the article:
So if earthlink people are on your mailing list, you'll get a challenge next time you send it out. It should only happen once, and from then on, you're email addy is "legit".
It's not like you get 9000000 challenges from everyone on the list. But if every ISP did it, you'd get a challenge from every ISP on the list.
This is the first step towards email being such a pain in the ass, that people just no longer bother using it.
Kiss SMTP and POP3 goodbye.
Folks, It's Opt In (Score:3, Informative)
There's a whitelist (Score:4, Informative)
The challenge-response system will be optional and free for EarthLink subscribers, Anderson said. It will allow users to automatically clear the e-mail addresses of friends, family members and other associates in their electronic address books, so those people would not receive the challenge e-mail.
That's called a "white list"-- a list of addresses you know are legitimate.
When someone responds to a challenge and you accept their response, they go on your whitelist.
When you turn on this gadget, add your mailing list addresses to your white list. If you suddenly stop getting a list, go find out if they changed their sending address and add it to your white list.
If that's too much of a burden, feel free not to use the service, and go back to complaining about spam.
Re:How do two people with C/R communicate? (Score:5, Informative)
My C/R setup (TMDA) automatically put anyone I send email to on my whitelist; therefore I'd get their challenge message.
How has this problem escaped me? (Score:2, Informative)
Is this true?
Of all my email accounts, the only one I ever get spam on is my yahoo account, which I set up pretty much to get spam on, since any websites I visit that require registration, I always give them the "spam" address I got for free. I don't even check that email for anything. Human beings are the only recipients of my paid email addresses. I am for measures like this though, because even though I'm not affected directly by spam, increased traffic on the net is bad for everyone.
We need to punish the sensless posting of one's own email address to anonymous sources. These are the same people that give out their address and phone numbers when they buy batteries from radio shack. Use your head, they don't want to know where you live so they can send you a case of scotch. They want to drink your beer, crash on your couch, sleep with your daughter, and have you pay them for the privelege.
You can do this yourself. (Score:5, Informative)
Re:How do two people with C/R communicate? (Score:5, Informative)
-Esme
Proper scenario, better way (Score:4, Informative)
Alice@me.com sends an email to Bob@you.com
Mailing program adds "Bob@you.com" to Alice's list of valid emails (after all, you're not often going to send email to somebody that you don't want responding, right?).
Bob@you.com sends a challenge to Alice@me.com
Alice@me.com accepts the challenge, since she already sent the original email to "Bob" and had him added as an authorized user
Alice authenticates to Bob's system, and all is good
Another way would be to make all "challenge" type emails follow a specific pattern - with little to no allowance for anything other than the challenge. Then, challenges will be accepted as legit without bouncing back-and-forth, and spammers cannot simply send a message as a challenge with extra spamcrap attached - and still cannot send non-challenging email.
Now, an ignorant spammer could send a flood of challenges just to be annoying, but this isn't very profitable as they wouldn't be able to contain penis/viagara/etc ads.
Challenge-response works as part of a whole (Score:3, Informative)
In theory, someone could send me a spamlike message and would have to reply to the autoresponder. In theory, a spammer could validate himself. In practice, those two things almost never happen. The system catches about 150 spams a day and over 90% of its autoreplies immediately bounce. Last time I analyzed it, only about 2% of my legitimate correspondents had hit the autoresponder (note, that's a fraction of a percent of my total legitimate email, since a given correspondent only has to validate once.)
I have yet to see a notification from Amazon, my bank, or other similar email trip the filter. Haven't had any of my correspondents complain yet, but I have had a couple of them ask how they can set up the same thing for themselves.
So if it's implemented carefully, I think this could be a big win for Earthlink subscribers and more or less invisible to everyone who communicates with them.
Re:why challenge-response won't work (Score:3, Informative)
That's a good point, but the solution is simple: throw-away addresses.
If you are an earthlink subscriber, you get an email address like nanogator@earthlink.net. (Hey, that useta be my address!) Then, Earthlink could provide a service where you create a unique address that expires after x amount of time. so nanogator.dkaf3fj39@earthlink.net becomes active, and that's the one you use. From there, you can add them to your whitelist.
It's a bit round-about, but that's the beauty of Earthlink. They're a major ISP. Surely places like Ebay will have to stand up to comply with the upcoming standard. It'll never happen if some people don't have issues like this.
It can work - if implemented correctly (Score:5, Informative)
First it is important to note that the challenge system at Mailblocks is not something that can be automatically replied to. Much like the signup verifications for many forum systems out there the Mailblocks challenge email is simply a link to a web site. On that web site is a dynamically generated .gif of a number. The image is formatted in such a way so as to make it difficult for screen scrapers to write an algorithm which can decipher the numbers in the image (multiple fonts, different colors, background noise). If ever a spammer figured out how to programatically decipher the image then Mailblocks simply has to rework their image generation system and stay one step ahead of the spammers.
Next you have throw away addresses. Maiblocks calls these trackers. When you create a tracker a number and short ID are appended to the end of your username. This email address is then immune to the challenge response and can either be delivered to a purpose built folder or directly to your inbox. So if you wanted to have an address to get receipts from you simply make a tracker named say [username]+receipts4325@mailblocks.com. Then any email to this address can be delivered to the +receipts folder in your inbox. If you start getting spam at that address you just delete the address and create [username]+receipts5563@mailblocks.com and start giving this out. It can be a little bit of work to maintain your trackers but compared to deleting 20-30+ spam mails from my accounts each day it's well worth it.
When an email is successfully delivered to your main address the originating address is entered into your address book including the reason why this address was validated (completed puzzle, user added). Mailblocks also adds the address of any outgoing mail you write to your address book so that responses can be properly delivered without challenge. Finally, if you are expecting something to appear in your email that doesn't the 'pending' folder holds all email that hasn't been validated for a certain amount of time before deleting. If you really want to you can go back and dig through the email there to find the one you want, validate it, and it will be delivered to your inbox. If something gets validated you don't want simply go to your address book and either delete it or check 'do not deliver mail from this address'. Viola. Also of interest is the fact that Mailblocks can provide the same security to any other mail account you have. It can check POP3, IMAP, accept forwards, and even screen scrape web mail to bring all of your mail to a central location. When it does it provides the same callenge-response capability through these other accounts.
Re:You can do this yourself. (Score:5, Informative)
When I first started using TMDA, I had problems with people not understanding the mechanism. My grandmother, for example, complained about "bounces" (how she interpreted the challenges).
So, to avoid those problems, I:
The only problem with the scheme: there are some spammers who are dumb enough to not get the hint, and respond to the challenge. They don't seem to realize that their response probably constitutes harassment via 'net, which is a crime in the U.S. (Spammer go to jail. Do not pass go. Do not collect $200.)
Re:You can do this yourself. (Score:5, Informative)
Whitelisting is important, and easy too. Just export your address book to a text file and copy the results to your whitelist (which is also text).
It's worth noting that you can also auto-whitelist anyone you send mail to by using their nifty little mail proxy. It sits and proxies for SMTP and adds all outgoing mail automatically to your whitelist, so whoever you sent that resume to will never see a challenge...neat!
P.S. Can't recommend the product enough.
Re:Adaptive teergrubing anyone? (Score:3, Informative)
ROFLMAO.
"teergrube" - German word for "tarpit".
Teergrubing FAQ [iks-jena.de]
Teergrubing is a good idea, but it dates back from the days when open relays, not open proxies, were sending the emails. One spammer (with dialup) would hit you from one relay (with broadband) from the spammer's own (dialup) connection, and the goal was to slow down the open relay so that the open relay wouldn't be able to spew as many emails. Eventually, the admin of the open relay would wonder why his outbound queue was so huge, or why Sendmail fell over and died because /var/spool got full, and secure his server. In the old environment (spammer has narrowband, must hunt down broadband by finding open relays to steal from), one teergrube could "fix" one open relay at best, and at worst, would at least prevent delivery of several hundred thousand spams.
Doesn't really work as well in a world with millions of open broadband proxies. The spammer no longer cares if any individual open proxy hits a teergrube, because there's plenty more bandwidth where that came from. (And because open proxy luzers tend to be clueless twits, they're less likely to notice even if their machine crashes.) In today's environment (plenty of bandwidth on both the spammer's end, and plenty of proxies to steal bandwidth from), teergrubing in its original form is somewhat less effective.
Re:Too drastic? (Score:2, Informative)
I've recently implemented my own Bayesian system on my server. While my first-cut was very CPU intensive, very straight-forward techniques can be made to make it extremely CPU-friendly. In fact, I'll bet my current Bayesian system is less CPU-intensive than a simple keyword-filter that has 5000 "keywords" in its database.
I don't use SpamAssassin and can't comment on its toll on the CPU, but there is no inherent reason why a Bayesian system can't be deployed by ISPs. About the only drawback I see is that you have to store a corpus for each user and that ends up being between 1MB and 2MB per user. But disk space is cheap...
How Earthlink's system actually works. (Score:3, Informative)
Here's the internal description of the service, which, by the way, is always going to be optional -- users have to turn it on manually. So fears of mass confusion from users when Earthlink turns this system on are a bit unfounded.
This is what the automated reply looks like:
And finally a more detailed description they supply:
Procmail... (Score:2, Informative)
Re:One problem with this system. (Score:2, Informative)
:)
Re:Having written a similar system, I have questio (Score:2, Informative)
Nobody here gets it - C/R based on FROM is doomed (Score:1, Informative)
The only way to effectively defend against SPAM is at the IP level - via MX from DNS.
Hotmail, yahoo, free mail clients etc. are all doing a good job of policing themselves. If they can't police themselves, then punt the server. The spamboxen which increase the scale of spam that can be sent are the real problem.
The other important thing to do is to TAG the messages that aren't on the whitelist rather than deleting them, so the user can still find them.
Is this harder to use than current mail? I say NO because the amount of spam that people have to deal with is now so bad that the costs of dealing with managing the list is less than the cost of managing the spam.
But half the poseurs/posters here don't even understand how whitelisting or SMTP work before they go blathering off about 'throw out SMTP' or 'I won't get my f*cking mailing list'
This DOES work. (Score:2, Informative)
Challenge/response systems DO work, and they work extremely well. I think those who have not used one should give it a try before throwing rocks.
Re:You can do this yourself. (Score:2, Informative)
-mazor