Stopping Spam Before It Hits the Mail Server 157
Al writes "A team of researchers at the Georgia Institute for Technology say they have developed a way to catch spam before it even arrives on the mail server. Instead of bothering to analyze the contents of a spam message, their software, called SNARE (Spatio-temporal Network-level Automatic Reputation Engine), examines key aspects of individual packets of data to determine whether it might be spam. The team, led by assistant professor Nick Feamster, analyzed 2.5 million emails collected by McAfee in order to determine the key packet characteristics of spam. These include the geodesic proximity of end mail servers and the number of ports open on the sending machine. The approach catches spam 70 percent of the time, with a 0.3 false positive rate. Of course, revealing these characteristics could also allow spammers to fake their packets to avoid filtering."
Spammers evolve from experience (Score:3, Insightful)
Just like other criminals, spammers must quickly respond to what actually works. In essence this is the flaw in any "security by obscurity" scheme, the bad guys simply respond to whatever works. If you get to try several billion times a day then you can try a whole lot of combinations.
.3% false positive is pretty high (Score:5, Insightful)
That means that in my office of 50 people, with an average of 50 emails per day (a very very low estimate), we'd get 7-8 false positives daily. I'd hear bloody murder if that was the case.
We get a lot more mail than that per day, and our spamassassin without autolearning (simply flag anything higher than 5.0) does a hell of a lot better job than that... down in the range of 1-2 false positives a month. Assuming a low daily average of emails (like my example), that's .002% false positives.
Re:I don't get it... (Score:3, Insightful)
False positive rate? (Score:5, Insightful)
0.3 would be terrible - three out of ten false positives. 0.3 percent - what the article actually says - is not too bad. But current techniques allow me to check the spam bin for such messages. This technique would pretty much preclude that capability, since the mail would never arrive at the server. I'm not sure that a rate of 0.003 would be acceptable under those circumstances.
Re:It'll work..except when it doesn't. (Score:4, Insightful)
I realize that you're kidding, but removing more than expected is something that I consider unacceptable. If it hits the mail server and gets shuffled off into a spam folder with 100 pieces of trash, that's fine. But if it's not even going to make it to the mail server, 0.3% is too high a false positive rate.
Wrong approach (Score:5, Insightful)
The fundamental property of spam is that it involves many similar messages going to a large number of destinations. That's what to look for. Google can do that, because they manage a very large number of mailboxes with a single system. SpamCop used to do that, but they had to be in the mail-forwarding business to do it and that was too expensive.
Trying to detect spam by looking only at the mail for a single account is inherently a form of guessing. The existing technologies are reasonably good, but not good enough that the spammers give up.
Re:"IP addresses, he notes, are easy to fake." (Score:3, Insightful)
Re:False positive rate? (Score:5, Insightful)
And when my mail filters blocks spam, it sends out a message with redirections to an alternative gsm-number telling them to call me so I can whitelist the adres.
That's called back scatter and its as bad as spam.
Think about it, my mail servers block about 35,000 spam per day. If they sent a message to each failed recipient with alternative instructions, that would be 35,000 messages I sent out. Some 34,990 of those messages would either be undeliverable or would get delivered to people who had nothing to do with the original message. You are effectively clogging up a bunch of innocent peoples mail systems with your messages.
Put it another way, suppose some spammer sends 1,000,000 messages with your email address spoofed as the sender. If everyone else did what you do, you would then receive 1,000,000 messages back to your inbox giving you alternate instructions to contact these people.
You wouldn't want that. Nobody else does either. So please stop.
MS -- Virus -- Bots -- Spam (Score:3, Insightful)
Why does it seem everyone ignores the real source of the majority of spam: Microsoft windows computers infected by viruses running botnets that send spam. Yes, is generated by other systems, but not nearly the amount that is being generated by MS based botnets.
How about everyone just send their frigen spam bill to MS. How about a class action for everyone to collect for the damage that MS does to networks around the World. Better yet lets just forward all the spam we get to MS. Let them sort it out.
Re:Not practical. (Score:2, Insightful)
Re:Not practical. (Score:3, Insightful)
And follow up with a phone call to make sure they got the letter.
Re:False positive rate? (Score:3, Insightful)
I do get your point really. But my dad (read: the boss) would not be happy if he missed a deal cause a million people who got spoofed got 1 mail from us telling them to call us if their message wasn't spam.
Read that over a few times. You are saying its ok to send out a MILLION unsolicited and annoying email messages (aka SPAM) to people who have never heard of you, so that your father won't miss a single deal?
How is that any different from rationalizing sending out a million direct marketing spam in the hopes of securing a single deal from it?
Bottom line, if your dad absolutely can't afford to "miss a deal", two things:
1) your backscatter system isn't going to necessarily work. Just because it worked 4 times is meaningless, you have no idea how many legitimate emails you lost. There is a high percentage change that your backscatter will be (correctly) identified as spam by other mail servers and discarded, so your notifications won't get delivered. And there is a high percentage that even if someone received your backscatter, they just deleted it. (I receive literally dozens of 'your mail could not be delivered' messages daily - some of them are backscatter, many of them are virus/malware pretending to be backscatter.) Which leads me to my next item:
2) if 'not missing a deal' is that important, then scan your own spam box for false positives. That's the sane way to handle this.
We send on every 1000 mails one message, telling them they got in the spambox and that they should call if it's not spam.
Please clarify this. Are you saying for every 1000 spam, you only send 1 notification? If so how do you choose which 999 spam you ignore vs the 1 you send a notification? Or are you saying only 1 in a thousand messages you receive is spam??
We are not the problem. The spammers are, so please, don't turn it around... I am not the problem
This is like seeing a drive by shooting in progress, whipping out your semi-auto and pumping as many rounds as you can in the general direction of the car. Your bullets are just as likely to hurt innocent bystanders as the criminals. Similiarly your email back scatter is just as harmful as the spam itself. You aren't the only problem, but you aren't part of the solution.
Now, go bug the dudes who don't want to make their precious smpt more secure.
Don't be naive. Secure email is trivial. Convincing everyone from Australia to Zimbabwe to switch to it is hard. And until YOU are willing to miss out on messages from people who haven't switched to your secure solution of choice, the problems will persist. And I don't see that happening anytime soon... you said it yourself... dad doesn't want to miss a single deal... no way in hell he's going to require that everyone who wants to send him messages conform to some new security regime... he'd miss messages left and right.