Researchers Build TCP-Based Spam Detection 81
itwbennett writes "In a presentation at the Usenix LISA conference in Boston, researchers from the Naval Academy showed that signal analysis of factors such as timing, packet reordering, congestion and flow control can reveal the work of a spam-spewing botnet. The work 'advanced both the science of spam fighting and ... worked through all the engineering challenges of getting these techniques built into the most popular open-source spam filter,' said MIT computer science research affiliate Steve Bauer, who was not involved with the work. 'So this is both a clever bit of research and genuinely practical contribution to the persistent problem of fighting spam.'"
Why do we keep doing this? (Score:5, Insightful)
We won't see a real solution to the spam epidemic until people acknowledge the simple truth that spam is an economic problem. There is still a lot of money to be made by sending out spam, with very little expense for the spammer. The profit margin is high enough that it is well worth their while to find various ways around filters and any other silly mechanisms we throw at them.
If you want to make an actual difference in the fight against spam, you need to approach the economic motivations behind it. If you stop of the flow of money to the spammers, you will stop the spam as well. Because no matter how much some people may want to believe otherwise, spam isn't sent just to piss you off and ruin your day. Spam is sent out because spammers are paid to do so. If they don't get paid, they won't send spam, it is as simple as that. Any other kind of countermeasure only prolongs the fight and throws more money in the wrong direction.
Won't work (Score:3)
Even if the spam click-though rate is 0.0%, there are still enough suckers born every minute to buy the service of spammers.
Re: (Score:2)
Here's an idea - recipient's SMTP server refuses e-mails unless they get 0.01 cents with it.
Re: (Score:2)
Re: (Score:3)
Re: (Score:1)
Anonymity is not a feature inherent in e-mail.
Yes it is. Anybody who can telnet to port 25 can send anonymous e-mails.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
There's also nothing to stop the spammers from forging the credentials of some other organization. Then we'd be hearing about Anonymous sending billions of spam messages, pretending to be BoA...
Re: (Score:2)
Re: (Score:2)
At least with postal spam they have to print and mail it at their own expense.
Electronic spam is even worse since often the sending is slaved to botnets full of hijacked computers and the costs are diverted to unwilling participants.
Re: (Score:3)
How about instead of elecrtronic mail, we devise a system where people write letters on physical paper and then we deliver those letters to the recipients. We could charge a nominal fee for the delivery, and that should end all "junk mail", right?
Re: (Score:3)
Here's an idea - recipient's SMTP server refuses e-mails unless they get 0.01 cents with it.
Don't bother trying to patent that idea. It has been proposed and even tried many times.
One problem with it is simply that there is no reliable mechanism in place to identify the responsible sender of every piece of email. Internet email is not a single system, but rather a loosely confederated mob of independently operated systems that mostly use a common set of protocols. Most email these days is spam, sent mostly by hijacked machines, of which most is rejected easily by most receiving systems. The bul
Re: (Score:1)
they get 0.01 cents
Did you really mean only 1/100 of a cent?
Re: (Score:2)
Yes, I did, I'm not using Verizon math.
The idea is to keep things cheap for legitimate e-mail senders (e-mail providers could even soak up that cost), but it becomes a noticeable cost once you're sending tens of thousands of e-mails.
Re: (Score:1)
great. so what do you propose? banning advertisements and referral programs? because I think most of us would be 100% behind that
Re: (Score:2)
Not everybody is rational, even less so with their marketing expenses. Really, most companies don't even know the return of their marketing expenses, thus they can't act rationaly.
Also, filtering is great for reducing the results of spam, including spammer revenue. There is no reason not to do both, educate users and filter spam.
Re: (Score:2)
Also, filtering is great for reducing the results of spam, including spammer revenue
Actually, it isn't, for at least two reasons:
There is no reason not to do both, educate users and filter spam.
Those are the two least useful tactics you can pursue. You would be better off praying to the flying spaghetti monster for a solution. My proposal is to actually get involved in t
Re: (Score:1)
Also, filtering is great for reducing the results of spam, including spammer revenue
Actually, it isn't, for at least two reasons:
The people who operate the filters != the end users of the mail system.
End users pay for the cost of operating the filters by seeing advertisements in their webmail or paying for the email service. And yes, this has been working well to prevent the vast majority of spam (something like 99.9% according to my GMail account) from landing in inboxes for 15 years or so at this point.
Re:Why do we keep doing this? (Score:5, Insightful)
The economic side has been tackled as well, and it turns out that it is not easier than the technological side. More importantly: It involves politics, and politics move slowly on all problems of the commons (i.e. low impact on many people).
Re: (Score:2)
The economic side has been tackled as well, and it turns out that it is not easier than the technological side.
In a way, though, it is. There are actually fewer actions that need to be taken from the economic side than from the technological side; indeed economic actions can have very measurable and lasting effects in a short amount of time while technological actions are generally worthless.
More importantly: It involves politics, and politics move slowly on all problems of the commons
You may have misread me on that matter. Economic solutions are not inherently political, even though politics is inherently tied to economics. However, the companies who are on the financial take in the matter can be influe
Re: (Score:3)
In a way, though, it is. There are actually fewer actions that need to be taken from the economic side than from the technological side; indeed economic actions can have very measurable and lasting effects in a short amount of time while technological actions are generally worthless.
Do you say that as an economist or as a technician? Because I would take a bet that the other side would say the same thing, only in reverse.
You may have misread me on that matter. Economic solutions are not inherently political, even though politics is inherently tied to economics. However, the companies who are on the financial take in the matter can be influenced without the necessity of legislative action.
If it were that simple, someone would have done it by now, don't you think? If it is just that nobody has done it, then why don't you?
Re: (Score:2)
If it were that simple, someone would have done it by now, don't you think?
It has been done, it's even been discussed on slashdot before [slashdot.org]. And it is far more effective than filters can ever hope to be.
Re: (Score:2)
It has been done, it's even been discussed on slashdot before. And it is far more effective than filters can ever hope to be.
Then why do I keep getting spam?
Many anti-spam solutions were extremely effective the first time around - until the spammers adapted. I remember when greylisting cut your spam to almost nothing. It seems to have almost no effect these days.
Re: (Score:2)
It's a problem that refuses to be solved since cutting off the flow of cash to spammers requires pissing off special interests that have the government in their pockets.
Re: (Score:2)
Spammers don't have a lobby. There is no special interest working for them, it's simply that the problem is so distributed that few people really care about it all that much.
Re: (Score:2)
No, but the credit card companies people use to pay for V14GR4 do...
Re: (Score:2)
For the same reason we have a 'War on Drugs'.
We seem to be blind to the fact (as a society or a government), that you cannot legislate or regulate a cure to a problem. People will always do what seems in their best interest, be it recreationally, economically, or otherwise.
Very little our government does actually address the core issue, it just places band-aids on top of it. This, I think at least partly because a democracy is a system of compromise and on
Re:Why do we keep doing this? (Score:5, Insightful)
The same can be said about pickpocketing, burglary and almost any other kind of crime. As long as technical measures can help with partially or temporarily alleviating the problems without causing disproportional side effects or requiring disproportionately large investments (i.e., not TSA nonsense vs terrorism, but more like door locks vs breaking and entering), I don't see what the problem is with developing and deploying them.
Re: (Score:2)
All we need is a global white list that allows trusted communication between peers. In the event spam is being sent from a member of the white list all of the email from that party would be flagged as suspect for 24 hours, then change to spam until the issue is rectified.
The problem is the lack of response from certain parts of the world, where I block tcp/udp connections from already. I have no issues with allowing people to communicate freely, but I have no issues with my libido and no need to buy Xanax.
Re: (Score:2)
And who would run the white list? The government? Which government?
Re: (Score:2)
Re: (Score:2)
When you have something that is 0.000001% effective and you can still make millions, there is no free market way of stopping it.
You're wrong on that. There is a free market way to do it. You can stop spam on the market by getting the businesses who currently do business with spammers to stop. Some of them aren't even aware they are working with spammers because they are working in large volumes and the spammers are small fry. Some of them are two or more degrees away from the spammer and might never make direct contact with them. Nonetheless, if you can interrupt the money flow, you can stop spam.
Re: (Score:2)
The biggest problem with spam is that spammers are cheating on expenses and getting away with it.
I guarantee you spam would drop in a hurry if there was some way to make a spammer eat his own IT bills. At present the ones who REALLY pay for spam are gullible boobs who let their computers get hijacked.
Re: (Score:2)
At present the ones who REALLY pay for spam are gullible boobs who let their computers get hijacked.
Correction - the boobs pay for about half the costs of spam. You are correct that spammers themselves pay a negligible portion.
However, the rest is paid by every person who accesses the internet, in any way, shape or form. Spam is consuming bandwidth, which costs users money even if their own machines are not propagating it. Spam is also consuming storage space on email servers, even if users never read it. Spam is consuming CPU time when filters are running, and spam is consuming human time to adj
Re: (Score:2, Insightful)
The spam problem is behavioral: spammers are sociopaths. That's why there are no ex-spammers: they can no more stop spamming than a pedophile can stop molesting children. They're (pick your terminology) mentally ill, sick, etc.
How do we know this? Because we can observe (and we have observed) that they continue spamming even when there's obviously no p
Re: (Score:3)
How do we know this? Because we can observe (and we have observed) that they continue spamming even when there's obviously no profit in it, nor any realistic hope of any profit in the future.
That is simply not true. There is plenty of money to be made in spam, and it is the motivating force behind it. The spammers that make the news when they get caught (almost always on other offenses) are especially wealthy relative to their home countries. Furthermore, the total investment for a spammer is minimal; they really just need to be able to talk a good game and get some time on a botnet to be able to make money fast. As we've seen, each time a spammer is thrown in jail or murdered , the spam v
Re: (Score:2)
First, a side note:
Spam is profitable only if you ignore the costs absorbed by people whose computers get hijacked into botnets that send the stuff.
In much the same way that grow ops are cheap when you jump the meter and rip off the electric company.
In both cases the perpetrators get away with securing a windfall because they dump their cost burdens on unwilling participants.
Now for the main point:
How is most spammed product paid for?
Re: (Score:2)
Re: (Score:2)
But to the spammer, it doesn't matter - they got paid ahead of time with no guarantee of results. And if the customer doesn't come back, no big deal - there's a lineup of other businesses needing "marketing services".
You made an error yourself in that statement. The vast majority of spam is not for existing domains, but rather for new ones. You can verify this yourself by looking through old spam; if you look at a spam message from a month ago and look at the spamvertised domain you will find it is not the same spamvertised domain that was listed in today's spam, even though they are selling the same products and using all the same web graphics, code, and template.
Furthermore if you run a WHOIS on domain that was
Fix email to work like IM... (Score:2)
Re: (Score:2)
Skip the ITWorld article (Score:5, Informative)
I'm sure 'itwbennett' would rather everyone go to his employer's website to read that article, but it is clearly not written (or edited) by anyone who has any basic clues about spam-fighting. Just reading the subtitle makes me cringe for the unfortunate "journalists" lassoed into writing it, as it was clearly done by spam neophytes in a desperate scramble for click-scrounging content. The article is vaguely about a paper presented almost a year ago at LISA '11. There are links to an abstract and the original paper at the LISA '11 site: http://www.usenix.org/events/lisa11/tech/ [usenix.org]
The general space of sniffing out spam by looking at TCP characteristics has been mined for years usefully with Symantec and MailChannels both offering proprietary tools that use such techniques and some open DNSBL's using TCP sniffing to identify sources, but it would be incorrect to believe that any one methodology will ever be a magical silver bullet against spam.
Looks like a copy of someone else's work... (Score:1)
Re: (Score:3)
Postfix has had throttling for several years now, based on the same basic concepts. I use Postfix with greylisting and to be honest, my Spamassassin and ClamAV filters rarely get hit. Since at least big spam attacks are by bots, and bots are primarily designed to just shove as much through as possible, greylisting alone does a spectacular job of killing them, though sometimes people get pissed when messages take a while to get to them from a recipient the first time.
Re: (Score:1)
Re: (Score:2)
So the first defense of greylisting has been defeated (I'm not seeing this in my logs though). But that still leaves the second advantage gained by it: by the time they get back to your smtpd they hopefully will be blacklisted.
Re: (Score:1)
maybe you need to increase the greylist-period. Most bots run at dsl accounts, which means they will get a new ip approximate every 24h. When you require a period of 24h for unknown senders, they will not be able to resend it.
Of course only a possible solution, if you do not need to get your e-mails as soon as possible. But when you need to, you do not want to use greylisting at all.
Please stop (Score:3)
I've always wondered how seemingly smart people can act so stupidly totally oblivious to the repercussions of their actions.
What happens when a busy computer that would cause it to naturally act in a similiar matter as a botnet zombie sends an email and that message is then flagged as spam?
Spammers are no fools or dinosaurs. They will simply adjust their spamming rate in zombie client below the threshold needed to induce effects needed to trigger the detection scheme.
End result as always is the same:
It won't stop anyone from spamming
It WILL make SMTP based Email even more unreliable than it currently is.
Re:Please stop (Score:4, Interesting)
This rather assumes that every MTA will have the same threshold. It is not necessary (or helpful) to have a security monoculture.
A very simple first defence against such rate tuning is to randomly vary thresholds substantially between systems and from time to time.
Rgds
Damon
95% accuracy (Score:2)
While 95% accuracy at detecting spam may sound like "wow", it's a very low rate. Simply using correctly configured greylisting gives an accuracy in the 99% range. So I doubt this technique really improves anything but it will allow to say 'we did it another way'. Given than more and more spam comes from official mail relays, accuracy will only increase when analysing the body of the mail.
This isn't a new technique...and it's inaccurate (Score:2)
In other news 99% (Score:2)
of all spam comes from dynamic addresses. Their method (95%) is worse than simply rejecting all email from dynamic IP's. I find greylisting dynamics for 36 hours and statics for an hour filters over 99% of spam. If one gets thru, I just blacklist the IP.
Been doing this for years (Score:2)
I've been doing this for years.
I use p0f to detect connections coming from windows and greylist them. Very little genuine mail comes from windows based mail servers.
I find there is little point greylisting mail from unix machines as very little spam comes from them.