Forgot your password?
typodupeerror
Security Spam The Internet IT

Researchers Build TCP-Based Spam Detection 81

Posted by samzenpus
from the beans-without-spam dept.
itwbennett writes "In a presentation at the Usenix LISA conference in Boston, researchers from the Naval Academy showed that signal analysis of factors such as timing, packet reordering, congestion and flow control can reveal the work of a spam-spewing botnet. The work 'advanced both the science of spam fighting and ... worked through all the engineering challenges of getting these techniques built into the most popular open-source spam filter,' said MIT computer science research affiliate Steve Bauer, who was not involved with the work. 'So this is both a clever bit of research and genuinely practical contribution to the persistent problem of fighting spam.'"
This discussion has been archived. No new comments can be posted.

Researchers Build TCP-Based Spam Detection

Comments Filter:
  • by damn_registrars (1103043) <damn.registrars@gmail.com> on Monday December 26, 2011 @02:11PM (#38495432) Homepage Journal
    People are looking at the wrong end of the problem with much of their efforts - and this is just another example of that. You cannot solve spam with filtering, detection, or legislative actions. We've seen time and time again that those are just time and money-sucking stopgap measures that ignore the reality of the situation.

    We won't see a real solution to the spam epidemic until people acknowledge the simple truth that spam is an economic problem. There is still a lot of money to be made by sending out spam, with very little expense for the spammer. The profit margin is high enough that it is well worth their while to find various ways around filters and any other silly mechanisms we throw at them.

    If you want to make an actual difference in the fight against spam, you need to approach the economic motivations behind it. If you stop of the flow of money to the spammers, you will stop the spam as well. Because no matter how much some people may want to believe otherwise, spam isn't sent just to piss you off and ruin your day. Spam is sent out because spammers are paid to do so. If they don't get paid, they won't send spam, it is as simple as that. Any other kind of countermeasure only prolongs the fight and throws more money in the wrong direction.
    • by Anonymous Coward

      great. so what do you propose? banning advertisements and referral programs? because I think most of us would be 100% behind that

    • Not everybody is rational, even less so with their marketing expenses. Really, most companies don't even know the return of their marketing expenses, thus they can't act rationaly.

      Also, filtering is great for reducing the results of spam, including spammer revenue. There is no reason not to do both, educate users and filter spam.

      • Also, filtering is great for reducing the results of spam, including spammer revenue

        Actually, it isn't, for at least two reasons:

        • The people who are willing to invest time and money in filtering aren't likely to click through and buy something based on spam any ways.
        • Total spam volume continues to increase in spite of filtering, which indicates it has not had any meaningful effect on the rewards for the spammer

        There is no reason not to do both, educate users and filter spam.

        Those are the two least useful tactics you can pursue. You would be better off praying to the flying spaghetti monster for a solution. My proposal is to actually get involved in t

        • by MrLizardo (264289)

          Also, filtering is great for reducing the results of spam, including spammer revenue

          Actually, it isn't, for at least two reasons:

          • The people who are willing to invest time and money in filtering aren't likely to click through and buy something based on spam any ways.

          The people who operate the filters != the end users of the mail system.
          End users pay for the cost of operating the filters by seeing advertisements in their webmail or paying for the email service. And yes, this has been working well to prevent the vast majority of spam (something like 99.9% according to my GMail account) from landing in inboxes for 15 years or so at this point.

    • by Tom (822) on Monday December 26, 2011 @02:26PM (#38495572) Homepage Journal

      The economic side has been tackled as well, and it turns out that it is not easier than the technological side. More importantly: It involves politics, and politics move slowly on all problems of the commons (i.e. low impact on many people).

      • The economic side has been tackled as well, and it turns out that it is not easier than the technological side.

        In a way, though, it is. There are actually fewer actions that need to be taken from the economic side than from the technological side; indeed economic actions can have very measurable and lasting effects in a short amount of time while technological actions are generally worthless.

        More importantly: It involves politics, and politics move slowly on all problems of the commons

        You may have misread me on that matter. Economic solutions are not inherently political, even though politics is inherently tied to economics. However, the companies who are on the financial take in the matter can be influe

        • by Tom (822)

          In a way, though, it is. There are actually fewer actions that need to be taken from the economic side than from the technological side; indeed economic actions can have very measurable and lasting effects in a short amount of time while technological actions are generally worthless.

          Do you say that as an economist or as a technician? Because I would take a bet that the other side would say the same thing, only in reverse.

          You may have misread me on that matter. Economic solutions are not inherently political, even though politics is inherently tied to economics. However, the companies who are on the financial take in the matter can be influenced without the necessity of legislative action.

          If it were that simple, someone would have done it by now, don't you think? If it is just that nobody has done it, then why don't you?

          • If it were that simple, someone would have done it by now, don't you think?

            It has been done, it's even been discussed on slashdot before [slashdot.org]. And it is far more effective than filters can ever hope to be.

            • by Tom (822)

              It has been done, it's even been discussed on slashdot before. And it is far more effective than filters can ever hope to be.

              Then why do I keep getting spam?

              Many anti-spam solutions were extremely effective the first time around - until the spammers adapted. I remember when greylisting cut your spam to almost nothing. It seems to have almost no effect these days.

      • by shentino (1139071)

        It's a problem that refuses to be solved since cutting off the flow of cash to spammers requires pissing off special interests that have the government in their pockets.

        • by Tom (822)

          Spammers don't have a lobby. There is no special interest working for them, it's simply that the problem is so distributed that few people really care about it all that much.

    • by wbr1 (2538558)
      For the same reason we have security theater.
      For the same reason we have a 'War on Drugs'.

      We seem to be blind to the fact (as a society or a government), that you cannot legislate or regulate a cure to a problem. People will always do what seems in their best interest, be it recreationally, economically, or otherwise.

      Very little our government does actually address the core issue, it just places band-aids on top of it. This, I think at least partly because a democracy is a system of compromise and on
    • by Halo1 (136547) <jonas.maebe@nOsPam.elis.ugent.be> on Monday December 26, 2011 @02:32PM (#38495634) Homepage

      The same can be said about pickpocketing, burglary and almost any other kind of crime. As long as technical measures can help with partially or temporarily alleviating the problems without causing disproportional side effects or requiring disproportionately large investments (i.e., not TSA nonsense vs terrorism, but more like door locks vs breaking and entering), I don't see what the problem is with developing and deploying them.

    • by t00le (136364)

      All we need is a global white list that allows trusted communication between peers. In the event spam is being sent from a member of the white list all of the email from that party would be flagged as suspect for 24 hours, then change to spam until the issue is rectified.

      The problem is the lack of response from certain parts of the world, where I block tcp/udp connections from already. I have no issues with allowing people to communicate freely, but I have no issues with my libido and no need to buy Xanax.

    • Re: (Score:2, Insightful)

      Actually, you're wrong. The problem is NOT economic. It'd be nice if it was -- because some obvious interdiction paths could be used. But it's not.

      The spam problem is behavioral: spammers are sociopaths. That's why there are no ex-spammers: they can no more stop spamming than a pedophile can stop molesting children. They're (pick your terminology) mentally ill, sick, etc.

      How do we know this? Because we can observe (and we have observed) that they continue spamming even when there's obviously no p
      • How do we know this? Because we can observe (and we have observed) that they continue spamming even when there's obviously no profit in it, nor any realistic hope of any profit in the future.

        That is simply not true. There is plenty of money to be made in spam, and it is the motivating force behind it. The spammers that make the news when they get caught (almost always on other offenses) are especially wealthy relative to their home countries. Furthermore, the total investment for a spammer is minimal; they really just need to be able to talk a good game and get some time on a botnet to be able to make money fast. As we've seen, each time a spammer is thrown in jail or murdered , the spam v

    • by shentino (1139071)

      First, a side note:

      Spam is profitable only if you ignore the costs absorbed by people whose computers get hijacked into botnets that send the stuff.

      In much the same way that grow ops are cheap when you jump the meter and rip off the electric company.

      In both cases the perpetrators get away with securing a windfall because they dump their cost burdens on unwilling participants.

      Now for the main point:

      How is most spammed product paid for?

    • by tlhIngan (30335)

      We won't see a real solution to the spam epidemic until people acknowledge the simple truth that spam is an economic problem. There is still a lot of money to be made by sending out spam, with very little expense for the spammer. The profit margin is high enough that it is well worth their while to find various ways around filters and any other silly mechanisms we throw at them.

      If you want to make an actual difference in the fight against spam, you need to approach the economic motivations behind it. If you

      • But to the spammer, it doesn't matter - they got paid ahead of time with no guarantee of results. And if the customer doesn't come back, no big deal - there's a lineup of other businesses needing "marketing services".

        You made an error yourself in that statement. The vast majority of spam is not for existing domains, but rather for new ones. You can verify this yourself by looking through old spam; if you look at a spam message from a month ago and look at the spamvertised domain you will find it is not the same spamvertised domain that was listed in today's spam, even though they are selling the same products and using all the same web graphics, code, and template.

        Furthermore if you run a WHOIS on domain that was

  • The best way to fight spam is do what IM systems has been doing, by whitelisting. So, 1st email triggers a white list query, and the rest wil be invisible... May be do this on a per ip or per domain basis...
    • by mcavic (2007672)
      Accepting email only from addresses or domains on your white list is a decent idea, as long as you check your spam folder every day for legitimate mail from new people. Otherwise, it's cumbersome to have to add someone to your list before they email you. You would need the ability to whitelist a whole domain, though, such as amazon.com, etc... something that address books usually won't let you do.
  • by wkcole (644783) on Monday December 26, 2011 @02:32PM (#38495632)

    I'm sure 'itwbennett' would rather everyone go to his employer's website to read that article, but it is clearly not written (or edited) by anyone who has any basic clues about spam-fighting. Just reading the subtitle makes me cringe for the unfortunate "journalists" lassoed into writing it, as it was clearly done by spam neophytes in a desperate scramble for click-scrounging content. The article is vaguely about a paper presented almost a year ago at LISA '11. There are links to an abstract and the original paper at the LISA '11 site: http://www.usenix.org/events/lisa11/tech/ [usenix.org]

    The general space of sniffing out spam by looking at TCP characteristics has been mined for years usefully with Symantec and MailChannels both offering proprietary tools that use such techniques and some open DNSBL's using TCP sniffing to identify sources, but it would be incorrect to believe that any one methodology will ever be a magical silver bullet against spam.

  • This REALLY sounds like a copy of Sendmail Inc.'s Rate Control component, which has been deployed to many sites for the last several years. Rate Control allows the admin to throttle or otherwise block email that breaks various TCP-related thresholds (messages/second, bad recipients/second, connections/second, etc.). Further, recent real world indications show that spammers are sending fewer spams per second from individual IP addresses--they make up the volume by increasing the size of the botnet, and coo
    • Postfix has had throttling for several years now, based on the same basic concepts. I use Postfix with greylisting and to be honest, my Spamassassin and ClamAV filters rarely get hit. Since at least big spam attacks are by bots, and bots are primarily designed to just shove as much through as possible, greylisting alone does a spectacular job of killing them, though sometimes people get pissed when messages take a while to get to them from a recipient the first time.

  • by WaffleMonster (969671) on Monday December 26, 2011 @02:53PM (#38495816)

    I've always wondered how seemingly smart people can act so stupidly totally oblivious to the repercussions of their actions.

    What happens when a busy computer that would cause it to naturally act in a similiar matter as a botnet zombie sends an email and that message is then flagged as spam?

    Spammers are no fools or dinosaurs. They will simply adjust their spamming rate in zombie client below the threshold needed to induce effects needed to trigger the detection scheme.

    End result as always is the same:

    It won't stop anyone from spamming

    It WILL make SMTP based Email even more unreliable than it currently is.

  • While 95% accuracy at detecting spam may sound like "wow", it's a very low rate. Simply using correctly configured greylisting gives an accuracy in the 99% range. So I doubt this technique really improves anything but it will allow to say 'we did it another way'. Given than more and more spam comes from official mail relays, accuracy will only increase when analysing the body of the mail.

  • First, we've known for many years that IP-level techniques can deal with a lot of spam. For example, using the Spamhaus "DROP" list in perimeter devices is so incredibly effective that anyone who isn't doing it may summarily be declared incompetent. As another example, perhaps more germane to this paper, see http://use.perl.org/~merlyn/journal/17094 [perl.org] -- which demonstrates how to use passive OS fingerprinting in the BSD pf firewall to throttle traffic from Windows systems. (I presume everyone is well awar
  • of all spam comes from dynamic addresses. Their method (95%) is worse than simply rejecting all email from dynamic IP's. I find greylisting dynamics for 36 hours and statics for an hour filters over 99% of spam. If one gets thru, I just blacklist the IP.

  • I've been doing this for years.

    I use p0f to detect connections coming from windows and greylist them. Very little genuine mail comes from windows based mail servers.

    I find there is little point greylisting mail from unix machines as very little spam comes from them.

COBOL is for morons. -- E.W. Dijkstra

Working...