Forgot your password?
typodupeerror
This discussion has been archived. No new comments can be posted.

Paul Graham Describes Dangers of Spam Blacklists

Comments Filter:
  • by Neil Blender (555885) <neilblender@gmail.com> on Thursday June 16, 2005 @03:22PM (#12834783)
    $idea will not help cut down on spam. In fact, it is detrimental. This has been know for $num_years years, but I feel I must prove that I am really smart by writing an article about it.
  • by nev4 (721804) on Thursday June 16, 2005 @03:22PM (#12834785)
    We've been blacklisted before and the sysadmins who run these things often WILL NOT remove you, no matter what. I'd take all the SPAM anyday vs. not being able to send legitimate emails.
    • Does this blacklist have a name?
    • by Vainglorious Coward (267452) on Thursday June 16, 2005 @03:37PM (#12834919) Journal

      I'd take all the SPAM anyday vs. not being able to send legitimate emails.

      Except that blocklists don't stop you sending email, they merely allow others to decide whether to accept that mail. Or do you think other people should be forced to accept any and every email you send?

      • Wrong (Score:4, Insightful)

        by autopr0n (534291) on Thursday June 16, 2005 @03:49PM (#12835042) Homepage Journal
        What they do is allow others to block email between two diffrent people, simply because they run the mail servers that sit between them. If it was only individual users who were using these blocklists, it would be a diffrent issue. But it's not.
        • So what (Score:5, Insightful)

          by Vainglorious Coward (267452) on Thursday June 16, 2005 @04:06PM (#12835208) Journal
          I reserve the right to block (or accept) any mail I choose on my own system. I also make that decision on behalf of my users, weighing the pros and cons, and especially the listing policies, of any RBLs. If I get it wrong, then yes, my users won't be happy. I'm all for doing what makes my users happy. Blocklists do make my users happy. They work. The fact that there's sqealing about the effect shows that they work. I reject utterly the contention that I should somehow be forced to accept anything I don't want to receive
          • Re:So what (Score:4, Insightful)

            by Chris Burke (6130) on Thursday June 16, 2005 @07:11PM (#12836976) Homepage
            I reject utterly the contention that I should somehow be forced to accept anything I don't want to receive

            And that means that you will readily accept someone else's decision on what you should and should not receive? You sound to individualistic for that, so I think you are probably missing the implications of these blacklists.

            What if you want to receive email from someone, but their block is in the blacklist your ISP uses? Can you call up your ISP and ask them to remove it? Can you get your friend to change their ISP so they are in a non-blacklisted block? In the past, I've seen people whose ISPs would block, for example, the entire University of Michigan. That made it pretty tough to communicate with them.

            You are absolutely under no obligation to accept anything. That's why I run a spam filter myself. But letting someone else's often arbitrary judgement control what you do and don't receive is contrary to the personal control that you (and I) want.

            Speaking of which, I'm glad I'm not one of your users.
          • Re:So what (Score:3, Insightful)

            by Chris Burke (6130)
            The fact that there's sqealing about the effect shows that they work.

            Um, no.

            The fact that there's squealing about the effect from non-spammers shows that they don't work.
    • by Seumas (6865) * on Thursday June 16, 2005 @03:38PM (#12834927)
      John Reid of the SBL told me this wasn't true-- that the SBL was still clean, and that they only blacklisted hosting companies' mail servers when they were spam hosts who took on innocent users as camouflage:

      He is right. That definitely is NOT how SBL actually operates. I have a site that is heavily trafficked (millions per month) and they blocked my email (from my own personal server) that has delivered mail for my site for seven years with absolutely no outgoing spam or relaying having ever occurred in its entire life.

      However, a spammer with false credentials faked his way into a hosting account with my colo provider and as a result, SBL blocked multiple entire submnets, rendering my entire site and service useless for almost an entire month (we deal with auctions, meaning nobody was getting closed notices, won notices, outbid notices, addresses to send payment, registration emails, lost password emails - and when they complained, I couldn't respond to help them and explain it to them).

      SBL couldn't have cared less. As far as they are concerned, if one IP is a source of spam, they all are. And they'll get to fixing it in their own damn sweet time.

      But the defense of SBL fan-boys is typically "well it's VOLUNTARY!".

      Yeah. Whatever. Fuck off.
  • A few comments (Score:5, Informative)

    by alanw (1822) * <alan@wylie.me.uk> on Thursday June 16, 2005 @03:23PM (#12834789) Homepage
    From Paul Graham's original article http://paulgraham.com/spamhausblacklist.html [paulgraham.com]
    any filter relying on the SBL is now marking email with the url "paulgraham.com" as spam
    The primary use of the SBL is to allow sysadmins to refuse e-mail coming from listed IP addresses. The mail should be rejected during the SMTP header conversation, and the senders of genuine (non-spam and non-virus) e-mails will receive a non-delivery report from their outgoing MTA.

    I assume that what Paul Graham is complaining about must be SpamAssassin, or some other content filter, applying a score to articles containing URLs, which when looked up in DNS resolve to listed IP addresses. This is much less acceptable, since the sender has no way to know that their e-mail may have been classified as spam.

    The details of the listing can be found at http://www.spamhaus.org/sbl/sbl.lasso?query=SBL279 45 [spamhaus.org]. This is a /32 - i.e. a single IP address. I don't know why Paul Graham's web site (which has that IP address) has been associated with textileshop.com, which has a completely different IP address.

    The other Yahoo listing on the SBL is also a /32.

    I also note in another of Paul Graham's articles http://paulgraham.com/sblbad.html [paulgraham.com] he claims

    The most notorious example is the MAPS RBL
    As any fule kno, the most notorious spam blacklist is SPEWS. ~
    • As any fule kno, the most notorious spam blacklist is SPEWS.

      ORBS, and its later reincarnation, ORBZ, also weren't exactly the nicest players on the field. I remember one incident where I couldn't send email to someone from a GMX [gmx.net] account, because GMX - a webmail provider not unlike Hotmail etc., with several million users - had ended up on their blacklists (I'm not sure anymore whether it was ORBS or ORBZ at the point that happened, but it matters little, anyway).

      This article [isp-planet.com]on the death of ORBZ has

    • # dig paulgraham.com MX

      ; <<>> DiG 9.2.4 <<>> paulgraham.com MX
      ;; global options: printcmd
      ;; Got answer:
      ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53349
      ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2

      ;; QUESTION SECTION:
      ;paulgraham.com. IN MX

      ;; ANSWER SECTION:
      paulgraham.com. 3600 IN MX 10 milter1.store.vip.sc5.yahoo.com.

      ;; AUTHORITY SECTION:
      paulgraham.com. 3600 IN NS st-ns1.yahoo.com.
      paulgraham.com. 360

    • Re:A few comments (Score:4, Informative)

      by mercuryresearch (680293) on Thursday June 16, 2005 @03:47PM (#12835030) Journal
      Seeing as how this exact situation happened to me this week, I can provide some light on the /32 IP address issue.

      In my case, I moved a server to a new colo facility. Most facilities have an IP block, and you get assigned an IP from it. Six months or a year ago that IP might have belonged to someone else. For me, it turned out in February a spammer installed a server at the colo, spammed from that server for a single day before the colo ISP turned them off. That IP got listed in Spamhaus; in the beginning of June I was assigned that IP.

      So, I ended up with a Spamhaus listing for my mail server's IP address -- and _I_ can't get it removed. Spamhaus expects the colo operator to contact them (which they did on my request) but even there, if the blacklist operator doesn't like the ISP/colo people, they can ignore the request.

      Fortunately Spamhaus listened and I got the record for my IP removed. But this showed me it was trivial for a non-spammer to inherit a blacklisted IP. I've added doing DNSBL checks on colo-assigned IP addresses for future moves to prevent any future issues.

    • Re:A few comments (Score:3, Informative)

      by sloanster (213766)
      I assume that what Paul Graham is complaining about must be SpamAssassin, or some other content filter, applying a score to articles containing URLs, which when looked up in DNS resolve to listed IP addresses. This is much less acceptable, since the sender has no way to know that their e-mail may have been classified as spam.

      Um, no. That's not how spamassassin works - spamassassin uses a wide spectrum approach - it can take into account whatever blacklists you want to consult, but an RBL hit in spamassass
  • by SSpade (549608) on Thursday June 16, 2005 @03:23PM (#12834793) Homepage

    ...his website is hosted on the same IP address as a spammer (textileshop.com) was on yesterday, and because of that he's seeing some of his mail blocked.

    There's certainly a need for thoughtful and hopefully positive criticism of blacklist behaviour. This article is not it.

    • by Skye16 (685048) on Thursday June 16, 2005 @03:28PM (#12834840)
      So...it's okay if he goes to Federal Pound-Him-In-The-Ass penitentiary just because he rented a car from a place that also rented a car to a crack dealer?

      Huh?

      Sorry, but that's still bullshit. He states it clearly in his article: You can't screw over innocents just to make the guilty pay. Does the your government put a neighbor family through torture just because you got a parking ticket? No. It's YOUR fault and YOU should be punished. Not some innocent bystander.
      • In the age of the internet...

        It's not like it's difficult to register a domain. With cars... it's a little more expensive and there are several registriations that take place.

        So two discern two cars in a particular rental agency is not the same as two domains on the same ip/subnet.

        Your comparison is fundamentally flawed.
        • Not in the slightest. You're basically saying "It's too hard otherwise". I'm basically saying "That's too goddam bad". You can't fuck over those who are innocent just to punish those who are guilty. If that means you can't win, then fine, you can't win. Deal with it.

          Or, of course, you can keep doing it, but you're still a prick. (General you, not specific you - I don't know you, so I wouldn't dare make that claim right off the bat :] )
    • Is he making an accusation that Spamhaus isn't taking the IP off of the SBL? If so, maybe it's because they won't accept his word in the matter, only the word of the people who actually admin the box. Too bad - *I* wouldn't accept the word of a hosted person that the spammer is gone, only the word of the *hoster*, who, if he ends up lying, should rightfully end up with a more permanent ban. Yeah, this sucks for the hosted people, but hey - move your site. Your hoster sucks and doesn't deserve your busin
      • by SSpade (549608) on Thursday June 16, 2005 @03:40PM (#12834950) Homepage

        Actually the IP address that's listed is store.yahoo.com.

        Yahoo hosting is riddled with spammers, and store.yahoo.com is where most of them live, and where they accept credit cards for their purchases.

        The SBL lists IP addresses that are involved in spam. 66.163.161.45 is involved in a lot of spam. It's not been removed from the SBL because, well, it's still actively being used by spammers.

        Because countless spammers register domains on a daily basis, yet point them at the same IP addresses some people choose to resolve the URLs in incoming email and bounce the mail if any of them resolve to particularly filthy IP addresses.

        66.163.161.45 is filthy. Blocking mail that has URLs pointing there will stop a fair amount of spam. Not an approach I'd use myself, but certainly a lot more effective (in terms of spam caugh and false positives) than many, many other approaches in widespread use.

        Paul chose to host his website there, despite supposedly knowing a lot about the spam issue. That was probably not a good call.

        • by deacon (40533) on Thursday June 16, 2005 @04:01PM (#12835151) Journal
          66.163.161.45 is filthy. Blocking mail that has URLs pointing there will stop a fair amount of spam. Not an approach I'd use myself, but certainly a lot more effective (in terms of spam caugh and false positives) than many, many other approaches in widespread use. Paul chose to host his website there, despite supposedly knowing a lot about the spam issue. That was probably not a good call.

          Let me reword your justification of of this behaviour so others can see the flaw in it more clearly:

          [66.163.161.45 is a filthy neighborhood. Lots of criminals live there. So, a group of vigilantes randomly started machine gunning people walking the street. Not something I'd do myself, I prefer to use a shotgun, but certainly more effective then using the court system. Paul chose to live there, and he should have known it's a bad area. If he gets shot at random, well, too fucking bad, he should have known better. Living there was probably not a good call.]

          Some days it's hard choosing between deleting 400 spams a day and dealing with the exsistance of "spam blocking" groups. Then I read a comment from an "anti-spam" person and I think I'll be safer choosing to work that delete key.

    • ...his website is hosted on the same IP address as a spammer (textileshop.com) was on yesterday

      I'd say this neatly demonstrates the problem with blacklists. I agree that the style is marred by the emotional state of the author, but then it's an essay on the guy's personal page.

      If you want some analysis, start with a personal exmample of mine: an ISP in Israel my parents used to use would occasionally get blacklisted. Since I'm behind company-level spam filtering there was nothing I could do about i

  • Vigilante it ain't (Score:4, Insightful)

    by Rosco P. Coltrane (209368) on Thursday June 16, 2005 @03:24PM (#12834805)
    The problem was, as vigilantes so often do, the guys at MAPS got carried away

    For some reason, journalists keep calling blackmail lists "vigilantes". But there's something they don't understand: nobody forces email system administrators to use those lists.

    These lists are provided by people for free. They decide to list bad email servers, but they may as well include any server they want. After all, who's to force them to provide quality of service?

    The real problem, of course, is that blacklists are needed in the first place. If ISPs did their jobs a little better (aol, hotmail and the likes), the amount of spam would already decrease significantly. And don't speak to me about chinese ISPs, since most spam comes from the US.
    • AOL actually does a good job filtering SPAM. I can't imagine AOL being used to send any significant amount of SPAM at this point, and it does a fine job of filtering incoming SPAM.
    • blackmail lists

      I meant blacklists of course...
    • by Maestro4k (707634) on Thursday June 16, 2005 @03:38PM (#12834924) Journal
      For some reason, journalists keep calling blackmail lists "vigilantes". But there's something they don't understand: nobody forces email system administrators to use those lists.
      No, but the non-spamming sites that end up on it would certainly disagree with you, they didn't do anything to merit the block.

      You seem to be confused about what a vigilante is, dictionary.com gives me this: "One who takes or advocates the taking of law enforcement into one's own hands." Note it doesn't say anything about them forcing others to agree with their views or take part in them. If you decide to take legal actions in your own hands, then you are, by definition, a vigilante. So it does apply here, just because they don't force anyone to use their lists doesn't change that.

      These lists are provided by people for free. They decide to list bad email servers, but they may as well include any server they want. After all, who's to force them to provide quality of service?
      TFA's point was that these lists start out listing just IPs/hosts/sites they know are sending spam, then later the power corrupts ("power corrupts, absolute power corrupts absolutely") them and they start using the power they've gained by their blacklist being used by many people to start trying to force ISPs to comply with them by blocking bunches of innocents at the same ISP. That indeed has happened, although I'm really not sure if it's happened here or not. The risk of it occuring is pretty high, humans are, after all, only human and it's hard to resist that temptation, especially when you're a strong enough anti-spam advocate to run a blacklist.
      The real problem, of course, is that blacklists are needed in the first place. If ISPs did their jobs a little better (aol, hotmail and the likes), the amount of spam would already decrease significantly. And don't speak to me about chinese ISPs, since most spam comes from the US.
      The real problem is human nature in all of this. In spam existing in the first place (greed), in ISPs not blocking things they should (laziness, lack of knowledge or time), in people actually buying from spam (greed (getting something cheaper than legal means would allow), sexual desire (gotta have a longer penis!) or just simply a criminal desire to purchase illegal goods (prescription drugs for example)) as well as humans becoming corrupted by power when their blacklists get to be popular.

      So basically if we can solve how to get people to stop being, well, people and giving in to baser instincts we can stop spam. Of course we'd also stop crimes of all sorts as well and we've not managed that in hundreds of years so I'm not holding my breath for it to happen.

    • by Valdrax (32670)
      For some reason, journalists keep calling blackmail lists "vigilantes". But there's something they don't understand: nobody forces email system administrators to use those lists.

      To be honest, I like his other analogy for blacklist maintainers -- terrorists. It's much truer to the point. Vigilante in my mind at least implies an attempt to go after the bad guys and protect the innocents thanks to the pop culture influence of TV, movies, and superhero comics.

      This doesn't describe blacklist maintainers.

      Bl
    • This argument is horseshit. It's been horseshit for years and it will always be horseshit. The blacklists exist for the sole purpose of allowing other people to block mail based on the data contained therein. The blacklist operators don't get off the hook for having some frickin' responsibility just because they're not holding a gun to anyone's head. They publish this information with precise knowledge of what it will be used for, so this argument is basically just the administrators trying to weasel out of
  • A Paradox? (Score:4, Insightful)

    by LegendOfLink (574790) on Thursday June 16, 2005 @03:24PM (#12834809) Homepage
    A blacklist for a blacklist for a blacklist...

    Personally, I find the need to disable more and more RBL's, because today a user might come thru OK, tomorrow, they're stuck in SORBS and considered a HIGH risk.
    • Yep. Turtles all the way down.
      I forsee a split between the www 'wild, wild, west' and private networks that you pay real cash money and have a smart card with certificates on it to play (or some variation on the military theme you see here [osd.mil]), just so the wheat is available, and you can surf the chaff if you want to.
      One wonders if some marketing twit won't tie these ideas to IPv6, as a forcing function to sell that technology to an otherwise indifferent market.
  • Actual quote I have heard on the subject of spam blacklists: "I don't care that you're not a spammer. Your ISP allows spammers in their midst and therefore you all go on the list. Get a new ISP."

    Oh, ok. Nothing like over reacting a bit.

    • by Uruk (4907) on Thursday June 16, 2005 @03:55PM (#12835099)
      No, the principle is that if ISPs know that this kind of overreaction will occur, they will make quite sure that they don't have spammers in their midst. In essence, it's an attempt to incentivize ISPs to police themselves.

      What's the alternative? Having some centralized, international spam cop whose job it is to clean up every ISP on the planet? If ISPs get a completely free pass on spam and don't have to care whether their subscribers are abusing other people or not, where is their incentive to prevent the abuse? The way you avoid the tragedy of the commons is by getting people to see their individual stake in the issue.

      Certainly the quote that you're pointing out isn't the most diplomatic or effective way of putting it, and I doubt this kind of thinking is behind that quote - it probably is the knee-jerk reaction that you're identifying it for. Still, the idea might have some merit.

    • There have been spam blacklists that worked that way; they mostly weren't worth using, except as SpamAssassin weights, and mostly nobody cares. And there have been Open Relay blacklists that blacklisted every mail server at an ISP to get their attention until they cleaned up open relays, even if only some of that ISP's customers had open relays.

      But this is different - this is ONE IP address - the SBL record identifies it as a /32. Virtual Hosting means that it's possible to have multiple domains all usin

  • Pure and simple... (Score:5, Insightful)

    by jellisky (211018) on Thursday June 16, 2005 @03:27PM (#12834835) Journal
    I had the unfortunate "joy" of being blocked by some of these draconian blacklists. My sister requested some information from me for a trip that she has upcoming via my yahoo.com account. After it bounced from her ISP saying that I was sending it from a "spam-hosting" ISP, I sent it from my mac.com account. Same schtick. After a couple other choices, I finally got it sent from my .edu account.

    Her ISP uses SpamBag for their blacklist. SpamBag? ScamBag is more like it.

    No wonder my sister is disenchanted by email. Her yahoo account got spammed to no end, then she can't get emails from most of her friends since they get bounced back by her ISP's stupid blacklist.

    Blacklists are fine and dandy in principle, but practice has shown them to be useless. IT managers, just drop them. They're more annoying than anything.

    -Jellisky
    • SpamBag is run by Sam Varshavchik, the author of Courier. A singularly most unpleasant and moronic individual.

      I had the misfortune to cross his path a number of years ago about an issue with Courier I believe or something else, I can't quite recall, and I will never forget it. He is one of the most vitrolic, annoying, moronic individuals I have ever come across. I'm amazed he was able to produce something as nice as the Courier MTA package, but I guess idiot savants like him can do good things. It's ju
    • by aaronl (43811)
      Yes, fun isn't it? Trying running your own email server from a Charter business link. Then try sending email to Juno or NetZero customers. Their mail server will give you a 550 denied. Proceed to have the ISP's ignore you, and the RBL jerks ignore you.

      The reason for the block? All Charter IP addresses have been put into a "residential" blocklist by one RBL nut that decided such a list was a good idea. Everyone knows that you should have to buy a T1 to send email. This is because people who really ne
  • by Alaren (682568) on Thursday June 16, 2005 @03:28PM (#12834841)

    I work in lower-level management for a major domain name registrar and a significant number of our tech support calls every day concern bounced emails. Between Spamhaus and our RBL, we bounce a lot of email.

    Worst of all, many RBL's (including ours) pretty much block any dynamic IP pool as "dial-up zombie hell"--but a lot of these IPs get reallocated to broadband as DSL coverage grows, and we end up blocking genuine mail routers and SMTP relays instead.

    We have a process for requesting the unblocking of an IP, but about 30% of the time the answer is "tough luck, pal. You (or your client who is trying to reach you) is blocked and we aren't unblocking you." There is no appeal process; if one of our admins decides not to unblock your IP or IP range, you can't reach any of our customers via email.

    In short, these tools may block a lot of spam, but they also block a lot of legitimate communication. I think the appropriate metaphor is that RBLs are like tourniquets--viable for quick fixes, but eventually they do more damage than they prevent.

  • but five minutes later they should have recognized the likelihood of unintended consequences and looked for a better solution, much as our fine lawmakers always do....oh, wait....
  • All blacklists get corrupted over time. On the other hand, new ones won't be very effective because they don't have enough spammers on them. You have to choose what false positive level is acceptable to you.
  • by tmk (712144) on Thursday June 16, 2005 @03:29PM (#12834848)
    I have found an interesting offer: pay 50 bucks and you are removed immediately from the spam list. Have a look here [uceprotect.net].

    Interesting: The company won't say who they are. [admins.ws] They say this was approved by local authorities, but this is bullshit. Local authorities can not brake federal law in Germany.
  • by Mr.Progressive (812475) on Thursday June 16, 2005 @03:29PM (#12834852)
    Blacklists have a structural flaw: there is no one to watch the watchers.

    Lisa: If you're the police, who will police the police?
    Homer: I 'unno, Coast Guard?
  • by redelm (54142) on Thursday June 16, 2005 @03:30PM (#12834855) Homepage
    ... the Watched, of course! Ruel enforcement isn't a heirarchy but a loop.

    Blocklists are made by people for others to use if they see fit. When they become unusable, they're no longer used. Personally, I use none. The cost to me of one false positive is greater than 1000 spams that leak through. No list is that good.

  • by a7244270 (592043) on Thursday June 16, 2005 @03:31PM (#12834874) Homepage Journal
    OK, so PG wrote some code in the past, and is generally a smart guy, and to be honest, I actually like his writing. I like it enough that I'll even read his stuff despite the fact that he uses an excessively narrow column width for his text which makes it very annoying to read. However, there are many blogs out there written by smart programmers, some with far, far, far more geek cred than PG.

    Why exactly is this a Slashdot story ?
  • I've been considering going to a whitelist only system.. Everyone I know gets on a whitelist, and my personal website/webpage will have a CAPTCHA and a way to suggest your name onto my whitelist.

    Pratical for me? Yes, but I wonder how well it would apply to other users.
  • by WebHostingGuy (825421) * on Thursday June 16, 2005 @03:37PM (#12834915) Homepage Journal
    We deal with this all the time. Leaving any IP on a blacklist for any period of time doesn't help. Most spammers nowdays spam and run. They unload from a hacked account through a broken formmail script or a zombie computer. After 36 hours they have dumped their million emails and moved on to another IP. Blacklists generally don't get this though. They just make a bigger and bigger list. The problem with this approach is that they already missed the spammer. One time we dealt with someone who was running a blacklist and when we asked why an IP was on the list they said because it spammed years ago. When we said we have controlled the IP for the past three years they said it doesn't matter. It's like give me a break...

    The solution to blacklists is to use an AOL model in which dynamic IP blocking is used. When spam is noted from an IP that IP is automatically blocked for 24-36 hours after the last spam comes in. That way the innocents are not being blocked and the spammers email doesn't make it through. There are a couple blacklists which do this but more should.

    Compare this to the opposite blacklists like BLARS which requires a thousand dollars for "him" to investigate whether an IP should be removed. I have never seen an IP which is not listed with BLARS.
    • Spamcop's RBL does exactly what you're suggesting. Their automated system automatically "retires" IP addresses from the RBL after set amounts of time. It goes one step further though, and determines the suitability for longer-term inclusion on the list based on the IP's history of spamming. It works exceptionally well.

      I have been the victim of the formmail exploit, and been RBL'd as a result. It was not difficult to get un-blocked. Yes, it was a hassle, but I suspect those that complain about being RB
  • by argent (18001) <peter@slashdot.2 ... m ['nga' in gap]> on Thursday June 16, 2005 @03:42PM (#12834971) Homepage Journal
    People switched from MAPS because the other lists were free, not because MAPS was too aggressive.

    "As of this writing, any filter relying on the SBL is now marking email with the url "paulgraham.com" as spam."

    Whisky Tango Foxtrot? *BLs block IP address ranges, not URLs.

    "Because the guys at the SBL want to pressure Yahoo, where paulgraham.com is hosted, to delete the site of a company they believe is spamming."

    1. Given that Paul's mixing up URLs and addresses of mail servers, I'm not prepared to take at face value the statement that SBL is blocking Yahoo's mail servers to pressure Yahoo to drop a "site", rather than (say) mail services Yahoo is providing the spammer.

    2. If Yahoo is providing services to a spammer and Yahoo refuses to deny those services to a spammer, than Yahoo is being "spam friendly", no matter what their reputation is, and they may well be depending on the many legitimate lists they're hosting to avoid responsibility for their actions. That's exactly the situation that John Reid is referring to in Paul's quote.

    I don't know what alleged spammer this is referring to, but what Paul's written is clearly not anywhere near the whole story.
  • by egburr (141740)
    The DUL is another very annoying list. Earthlink reports all of its cable modem customers to DUL because we are forced to use "dynamic" addresses with DHCP. My address is so dynamic it has changed once since I because a customer, and that change occurred three years ago. For DSL customers, Earthlink offers a special service: a static IP address for only $15/month extra. Cable subscribers don't get that option. I really have to wonder how that static address could possibly cost them any more to maintain than
  • by bitflip (49188) on Thursday June 16, 2005 @03:47PM (#12835027)
    I use blacklists all the time. Rather than simply rejecting the mail, if the server is on a blacklist, the initial OK is delayed by five seconds.

    If you're sending a ton of mail, i.e., spam, little of it gets through. If you're only sending one or two messages, ie, likely legit mail, it goes through just fine.

    Combined with more specific stuff further back (bayes, et. al), it's been quite effective at reducing the amount of spam sent, and the amount of mail that gets scanned.

    The problem isn't blacklists, its how people use them.
  • Blacklisting is clearly just opening more oppurtunies for cyber-crime: spammers threatening to get companies blacklisted by major ISPs unless they pay up. Sending a few emails from fake addresses to the right places is a lot easier than organising DoS attacks from BotNets.

    Loss of email hurts more too.
  • by slavemowgli (585321) on Thursday June 16, 2005 @03:54PM (#12835086) Homepage

    Interestingly enough, the owner of the acme.com domain who was recently featured in a story due to his getting more than a million spam mails (well, attempts to send spam) a day, agrees:

    DNS-RBLs - Domain Name System Realtime Black Lists. In theory the idea is fine. You have a set of sites that you blacklist, and you want to let other folks use the same list so you distribute it using DNS, which is a nice efficient de-centralized database. What's not to like?

    Well, I don't know why, but in practice every single DNS-RBL eventually comes under the control of power-hungry weenies. They start listing sites unreliably, and if you complain you find yourself listed. And there's usually no way to get off the list.

    A lot of people tell me I'm wrong about this. They say that certain DNS-RBLs are ok, with objective criteria for inclusion and simple procedures for getting off the list. The thing is, they give conflicting recommendations for which lists are good and which are bad. Some of these folks recommend lists which I know from personal experience are bad.

    This problem is really inherent in the way DNS-RBLs are set up. You cede control of your mail system to a third party, with no real possibility of checking how they are doing. The people running the lists get overwhelmed with bogus feedback from spammers and/or idiots, to the point where they assume all their mail about the lists is from spammers and/or idiots.

    If the lists you use have not yet descended into corruption and chaos, consider yourself temporarily lucky.

    Do not use DNS-RBLs.

    (from http://www.acme.com/mail_filtering/shame_frameset. html [acme.com])

  • What a clusterfuck (Score:4, Interesting)

    by maynard (3337) <j...maynard...gelinas@@@gmail...com> on Thursday June 16, 2005 @04:03PM (#12835173) Journal
    blocking spammers via a central database just doesn't work. The spammers are constantly moving from zombie client to zombie client in huge waves of hundreds of thousands of infected systems, making the RBL always filled with obsolete and incorrect information. The problem - as everyone knows - is that the protocol is fundamentally broken. It's a tragedy of the commons played out in front of our eyes.

    By allowing the abuse it's outcome becomes a certainty. We're going to have to bite the bullet and dump open SMTP. And I think we're going to have to do this quickly. The levels of SPAM continue to rise. I often see ten to twenty times as many spam connections on my mail servers than legitimate connections, and this is a constant, flowing, amount of SPAM 24/7. Even with RBLs, spamassassin, etc, SPAM still gets through. The solution will not be found with another bandaid. It's time to dump SMTP and move to something that demands cryptographic authentication for users and hosts before allowing the transport session to complete. --M
  • Although MAPS did, indeed, only blacklist the actual spammers at the beginning, they changed not because they 'got carried away' (Paul Graham's words), but because the spammers adapted.

    Here [online2000.net] is the link, that responsible editors would've offered in a story like this...

  • by otter42 (190544) on Thursday June 16, 2005 @04:20PM (#12835405) Homepage Journal
    This is, strictly speaking, terrorism: harming innnocent people as a way to pressure some central authority into doing what you want.

    No. No... No, there's just something not right about that. I'm pretty sure that the definition of terrorism includes the idea of terror somewhere...

    Ahhh. That's more like it: Terrorism: the unlawful use or threatened use of force or violence by a person or an organized group against people or property with the intention of intimidating or coercing societies or governments, often for ideological or political reasons.

    Yeah, violence should induce terror. Not being able to send emails to my girlfriend, as hair-raising an idea as that might be, just doesn't seem to be in the same league.

    And just in case Mr. Graham is too lazy to find a dictionary to look up hyperbole for himself: hyperbole - n : extravagant exaggeration
  • by jdunlevy (187745) on Thursday June 16, 2005 @04:26PM (#12835478) Homepage
    From TFA [paulgraham.com],
    As of this writing, any filter relying on the SBL is now marking email with the url "paulgraham.com" as spam. Why? Because the guys at the SBL want to pressure Yahoo, where paulgraham.com is hosted, to delete the site of a company they believe is spamming.
    E-mail w/ the 'url "paulgraham.com"'? The SBL doesn't check URLs, it'd doesn't even check domain names, it checks IP numbers. paulgraham.com resolves to [66.163.161.45], which is listed in the SBL [spamhaus.org] (details for SBL27945 [spamhaus.org]), but since this isn't a mail server, I don't see how e-mail from paulgraham.com gets marked as spam by users of the SBL. I note that the MX record for paulgraham.com is milter1.store.vip.sc5.yahoo.com [216.136.232.238], which is not in the SBL [spamhaus.org]. He never mentions what he uses as his smtp server, but I'm supsecting it either not the SBL -- or it's in for a different reason than he thinks.

    Also, for what it's worth, I've found the SBL incredibly reliable (except recently, when I've found it's been increasingly unreachable at peak times), but I check it as one of many spamassassin rules -- I don't mark e-mail as spam just because it's in the SBL, though the way I have spamassassin score things, it doesn't take much more...

  • by jellomizer (103300) * on Thursday June 16, 2005 @04:30PM (#12835520)
    Just block the sub net 0.0.0.0
  • Distributed List (Score:3, Interesting)

    by suwain_2 (260792) on Thursday June 16, 2005 @05:23PM (#12836017) Journal
    The problem with blacklists is that -- the guy who recently had a story on spam here, at acme.com, put it nicely -- blacklists start off good, but always turn corrupt and start blacklisting excessively.

    Suppose a "distributed" blacklist were created. I could blacklist the whole Internet, but I'd be the only one, so it wouldn't mean a thing. On the other hand, if 75,000 people have blacklisted an IP, there might be something there.

    It needn't be totally distributed, I don't think. A community-run site, where, whenever you get obvious spam, you post the originating IP, could work. You'd post it, and that IP would have, say, 10 "points." The rating would "decay" by one point a day, so a site listed, but that went clean, would quickly leave the list: in ten days, each rating would be down to zero.

    You could then simply query the site for a given IP, and it'd return the "points" a site had. This also allows you a lot more customizability: if you were obsessed with blocking all potential spam, you could block anything with more than 5 points. If you wanted to be careful, you might set it to, say, 1000 points.

    Unless the people running the site keeping track of the ratings begin blatantly making up ratings, this idea means that a blacklist is much less immune to being "bad." And it allows IPs to "fade" out of the list over time.
  • by Pig Hogger (10379) <pig.hogger@NoSpAM.gmail.com> on Thursday June 16, 2005 @05:31PM (#12836109) Journal
    There are many, many private blocklists that are not advertised anywhere.

    Here is my very own private /etc/mail/access blocklist which I use on my own mail server:

    #
    12.217.112 550 Mediacom. Heh. What a fucking spamming cesspool. So why not eat shit and die???
    12.217.113 550 Mediacom. Heh. What a fucking spamming cesspool. So why not eat shit and die???
    12.217.114 550 Mediacom. Heh. What a fucking spamming cesspool. So why not eat shit and die???
    12.217.115 550 Mediacom. Heh. What a fucking spamming cesspool. So why not eat shit and die???
    12.217.116 550 Mediacom. Heh. What a fucking spamming cesspool. So why not eat shit and die???
    12.217.117 550 Mediacom. Heh. What a fucking spamming cesspool. So why not eat shit and die???
    12.217.118 550 Mediacom. Heh. What a fucking spamming cesspool. So why not eat shit and die???
    12.217.119 550 Mediacom. Heh. What a fucking spamming cesspool. So why not eat shit and die???
    24 550 Comcast, when you'll have cleaned your zombies, you can knock here. Not before.
    24.174 550 Chuck Jones must be spinning in his grave when he see he's associated with spam. Close port 25, fuckers.
    59.0 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.10 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.1 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.11 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.12 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.13 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.14 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.15 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.16 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.17 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.18 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.19 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.2 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.20 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.21 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.22 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.23 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.24 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.25 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.26 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.27 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.28 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.29 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.3 550 It's not surprizing that a country split in two like Korea would have a totally fucked-up "internet".
    59.30 5

  • by Steve Linford, Spamh (765605) on Thursday June 16, 2005 @05:48PM (#12836286) Homepage

    Gentlemen,

    You do realize that Paul Graham is in the business of pushing Bayesian anti-spam filtering, which he claims as 'the best' solution to spam. For a long time Graham has been spreading FUD about other anti-spam solutions, in particular blocklists. We're well used to hearing utter bollocks about blocklists spread by him.

    Yesterday we listed on the SBL an IP of a spammer which as luck would have it is being shared by Paul Graham. We of course can not simply give the spammer carte blanche to spam our users because Paul Graham is also using the same IP. Graham has no concern for the fact he's sharing his IP with a spammer, and rather than contact his ISP to ask what a spammer is doing sharing his IP he simply sees a PR oppurtunity to bolster his "blocklists are evil, bayesian is good" campaign. I'm only surprized this actually made Slashdot.

    Steve Linford, CEO, Spamhaus
    • by Dr.Dubious DDQ (11968) on Thursday June 16, 2005 @07:59PM (#12837309) Homepage

      Considering how much my spam has been reduced by the SBL (anywhere from at least 50% up to 75%) I'd like to just say:

      The mail servers under my control have always subscribed to the SBL-XBL (well, more accurately, before the XBL was established it was the SBL and cbl.abuseat.org. The latter is dedicated to short-term [72 hours, as I recall] blocking of e.g. spammers operating on DSL or cablemodem lines who are likely to appear on an IP address once or twice and then get kicked off. The CBL is now also represented in the XBL). I have so far, in the last 3-4 years or so, only been able to confirm 1 and 1/2 "false" positives in that entire time - one was from a person in China who was using a confirmed spam-haven ISP, the "1/2" from a company that, after an informative response from the CBL people, I believe were listed for appropriate reasons. In any case, the latter case cleared itself up when they were automatically re-removed from the CBL [they'd been there before] and the email lost WAS an advertisement anyway...)

      I have noticed the numerous stories of overzealous blocklists, which are obviously a bad thing, but I can't think of a way to reasonably put the SBL in that category...

      Besides, bayesian filtering only works AFTER the spammer has been allowed to tie up my mail server's bandwidth (and then allows them to tie up your mail server's CPU time with the bayesian analysis). I prefer to cut off known spammers before that point whenever possible. THEN I pass the remaining messages through SpamAssassin. Back in the early days of spam, I used to actually go to the effort of picking apart the mail headers and looking up the abuse addresses for the ISP whence the mail came AND the hoster of the spammers website (and on one or two occasions, even the registrar for the spammer's domain name, when I could confirm that the information was falsified). It's been a long time since I was able to keep up doing that with the volume of spam coming in, but I still can't stand the thought of allowing spammers to take ANYTHING from me that I can prevent...

  • Terrorism? Hardly. (Score:3, Insightful)

    by ChaosDiscord (4913) on Thursday June 16, 2005 @06:45PM (#12836796) Homepage Journal

    Graham has written some insightful and well thought out stuff, but this is just sloppy:

    This is, strictly speaking, terrorism: harming innnocent people as a way to pressure some central authority into doing what you want.

    I find it amazing that blacklists which mail servers must opt-in to use are somehow terrorism. Are you suggesting that these innocent people have some fundamental right to contact my mail server and send mail? They certainly don't; it's my mail server. I can use any methods I like to filter out mail, including chosing to rely on one of the IP blacklists. This can only be terrorism if random people have some sort of human right to send mail to my machine. I hardly think that's a right.

    Come to think of it, apparently organizing against tangentally related people to stop another problem is terrorism? By that strange standard you could call advertiser boycotts terrorism: you're trying to influence some media outlet by negatively influencing advertisers on that outlet. They often have the same claim of innocence ("I didn't know that they would run that article! I just buy bulk advertising rates.")

    (Now there are problems with blacklists, perhaps most significantly that many ISPs use them without informing their subscribers or allowing them to opt out. Blacklisting unaware users who happen to share a machine with a spammer's website is definately a complex question.)

  • by TCM (130219) on Thursday June 16, 2005 @07:10PM (#12836963)
    Going away from SMTP, I am currently running a Squid HTTP proxy with a quite long blacklist of URLs and networks of "marketing" and "ad" companies.

    I find myself doing for example a lookup of ad.marketingscum.com followed by a whois lookup of the IP address. If I find that they own a larger network like

    NetRange: 216.73.80.0 - 216.73.95.255
    CIDR: 216.73.80.0/20
    NetName: DOUBLECLICK-NET

    I enter the complete network into my blacklist. Are there any realtime blacklists for this purpose? This would be quite useful, wouldn't it?

If it's worth doing, it's worth doing for money.

Working...