Forgot your password?
typodupeerror
Bug Spam IT

SpamAssassin 2010 Bug 115

Posted by Soulskill
from the who-could-have-known-when-2010-would-happen dept.
SEWilco writes "You might want to check your spam folder, as SpamAssassin has a rule which is tending to mark email sent in 2010 as spam. There is some discussion in a bug report. The SpamAssassin Wiki FH_DATE_PAST_20XX page doesn't have discussion, but it was updated today with a different date rule."

This discussion has been archived. No new comments can be posted.

SpamAssassin 2010 Bug

Comments Filter:
  • Instead of having one millenium bug, letting it do it's thing and get it over with we get similar bugs every year. The 2007 zune issue, the impending 2038 issue and this. Of course many more similar bugs that don't deserve their own slashdot article. If they had decided to start using 64-bit time on the 1st of January, 1970 none of these problems would have happened
    • by fuzzyfuzzyfungus (1223518) on Friday January 01, 2010 @09:19PM (#30618110) Journal
      Given what memory cost in 1970 [jcmit.com], I suspect that using 64-bit time would have been an expensive decision.

      A lot of gross little hacks look like (and are) great ideas when hardware costs a fortune and you don't yet know how persistent legacy stuff is going to be.
      • You're almost right, but actually you miss the point. Back in 1970, RAM cost around $1/byte, so using 32 bits instead of 64 saved you $4/record. However, that wasn't what caused the millennium bug, it was choosing a silly representation to use to store the date. If you wanted to, you could have stored dates as a 32-bit quantity in seconds since some fixed time and been able to handle any date in a 140 year period. You could also have performed calculations on these dates very quickly (just simple arithm

    • by JWSmythe (446288)

          I'm actually thankful that they ran this one as a story. After seeing it, I remembered that rule. I just hopped in and fixed my servers.

          The better rule would have been to watch for dates more than X days from today. But, they're using regular expressions, so that makes it a bit harder.

      • by ls671 (1122017) *

        > I'm actually thankful that they ran this one as a story.

        Yep, not that /. is a security list, but it occurred to me a few times that I first learn about similar issues on /.

           

    • by Sparr0 (451780)

      I think 2038 is a nonissue. Technological progress is accelerating. It took us 30 years to upgrade from 2 decimal digits to 4 for storing years. It will not take 30 years to upgrade from 32 bits to 256 bits for time.

    • by snaz555 (903274)

      If they had decided to start using 64-bit time on the 1st of January, 1970 none of these problems would have happened

      The pattern:

      ##{ FH_DATE_PAST_20XX
      header FH_DATE_PAST_20XX Date =~ /20[1-9][0-9]/ [if-unset: 2006]
      describe FH_DATE_PAST_20XX The date is grossly in the future.
      ##} FH_DATE_PAST_20XX

      Will match 2010 no matter how many bits are used to keep time. This is a problem that has been know for years; it's a total embarrassment to OSS practices that it wasn't fixed before 1/1/10, before becoming critical. It's not based on a bad architectural decision, or even particularly bad code - it's just a

  • crapola (Score:4, Interesting)

    by DNS-and-BIND (461968) on Friday January 01, 2010 @09:16PM (#30618076) Homepage

    My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon. Is there any way around this that doesn't involve root access? (I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file" or whatever.)

    Oh yeah, the other wonderfully helpful stock response "stop using the software if you don't like it". Sure, I'd love to go back to getting 500 spams a day.

    • Re: (Score:2, Informative)

      by Shikaku (1129753)

      Is there any way around this that doesn't involve root access?

      No.

    • Re:crapola (Score:4, Funny)

      by godrik (1287354) on Friday January 01, 2010 @09:24PM (#30618182)

      I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file"

      Yes. That's easy! First edit the /etc/spamassassin.conf file!
      Wait a sec...

    • Re:crapola (Score:4, Interesting)

      by DaMattster (977781) on Friday January 01, 2010 @09:31PM (#30618252)
      Well, I am not sure what OS your running, but you can use OpenBSD Spamd which works very well. Rather than taking a defensive approach, Spamd goes on the offense by allowing known spam-sending IP addresses to attempt to send to you but throttling the connection down to 1 byte per second. This shakes most people off with no perceivable impact on your part. Even if the spam bot decides to wait the entire time to complete the connection, Spamd ends up dropping the message anyway. I use this solution in my business and I've gone from getting 500+ per day to maybe 2 per week. It is delightfully elegant.
      • Re:crapola (Score:5, Funny)

        by karnal (22275) on Friday January 01, 2010 @09:36PM (#30618310)

        I think you missed the "My provider runs spamassassin" part of the parent post.

      • Re: (Score:2, Informative)

        by Mana Mana (16072)

        OpenBSD spamd(8) is wholly unrelated from spamassassin spamd. FYI.

        OpenBSD spamd(8) has no code from any other project. Its similarity in appellation is name deep.

        OpenBSD spamd(8) approach is different and was created by deraadt@.

        • OpenBSD spamd(8) is wholly unrelated from spamassassin spamd

          Not entirely. You can combine the two quite well. The spamdb tool can be configured quite easily to take input from SpamAssassin, so if someone sends you spam they then get blacklisted or more aggressively greylisted.

    • Re:crapola (Score:5, Informative)

      by ngc5194 (847747) on Friday January 01, 2010 @09:35PM (#30618296)
      "Is there a way to work around this that doesn't involve root access?"

      Yes, but it isn't a good way. Check your scores file for the scores associated with the FH_DATE_PAST_20XX. This indicates the number of points added to the spam score of every message that fails this test. Basically, increase your spam threshold by this amount until you can apply this patch.

      Good for a quick-n-dirty fix.

    • Re: (Score:3, Informative)

      depends--i have a cheap n cheerful shared hosting account with the same issue, but i do have cpanel access which allows me to override the score for any rule--check out the last link in the summary basically if you have access to local config files (even through a frontend like cpanel) you can do it without root access
    • Re: (Score:1, Offtopic)

      by darkpixel2k (623900)

      My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon. Is there any way around this that doesn't involve root access? (I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file" or whatever.)

      Oh yeah, the other wonderfully helpful stock response "stop using the software if you don't like it". Sure, I'd love to go back to getting 500 spams a day.

      My car is so old it doesn't have an airbag. I heard about this glitch where I have a higher chance of dying because my car doesn't have an airbag. Given my track record in buying new cards, I doubt I'll update my gremlin anytime soon. Is there a way around this that doesn't involve require spending money? (I love helpful responses from idiots that start with "first go to NAPA" or whatever.) Oh yeah, the other wonderfully helpful stock response "stop driving the car if you're worried about dying in a wr

    • Re:crapola (Score:4, Informative)

      by mhrivnak (752549) on Friday January 01, 2010 @11:44PM (#30619228)

      The new rule gets picked up when "sa-update" is run. spamassassin deployments should run sa-update automatically on a regular basis, for example every day via a cronjob. Thus, most deployments will pick up the update automatically tonight if a sysadmin doesn't do it first.

    • Simplest non-root way (assuming they have it allowed): Edit the ~/.spamassassin/user_prefs file.

    • Re:crapola (Score:5, Informative)

      by nabsltd (1313397) on Saturday January 02, 2010 @12:10AM (#30619362)

      My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon. Is there any way around this that doesn't involve root access?

      If you have shell access, it should be trivial, although you do have to edit a file.

      Add the following to ~/.spamassassin/user_prefs:

      score FH_DATE_PAST_20XX 0.0

    • Re: (Score:3, Interesting)

      by Penguinshit (591885)
      Not to rub it in but I want to give a shout to my provider (Cruzio) who also use Spamassassin and were apparently on top of this (as they usualLY are0. i haven't noticed anything wrong with my email today. Thanks Cruzio!
    • My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon

      If they are slow enough updating, it's possible you don't have the bug. I haven't bothered to update my home mail server past Ubuntu 6.06 LTS, since it works fine. The version of Spamassassin it uses, 3.1.7, does not appear to have this bug.

    • Re: (Score:1, Troll)

      by Sleepy (4551)

      > (I love helpful responses from idiots that start with "first, edit the /etc/spamassassin.conf file" or whatever.)

      >Oh yeah, the other wonderfully helpful stock response "stop using the software if you don't like it". Sure, I'd love to go back to getting 500 spams a day.

      "Idiots?"

      You might have a point if the ONLY step began with "first". But here you see a bunch of people answered you.. and yes, ALL of them began with editing the "whatever" file.

      You know... paid support for SpamAssassin IS available.
      I

    • Oh yeah, the other wonderfully helpful stock response "stop using the software if you don't like it". Sure, I'd love to go back to getting 500 spams a day.

      Is the alternative of receiving no mail more palatable?

    • My provider runs spamassassin, and given their track record in updating their other software, I rather doubt that they'll update spamassassin anytime soon. Is there any way around this that doesn't involve root access?

      Not that I know of, but perhaps you might have access to a local.cf file in which you can set the score of the rule to a lower value or ignore it by setting the score to 0. Before the update I did this (in my case) by adding the following line to /etc/mail/spamassassin/local.cf: score FH_DATE_PAST_20XX 0.0 The new rule however is not released in a new spamassassin release yet, but the rules are pushed through the udpate channels, so perhaps your provider might have received the update. Othwerwise it is w

  • What the... must all SpamAssassin rule be regexes? Because I can't see any (non brain dead) reason why not to implement this by just checking if $current_year < $header_year if that's not the case.
    • by Hal The Computer (674045) on Saturday January 02, 2010 @02:33AM (#30620066)

      Your solution doesn't work.
      It fails on new years eve if someone is in a different time zone or if their clock is slightly off.

      I'd suggest that any message sent more than seven (pick your favorite number) days in the future is spam.

      • Re: (Score:3, Interesting)

        by jamesh (87723)

        Your solution doesn't work.
        It fails on new years eve if someone is in a different time zone or if their clock is slightly off.

        You need to talk to Microsoft - they could have used someone like you when they were building their 'stampinf' tool that I can't use until 10am in the morning (11am during DST) because it stamps UTC which then confuses another tool in the build chain...

      • ... or it could be an important message from your future self, containing instructions for making a time machine that send mails backwards in time.
  • FIX details: (Score:5, Informative)

    by drDugan (219551) * on Friday January 01, 2010 @09:27PM (#30618216) Homepage

    this is also happening on Ubuntu server, running Spamassassin 3.2.5

    The linked article references a workaround:
    add this line to the "local.cf" spamassassin config file, on this system is was /etc/spamassassin/local.cf

    score FH_DATE_PAST_20XX 0.0

    If you're running spamassassin as a daemon, you *may* also want to restart spamd
    with something like:

    sudo /etc/init.d/spamassassin restart

    This solution simply removes the rule by setting the score for that rule to 0.
    You'll want to undo this once a solution is deployed.

    • Re:FIX details: (Score:5, Informative)

      by KiloByte (825081) on Friday January 01, 2010 @09:33PM (#30618274)

      Since nearly 14 hours ago, you can simply run "sa-update".
      It is in cron.daily in the default install, too.

    • by nmb3000 (741169)

      score FH_DATE_PAST_20XX 0.0

      You'd probably be better off changing the rule instead of the score. Putting something like:

      # Fixes bug: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6269 [apache.org]
      header FH_DATE_PAST_20XX Date =~ /20[2-9][0-9]/ [if-unset: 2006]
      describe FH_DATE_PAST_20XX The date is grossly in the future.

      in your local SA config would probably be better (unless you really want to just drop the rule entirely).

      That said, you'd probably be better off updating everything with sa-update, but this f

    • by rduke15 (721841)

      If you are running amavis, sa-update alone is not enough. You also need to restart amavis so that it picks up the new rules. On a Debian system, that would be something like:

      sa-update && /etc/init.d/amavis restart

      The spamassassin daily cron job does sa-update, and reloads spamassassin, but doesn't reload amavis.

  • by tomp (4013) on Friday January 01, 2010 @09:42PM (#30618378) Homepage

    From the "fix"

    > FH_DATE_PAST_20XX
    > change '/20[1-9][0-9]/' to '/20[2-9][0-9]/'

    That's no fix, it just puts the problem off for another 10 years. Why call the rule FH_DATE_PAST_20XX, shouldn't it be FH_DATE_PAST_201X? At least then the hack would be documented.

    • Re: (Score:3, Funny)

      by JWSmythe (446288)

          I just fixed a mail server for someone (after reading the article), and told them to remind me in 10 years to fix it again. :) Hopefully by then, they'll have rewritten the rule so it behaves better.

      • by 6Yankee (597075) on Saturday January 02, 2010 @12:01AM (#30619310)

        Exactly. This is the right "fix" right now - the bug is out there causing real problems, and the fix (while definitely a filthy hack) is well-understood and can be pushed out immediately. If the same thing were to happen ten years from now (or the threshold quietly got pushed back to 2030), that would be nothing short of negligent.

    • by jamesh (87723)

      That's no fix, it just puts the problem off for another 10 years.

      I had that thought too. Then I had another thought which was 'Meh'.

    • This won't help
    • I can only guess that it's a rule to flag e-mails as being spammy (all of my new years' wishes to brits and americans I got replies to with the subject tagged "{Spam?}") when they are 'sent from the future'.
      I can also only guess that this is done via a regex on the date with the date being somewhere far into the future (but that future kinda crept up on us the other day, right?) so as not to burden CPUs with actual date conversions to universal time and doing comparisons on that, and flagging

      • by Anders (395)

        2. spammers will be readily aware of this rule and really why would they make an effort to inject a future date anyway when their mail daemons will happily use the -current date-

        They will use dates from the future to have their offerings shown at the top of the mailbox.

        • by mikael_j (106439)

          Then there's the other date-mangling tactic, setting the date weeks or months in the past, I'm guessing they're doing this to make recepients think that the spam is some old legit mail they forgot about, it's annoying when one of those slip through though...

          /Mikael

    • by tendays (890391)

      Why call the rule FH_DATE_PAST_20XX, shouldn't it be FH_DATE_PAST_201X? At least then the hack would be documented.

      Changing the rule name would break existing configuration files changing the rule score and/or description (like in that comment [slashdot.org]).

  • That is so last decade ;P
  • Nice.... (Score:5, Funny)

    by kramer (19951) on Friday January 01, 2010 @09:59PM (#30618530) Homepage

    [url]https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5852[/url]

    Noticed 14 months ago. Fixed 5 months ago. Released today.

    • Re: (Score:3, Insightful)

      by Bigjeff5 (1143585)

      Hey! Nice to see open source software gets fixed so ultra fast! :P

      To be fair though, at least they released it the day it broke things, why didn't they release it by yesterday? Then the default cron job would have picked it up on most servers and nobody would have noticed.

      • by Anders (395)

        Hey! Nice to see open source software gets fixed so ultra fast! :P

        My sarcasm-o-meter is broken, so I cannot tell whether you are kidding. But it is indeed impressive to have a fix within hours, on a holiday.

        To be fair though, at least they released it the day it broke things, why didn't they release it by yesterday? Then the default cron job would have picked it up on most servers and nobody would have noticed.

        Why didn't they release it before someone noticed? Well, that's a good question. Maybe it was because nobody had noticed?

      • Re: (Score:1, Insightful)

        by Anonymous Coward

        Why didn't they release this when it was fixed.

    • by Pretzalzz (577309)

      spamassassin:
      Installed: 3.2.5-6
      Candidate: 3.2.5-6
      Version table:
      *** 3.2.5-6 0
      701 http://ftp.us.debian.org/ [debian.org] testing/main Packages
      70 http://ftp.us.debian.org/ [debian.org] unstable/main Packages
      100 /var/lib/dpkg/status

      apt-get updated 30 seconds ago.

      • Re: (Score:3, Informative)

        by John Hasler (414242)

        toncho/~ sudo apt-get install spamassassin
        Reading package lists... Done
        Building dependency tree
        Reading state information... Done
        Suggested packages:
        libmail-dkim-perl
        Recommended packages:
        re2c
        The following packages will be upgraded:
        spamassassin
        1 upgraded, 0 newly installed, 0 to remove and 1086 not upgraded.
        Need to get 1097kB of archives.
        After this operation, 0B of additional disk space will be used.
        Get:1 http://ftp.us.debian.org/ [debian.org] unstable/main spamassassin 3.2.5-7

        • by Bigjeff5 (1143585)

          I know y'all are all proud about how fast it got fixed, but in reality it was discovered over a year ago, fixed half a year ago, and was only released today.

          Why? Why not at least release it yesterday so most people's daily update would pick it up and make the whole thing invisible?

          • Re: (Score:3, Informative)

            by doshea (1711648)

            It was an oversight. The rule fix got committed but never added to the update channel. Nobody noticed before it was too late.

          • > I know y'all are all proud about how fast it got fixed...

            Where do you get that? I posted to inform people running Debian that they need not apply the patch. That's all. I expressed no opinion about the speed of the fix.

  • So this is why my Schwab Stock Alert was marked as Spam this morning.

  • by Anonymous Coward

    Ahh, the y2k.01 bug, naturally.

  • holy crap (Score:2, Informative)

    by Anonymous Coward

    Thanks for the heads up, my kid's birthday party is next weekend and when I look at the spam folder it turns out 3 more people have replied that I hadn't seen.

  • Great workaround (Score:4, Informative)

    by xororand (860319) on Friday January 01, 2010 @11:00PM (#30618978)

    The suggested fix is just silly... They postpone the problem to 2020-01-01:
    3) change '/20[1-9][0-9]/' to '/20[2-9][0-9]/'

    • by doshea (1711648)

      We're likely going to drop the rule all-together in the future. In the interest of getting the fix out fast, with the least confusion of users, we decided to just make the change to 2020 for now.

      Although... maybe a little visibility for the project every 10 years wouldn't hurt. :)

      • by Phroggy (441)

        Ideally it should use an offset from the current date (which assumes the system clock is correct, but on a mail server it really REALLY should be).

    • by Anonymous Coward

      Have you ever done production operations before?
      Goal 1: Get People Working Again.
      Goal much-farther-down-the-list: Document, Debug, and get people on the issue to properly patch/ensure it never happens again.

  • End User (Score:3, Interesting)

    by Kenshin (43036) <kenshinNO@SPAMlunarworks.ca> on Friday January 01, 2010 @11:39PM (#30619198) Homepage

    As an end user, with no real ability to configure SpamAssassin other than through cPanel, what can I do about this? (Unless my webhost is right on top of it.)

    I've disabled it, for now.

    • by Bigjeff5 (1143585)

      sa-update, assuming you can run that through cpanel.

    • Re: (Score:3, Informative)

      by 6Yankee (597075)
      Making sure they're aware of the issue might be a good place to start.
    • by doshea (1711648)

      If you can't get the rule to stop firing (by changing the rule or zeroing the score) you could increase your spam threshold by the amount of the rule's score (anywhere from 2 to 3.6 or more depending on your config). This would essentially cancel out the rule.

    • as pointed out in a couple of places above, without root access you may still be able to edit your own user prefs for spamassassin, either by editing the /.spamassassin/user_prefs file, or through cpanel dpending on your provider's setup--in the configure options (where you can set required_score for example) you add the FH_DATE_PAST_20XX rule to one of the 'score' boxes with a value of 0--but note that this disables the rule entirely--i did it last night and my inbox was full of spam this morning, currentl
  • by darthwader (130012) on Saturday January 02, 2010 @01:19AM (#30619700) Homepage

    "You might want to check your spam folder, as SpamAssassin has a rule ...

    Thanks for the heads-up. There was a very important e-mail from the Internet Lottery people telling me my e-mail address had been picked as the winner of the EUR 20,000 prize. All I have to do is send them $200 by Western Union to cover the processing fees. And to think I almost missed it!

    It's terrible that SpamAssassin flags such important messages as spam.

    • by cdfh (1323079)

      I hate it when I make a joke and I get modded "+5 insightful". Mod the stupid comments "funny", not "insightful", pleas

      Moderators, please note: he's not being funny --- he's being insightful [xkcd.com].

  • Only a small fraction of email I get is legit, so if it dumps all messages into the spam folder, it is pretty much doing the right thing

  • by doshea (1711648) on Saturday January 02, 2010 @04:28AM (#30620564)

    Clearly we dropped the ball on this one. As far as I know it's our first big rule screw up in the project's 10 years. If you're going to screw up you might as well do it well.

    I posted the following note to the Apache SpamAssassin website (http://spamassassin.apache.org/). Updates are available via sa-update, please run sa-update immediately. It's included in all versions of 3.2.x (the affected version of SpamAssassin). Alternatively zero the rule's score in your local.cf file if you have access to it. If you don't, increase your spam threshold by 3.6 points if your mail provider allows you to do that.

    Y2K10 Rule Bug - Update Your Rules Now!

    2010-01-01:

    Versions of the FH_DATE_PAST_20XX rule released with versions of Apache SpamAssassin 3.2.0 thru 3.2.5 will trigger on most mail with a Date header that includes the year 2010 or later. The rule will add a score of up to 3.6 towards the spam classification of all email. You should take corrective action immediately; there are two easy ways to correct the problem:

    * If your system is configured to use sa-update run sa-update now. An update is available that will correct the rule. No further action is necessary (other than restarting spamd or any service that uses SpamAssassin directly).

    * Add "score FH_DATE_PAST_20XX 0" without the quotes to the end of your local.cf file to disable the rule.

    If you require help updating your rules to correct this issue you are encouraged to ask for assistance on the Apache SpamAssassin Users' list. Users' mailing list info is here.

    On behalf of the Apache SpamAssassin project I apologize for this error and the grief it may have caused you.

    Regards,

    Daryl C. W. O'Shea

    VP, Apache SpamAssassin

    • Re: (Score:3, Interesting)

      by Antiocheian (859870)

      Thanks. SpamAssassin has saved a lot of time on my life.

      I was using Spamcop and got a bit zealous on reporting so spammers flooded my email address out of vengeance. It was then when I learned that reporting spam to admins only works in a few cases while the majority of spam is sent either through botnets or from countries with relaxed or no spam laws.

      SpamAssassin removed 99% of the spam without ever giving me problems of blocking wanted messages...

      Respectfully, thanks.

    • Working great over here - all I had to do was run sa-update.

      Thanks for the hard work!

  • It is 2100... even if you don't like how 2k01 sounds.
  • This rule is a little bit more fine-grained. It will not allow anything to be sent after 2014.

    edit the file /usr/share/spamassassin/72_active.cf

    ##{ FH_DATE_PAST_20XX
    header FH_DATE_PAST_20XX Date =~ /20[1-9][4-9]/ [if-unset: 2006]
    describe FH_DATE_PAST_20XX The date is grossly in the future.
    ##} FH_DATE_PAST_20XX

    I wish variables could be used in regexps like these, like $year + 2 would be a nice start....

    • by doshea (1711648)

      I recommend against the above suggestion for a number of reasons:

      - /usr/share/spamassassin is not the place to edit your rules. Edit your local.cf file.

      - We will be dropping this rule or changing it to a non-regex rule that actually takes the current date into account. We'll do whichever our statistical data suggests would be most beneficial. Once we've done this, we'll push another update via sa-update. I wouldn't risk forgetting that you've modified a rule that's going to hit you again down the road.

      B

      • by freaker_TuC (7632)

        Actually a better rule taking that in account is:

        ##{ FH_DATE_PAST_20XX
        header FH_DATE_PAST_20XX Date =~ /20(1[4-9]|[2-9][0-9])/ [if-unset: 2006]
        describe FH_DATE_PAST_20XX The date is grossly in the future.
        ##} FH_DATE_PAST_20XX

        I've made a post entry about this on my blog at http://development.artistblog.me/2010/01/spamassassin-2010-bugfix/ [artistblog.me] ; although I've heard cases where sa-update didn't work, so I wouldn't know which way would be the best since there are more ways to skin a cat ...

  • Thanks for the timely tip (no pun intended) :-)

Your own mileage may vary.

Working...