ClamAV Forced Upgrade Breaks Email Servers

An anonymous reader writes "A couple of weeks ago Sourcefire announced end-of-life for version 0.94 of its free ClamAV antivirus package (and in fact has been talking about it for six months). The method that Sourcefire chose to retire 0.94 was to shut down the server that provided its service. Those who had failed to upgrade are scrambling now. Many systems have no choice but to disable virus checking in order to continue to process email. I am very glad I saw the announcement last week!"

Got This Bounce This Morning

[WrongSizeGlass]

Diagnostic-Code: smtp;
451-4.5.0 Error in processing, id=02792-02, virus_scan FAILED: virus_scan: ALL VIRUS SCANNERS FAILED: ClamAV-clamd av-scanner FAILED: CODE(0x83d7540) Too many retries to talk to /var/spool/amavisd/clamd.sock (Can't connect to UNIX socket /var/spool/amavisd/clamd.sock: No such file or directory) at (eval 55) line 310.

ClamAV-clamscan av-scanner FAILED: /usr/bin/clamscan unexpected exit 50, output="LibClamAV Error: cli_hex2str(): Malformed hexstring: This ClamAV version has reached End of Life! Please upgrade to version 0.95 or later.

At least their error messages are descriptive and informative.

Re:*Correction*

[WrongSizeGlass]

From the link:

Starting from 15 April 2010 our CVD will contain a special signature which disables all clamd installations older than 0.95 – that is to say older than 1 year.

[snip]

We recommend that you always run the latest version of ClamAV to get optimal protection, reliability and performance.

Thanks for your cooperation!

FYI, ClamAV, DOA != cooperation.

EOL annountment from Oct 2009

[Anonymous Coward]

End of Life Announcement: ClamAV 0.94.x
Oct 5, 2009

All ClamAV releases older than 0.95 are affected by a bug in freshclam which prevents incremental updates from working with signatures longer than 980 bytes.
You can find more details on this issue on our bugzilla (see bug #1395)

This move is needed to push more people to upgrade to 0.95 .
We would like to keep on supporting all old versions of our engine, but unfortunately this is no longer possible without causing a disservice to people running a recent release of ClamAV.
The traffic generated by a full CVD download, as opposed to an incremental update, cannot be sustained by our mirrors.

We plan to start releasing signatures which exceed the 980 bytes limit on May 2010.

We recommend that you always run the latest version of ClamAV to get optimal protection, reliability and performance.

Thanks for your cooperation!

Re:[clamav-announce]

[entrigant]

announce lists are intentionally very low traffic. I'm subscribed to over 50, and I rarely receive more than 4 or 5 mails a week at most.

Re:so clam breaks if a remote server is down?

[mysidia]

It wasn't the server going away. They delivered an update designed to kill it

The Windows equivalent would be Microsoft Delivering a critical update with XP designed to disable windows, because you haven't updated to Vista yet.

In other words, they used the automatic update service against their own users.

From now on, my recommended course of action is that all mail administrators running clamav should REMOVE or DISABLE any automatic updates of ClamAV rules, make sure to comment out any crontab entries for freshclam.

Until the developers can either grow up and stop doing stupid shit such as abusing auto-updates to disable their own product.

Or do what they should do... include a method for automatically applying version updates.

Or force auto version update instead of disabling.

Re:Alternative

[compro01]

It's quite a bit more extreme than just shutting down one of their servers. They issued a final "signature" update that literally caused each installation of that version to stop functioning.

From the announcement [clamav.net] :

Starting from 15 April 2010 our CVD will contain a special signature which disables all clamd installations older than 0.95 - that is to say older than 1 year.

Re:No fallback ?

[1s44c]

I had two mail servers, on two Internet connections. If either went down I'd get an alert and could fix it without mail being affected. I didn't expect both to stop processing mail at the same time. It's always the stuff you don't expect to fail that fails.

My mail was queued on DMZ mailers so nothing was lost, but it was delayed. Some of it may have been business critical.

Debian Debs Outdated

[TypoNAM]

I just tried to update:

# cat /etc/debian_version
5.0.4

aptitude output during update:

Setting up clamav-daemon (0.94.dfsg.2-1lenny2) ...
Starting ClamAV daemon: clamd LibClamAV Warning:
LibClamAV Warning: *** This version of the ClamAV engine is outdated. ***
LibClamAV Warning: *** DON'T PANIC! Read http://www.clamav.net/support/faq ***
LibClamAV Warning:
LibClamAV Error: cli_hex2str(): Malformed hexstring: This ClamAV version has reached End of Life! Please upgrade to version 0.95 or later. For more information see www.clamav.net/eol-clamav-094 and www.clamav.net/download (length: 169)
LibClamAV Error: Problem parsing database at line 742
LibClamAV Error: Can't load daily.ndb: Malformed database
LibClamAV Error: cli_tgzload: Can't load daily.ndb
LibClamAV Error: Can't load /var/lib/clamav/daily.cld: Malformed database
ERROR: Malformed database

It appears debian repositories also need to be updated. :(

NOTE: I removed the * (star) chars from the warnings due to junk filter.

Re:Hm...

[Anonymous Coward]

IIRC, ClamAV doesn't have real-time scanning anyway. Does it have a first party mail server scanning plugin now, or am I totally misunderstanding the issue here.

yes it does and has had it for a while

[me@server clamav-0.96] ./configure --enable-milter

works with sendmail and postfix

Overconfidence

[gmuslera]

A lot of server stuff in linux work so well that you can even forget that it is running at all, for years. Clamav is such kind of software, you install/configure it, set the automatic signature updates, and forget that it is there. But still, some periodic checks in logs that all are working as expected is good, even if is just some artificial ignorance [ranum.com] well applied, specially when clamav started warning on this months ago.

Re:*Correction*

[GungaDan]

Definitions were upgraded, though, weren't they? Just the engine was a year old...

Re:*Correction*

[compro01]

The definitions were up to date (but would become out of date when they started pushing large (>980 bytes) definition updates next month, which the old version cannot handle), but the version was not.

Misleading, yes?

[thePowerOfGrayskull]

"ClamAV forced upgrade breaks email servers" should read "Failure to upgrade despite six months warning breaks email servers" or "Inattentive server admins cause massive downtime".

Re:Debian Debs Outdated

[iYk6]

The ClamAV package in Debian Lenny-Volatile is 0.95.3. You're using the package from Debian Lenny, which is stable, and doesn't mesh well with ClamAV, which is either the latest and greatest or broken.

Debian Volatile is meant specifically for this kind of thing.

Re:Alternative

[b0bby]

Honestly, for things like this that I don't have the time to do right I prefer to let someone else do them. In this case, why not route your mail through Postini or another service? I'm pretty sure that I can't hope to do a better job filtering than Google...

Re:Alternative

[CoolQ]

Uh, it HAS been filling your log files with warnings about upgrading for months, if not years. It's pretty f'ing explicit:

LibClamAV Warning: *** This version of the ClamAV engine is outdated. ***
LibClamAV Warning: *** DON'T PANIC! Read http://www.clamav.net/support/faq [clamav.net] ***

--Quentin

Re:Alternative

[Anonymous Coward]

Postini is cheap and works.

Volatile

[XanC]

You really should use the volatile repository. It provides updated versions of packages that are required to change (like antivirus), compiled for stable. You end up with stable + required updates.

Re:So you had 6 months to upgrade

[gparent]

http://lists.clamav.net/mailman/listinfo/clamav-announce [clamav.net]

Re:So you had 6 months to upgrade

[masdog]

I had modded this overrated, but this really deserves a reply.

You're in the wrong place if you expect sympathy. There are a lot of other sysadmins here. There are a lot who wear all of the hats. You're not alone.

You had a poorly designed or poorly implemented mail system. That isn't clamAV's fault. It's not their fault that you didn't upgrade or check your system logs. This is no different than forgetting to pay the maintenance bill on a commercial mail gateway or hosted solution. Would you blame Symantec, McAfee, Microsoft, or CA if you didn't pay the bill and your mail stopped flowing?

The fact that you didn't follow a blog or mailing list about a critical piece of your infrastructure says a lot about you as a sysadmin. They're even on Facebook and Twitter. If you can't take the time to keep an eye on your mail gateway or antivirus product, what else aren't you keeping up on. Think about that for a few minutes, set up a Google reader account, and then start subscribing to blogs. If you have a smartphone, add Google reader to your RSS Reader. It makes good bathroom reading.

Re:Alternative

[jim_v2000]

Yes. Especially when there was six months warning that it was going to happen.

Also, I'd rather it stop working then keep working and not get definition updates.

Re:Alternative

[SunFireSpaz]

Join their announce mailing list at http://lists.clamav.net/mailman/listinfo/clamav-announce [clamav.net] and you will be notified about these type of things.

Re:So you had 6 months to upgrade

[Anonymous Coward]

I'd also like to add that there is no excuse to ignore log from any production sistem. It's extremely quick and easy to filter out all the "normal operational" messages, so that everything that goes in the log is a problem to be dealt immediately or a new rule for the normal messages filter.