Forty Percent of All Email is Spam

PCOL writes "There's an interesting article on spam in today's Washington Post which includes an inside look at AOL's spam control center in Northern Virginia. The story reports that roughly 40 percent of all e-mail traffic in the US is now spam, up from 8 percent in late 2001 and nearly doubling in the past six months; that AOL's spam filters now block 1 billion messages a day; and that spam will cost U.S. organizations more than $10 billion this year from lost productivity and the equipment, software and manpower needed to combat the problem."

Speaking from Experience

[DLG]

In the past 2 months, using a combination of tools including SpamAssassin, I have managed to block approximately 32000 spam mail a week. This is more than 50% of our incoming mail.

I will note that in general this is only coming to around 20% of our users. It is approximately 100 messages per user per day. This actually seems reasonable compared to one of my email accounts that is on a webpage.

So I would say the only reason the amount of spam is so low is that enough people in our firm don't give out their firm email addresses on the internet to strangers.

Although they do miss out on alot of great offers for Hovercraft Toys.

Spammunition

[BlackjackGuy]

My spam problems have almost entirely gone away since installing Spammunition [upserve.com]. It's a bayesian filter for MS Outlook. Wish I didn't have to use MS Outlook but it's a requirement at work.

Bayesian filters are definitely the way to go. They flat-out *work*. Other programs I've used just didn't perform, like Cloudmark Spamnet.

Not ambiguous.

[Anonymous Coward]

I don't have any problem determining what spam is and what it isn't. Why would there be any ambiguity?

Re:Take this with a grain of salt

[Zathrus]

The srticle states that 40% of Internet traffic is Spam

No, the article states that 40% of email is spam.

Which, frankly, seems low. But perhaps they're including corporate email, which often sees a much lower spam level.

I'm still trying to find estimates on how much of all Internet traffic is from SMTP -- I've seen estimates of anything from 5% to 30%.

White list with pass code

[Continental Drift]

My Eudora filters allow me to auto-reply to mail coming from someone not already in my address book. The auto-reply tells the writer to try again and put a code word in the subject line, which the filters will then bypass. This is very effective, and since I implemented it, I don't see spam. It is a bit of a pain for people writing to me the first time.

Now, a white list like this can be bypassed by a spammer claiming to be a friend of mine. It can't claim to be me, because my filters automatically delete anything sent to my address claiming to come from me. I'm wondering if anyone else who has implemented a white list for themselves has seen any problems with it.

Re:I believe it.

[chef_raekwon]

like anyone'e opinion/ideas on what may be done about the spam issue besides filters.

all i did was register a new domain, run smtp/sendmail/squirrelmail from home (dsl connection). this really is a $40 solution, provided you already have the hardware (you have to pay for the domain).

Make sure you don't give out your address too much, and spam becomes non-existent. if, and when you start receiving spam, turn on spam filters (they come with squirrelmail). if this fails, just change your email address, cause damn, you're running the server!

Re:What say you "just hit delete" crowd?

[Azghoul]

I wouldn't support legislation. Ever.

Of course, the hue and cry of the masses will eventually bury any other viewpoint.

I currently have four email accounts.

1 is my work email, only messages to and from people I work with. I have never received a spam to that account.

1 is an old work account that I still occasionally use. No Spam received for 2 years. Then I accidentally put it in when I registered a domain with those fucks at Verisign (sorry for the french). Now I get about 20 spam per day.

1 is a throwaway Netscape.net free account: Sign up for all web forms, stupid shit with this one. Gets mostly spam, but I don't care.

1 is a private family account that only a few people know. No spam there.

There's a solution, it's in using email intelligently. But like I say, the great unwashed AOL users will whine until their gov't wastes more of my tax money.

Sliding scale

[phorm]

I think this could almost be measured on a sliding scale based on lifetime of an account. Once a user opens a new account - unless the email address is easily guessable or his email provided sells it off - spam volume per real email will be low.
Then, you get a few friends your email. General email volume increases. You sign up for some server or other and forget to use a protect email... spam starts to drip in.
A little while later, the drip becomes a trickle as your email gets sold again, and again, and spreads like splitting amoebas.
Then... a few friends send you e-cards around Christmas, or invite you to some joke sites etc. Not your really gonna get it (I strongly b*tch-out any who e-card me at my work address).

To top it off, a LUG or whatever you are posting to puts their history on a public website... you start getting picked up by spam-spiders.

So over time, one will go from maybe 0-5% spam, to 50+% spam. As more people get you in their address books, the more likely it is that somebody will let your email slip to a spam-source. And spam-sources sell your email to other spam-sources... it spreads like wildfire.

The best way to protect yourself is to use a difficult-to-guess, 9+ character email, for which you never sign up for anything with, and only give to people you trust not to e-card you or have "sniffers" installed on their system which gives away the address book. Using bounce addresses might help also, as you could then switch bounces but still pull from the main email, and then filter the ones that get messy or drop them.

It's not just quantity but SIZE

[magarity]

Spam is not just a problem of numbers of emails, but also how big the darn things are. My filter's stats so far for this month reveal that while spam is barely over half of the quantity of mail I get but is over FOUR TIMES the size of real email:

Total Volume Sent on as Clean Mail: 211 (342.3KB ) 44.8%
Total Spam Messages: 260 (1.4MB ) 55.2%

This is the most important evil of the spam flood; not only do I not want it but it's huge!

Re:Optimistic

[kring]

I run a small site (~100 users) and our spam filter, which is designed to be relatively forgiving, catches about 35% of the total messages that are handled by our mail server. 40% seems pretty low to me.

The Spam Solution: Re-Costshifting

[Dion]

The base problem with spam is that it shifts the cost to the victim, the only technical solution is to shift that cost back to the sender so all (or most) costs are transfered to the sender of the mail rather than letting the receiver bear the cost of storage

An exelent proposal is IM2000 [manxome.org].

Re:I thought about it, and you know what?

[dubl-u]

Corporate speech and individual speech are equally protected under the First Amendment.

Wrong [abuse.net].

Re:Spam is like TV advertising

[magarity]

Umm, televison advertisements subsidize television programming. Junk mail subsidizes postage. Newspaper ads, radios ads, magazine ads, etc, etc do the same for their respective mediums. How does spam help pay for my internet connection? ABSOLUTELY NOT AT ALL. All it does is increase my ISP's costs on behalf of a freeloading spammer.

40% is an understatement

[Burdell]

I just installed an upgraded spam filter server at the ISP I work for, and we are now filtering out almost 70% of inbound mail as spam (with basically zero false positive complaints). We combine Brightmail [brightmail.com] with the three main MAPS [mail-abuse.org] lists (RBL, DUL, and RSS), as well as the basic DNS based checks (for valid domains, etc.) built into the mail server, with Brightmail catching the most by far.

You can see our mail stats here [hiwaay.net].

Re:A 3 Point Program to Eliminate Spam Completely

[Cheeze]

1. what happens with Mr. DumbGuy sets up a proxy on his dialup account, and then doesn't take the necessary steps to secure it? That would technically not be the ISP's mail server, but much more spam comes from these types of instances that large mail servers being used for outgoing spam.

2. if you "legally" require software to contain certain settings, and that software is open source, it would be pretty easy to get around any settings that are "legally" put in place. This is called tarpitting, and is already used on many mail servers, but there is no reason to make it a law.

3. what happens when yahoo.com or aol.com get on that list. What, you think all spam comes from an end user?

Your 3 point program has lots of holes. One of the biggest holes is the fact that most of the spam comes from sources outside the US. Brazil, Japan, Taiwan, Singapore, Russia ,etc all send more spam than open proxies in the US. Your 3 point program would not address anything outside the US. When you have laws that force their ideas upon a part of the internet, all of the stuff you were trying to get rid of in the first place will just move outside of the US's jurisdiction.

Comment removed

[account_deleted]

Comment removed based on user account deletion

100%-ish effective spam-prevention technique

[UberQwerty]

I have a real, useable e-mail account that never recieves any spam at all, and I never delete/filter legitimate mail! How is this possible?

I have two e-mail addresses. One gets nothing but spam, and the other gets no spam at all.

I have a free account at hotmail.com and a private one on a server that isn't owned by a big business. When I'm giving my address to someone I know personally, I give the private one. When I have to give an e-mail address to sign up for some service or to get some account, or basically whenever I'm giving my e-mail address but I don't know who is getting it, I give my hotmail account.

Result:
-My hotmail account occasionally gets confirmation e-mails when I've just created one of those free accounts for some website, but I always know when they're coming. Otherwise, it just collects spam, which I periodically delete (and block the addresses it came from).
-My personal account never gets spam.

(I have a university account that forwards to my private account, so occasionally it gets what could be called "spam" that's aimed at univ. students, but if I stop the forwarding it stops the spam, so I don't really have a problem.)

Re:Accuracy

[wawannem]

You know... You could fix this kind of situation yourself. If you set up a real DNS zone, AOL would have no way of knowing you aren't running a legitimate mailserver. Shell out a few bucks to get a name, then spend a day or two figuring out BIND (or worse WinNT DNS), then viola! You will be doing it correctly!! And who would have thought, when you do it right, ISPs will honor it!

Re:Good percentage

[badfinch]

According to a site that keeps stats live for their filter for all mail proccessed 50.7% are detected spam from bulk senders. The site is http://www.herbivore.us

Disposable Email Addresses -- Effective?

[angle_slam]

Does anyone here use a Disposable email address service? Examples of such services include the following:

• Spam Gourmet [spamgourmet.com]
• Spamex [spamex.com]
• Sneakemail [sneakemail.com]
• Mailsehll [mailshell.com]
• Emailias [emailias.com]

General information about disposable email addresses can be found in this PC Magazine article [pcmag.com] and this about.com article [about.com].

Briefly, I'll explain how they work in theory. After signing up with a disposable email service, they give you a disposable email address that you can, for example, enter into forms. Mail sent to that disposable email address gets automatically forwarded to your email account of choice. But here's where they supposedly come in handy. You can sign up for a different disposable email address everytime you fill in a web form. If you start getting spam, you can look at the disposable email address the spam was sent to and you can do 2 things: (1) cancel the disposable email address so you no longer get spam sent to that address; and (2) you know who gave out your disposable address and you can take whatever action you deem appropriate.

This seems like a cool product, in theory, but I haven't seen anyone with real world experience with these services. If anyone here can describe their experiences, it would be greatly appreciated.

Re:Take this with a grain of salt

[Captain Beefheart]

"But perhaps they're including corporate email, which often sees a much lower spam level." ...Except that drumming up a corporate e-mail address is usually as simple as adding the first letter of the first name to the last name, as in [e-mail] or [e-mail]. I've gotten several spams to a relatively high-profile domain, the specific address of which had not been used externally, had not been in someone else's CC field externally, and had only existed for a few days before the spam started trickling in.

Re:Losing a figurative war on spam

[Anonymous Coward]

You'd probably set up a negotiation for with the assurance of a refund if it is not spam. So, my mail server may say, e.g. ``I haven't seen you before, your message wasn't signed by someone that I trust or your message doesn't come from a mailserver that I trust, so if you want this email to go through you must give me $5 which I shall return to you if I determine the email is not spam.''

Interesting points here:

1. the amount is larger than most proposals that I've seen. This is necessary since I get quite a bit of spam in my US mailbox and that costs $0.10. The amount should be enough that people will think about it quite a bit. The amount should also reflect how much my annoyance at receiving the spam is worth.
2. The assurance the the deposit will be returned if the email is deemed legit. You'd want MUA support for this one.
3. The lack of charge to mail that you are expecting in some way.
4. The other person gets a chance to deny the negotiation. So the developer simply won't pay and if you require it you won't get the email.

All that said, I don't really like the idea. Decent filtering is good enough.

Of course, think of the money making opportunity when a spammer writes software that screws up the negotiation! A simple mistake like:

if (amount < 100)

rather than

if (amount < 1.00)

could make you a hundred bucks. Or something like that. Then we could have the reverse wars where the anti-spam people try to write software that negotiates in such a way that it confuses spamware into giving them lots of money! woohoo!

Rackspace

[Skapare]

It just seems to odd to refresh the page to see more comments about spam, and I get a banner ad promoting one of the larger spammer hosters in the US ... Rackspace. Those who sign up for service from those scumbags are just as bad as the scumbags because that effectively helps support the spam they keep pounding my servers with. So far today, 98 attempts just from Rackspace addresses. Yesterday there was a total of 240.

And while previewing this comment submission, yet another Rackspace banner ad. Don't these guys know I'm never, ever, going to pay them for any services?

Perhaps AOL email is that bad,

[Lord Kestrel]

but inside corporations, it's more like 98% real email, and 2% jokes/spam/pr0n/whatever. Speaking from my experience (I receive upwards of 600 internal emails a day), almost all of it is work related. Email from the Internet isn't all non-spam, but spam is still only 2-4% of the email I receive.

Re:Spam Control

[letxa2000]

I haven't seen forged headers used extensively for some time. The only thing I really see being forced is the "From" address and the Reply-To address, along with the "HELO" command in the SMTP exchange. But forging "Received" headers seems to have become less frequent.

I think that's because spam is, by nature, evolutionary. What works for now is quickly picked up on and then they have to move on to something else. The only people really interested in "Received" headers are syadmin type people that are going to be able to recognize forgeries anyway so they don't gain anything by doing it.

What blows me away is how many are spamming directly from their DSL connections these days. They just don't care and apparently the DSL providers just don't do anything about it. I can see throw-away dial-ups being used to spam, but I find it amazing that someone would risk a DSL connection to spam. The fact that they DO risk their DSL connection suggests to me that it isn't really much of a risk. :(

I also think the anti-spam approach has come down more to filtering and looking for a new protocol than reporting spammers. While some spam reports actually result in action, most don't--and those that do you are seldom informed of that so it seems that you are making spam reports that go into a blackhole. I gave up on reporting spammers two years ago--except for extreme cases that border on DOS attacks.