DSPAM v2.10 Released 234
Nuclear Elephant writes "DSPAM v2.10 is finally available, after four months of development. This is the first stable release to include Bayesian Noise Reduction which was recently mentioned on Slashdot and in Wired News as an algorithm providing accuracy levels as high as 10x that of a human. Some other new features include Neural Networking - which finds nodes in a network that are contextually similar to form a decision matrix, Global Filtering - which provides SpamAssassin-like out-of-the-box type filtering for new users until they build up their own wordlist, Automatic Whitelisting - which automatically learns who your trusted senders are, and many other optimizations and enhancements. Head on over and download the latest tar ball."
Cool! (Score:5, Funny)
Re:Cool! (Score:5, Informative)
Re:Cool! (Score:2)
Problem is ... (Score:2)
the "10x better" means 10x lower failure rate. The wording almost seems meant to deceive. The idea is that if you misidentify 10 messages out of 100, the filter would only misidentify 1. Since you made 10x as many mistakes, the filter was 10x as accurate as you were.
The problem with that is that "Spam" is defined by humans and not computers. "Anti-Spam" software is programmed to *try to* filter out what the human would consider Spam..
So if a human says Email X is Spam, then it is that human's Spam.
Re:Problem is ... (Score:4, Interesting)
That is true if the human is looking at a single email. Now give the same human a mailbox with 2000 messages, 1000 of which are spam (by his standards). He won't be thinking twice about calling the message spam and getting rid of it, so he's bound to makea couple of mistakes (happend to me a while ago, one of my friends has her email @ladymail.com and the Subject was in Latin - random to me. I called it spam befere even reading Hello,...).
The claim that is being made is that if this poor man overlooks 10 spam emails, dspam will only overlook one. Whether that's true or not is another thing, and would again depend on the circumstances, but I believe it would apply to me.
Man is the measure of all spam (Score:2)
If I say it's SPAM, it's SPAM. If I say it's not SPAM, it's not SPAM. No filter can possibly be better than I am, and I don't want any filtering software claiming that it knows better than I. A personal message from a friend is still a personal message from a friend even if the subject line is "Hi" or "I love you."
The real problem (Score:4, Insightful)
I propose we start spamming. Anyone who responds gets a nice l'il pistol whipping and is returned to their comptuer. After the first news report, people will be afraid to respond to spam.
Re:The real problem (Score:5, Funny)
Re:The real problem (Score:3, Funny)
The German government is cracking down on people like you.
Heh, I had the same idea... (Score:2)
But yeah, that would probably kill the spam market pretty well.
Re:Heh, I had the same idea... (Score:2, Funny)
Re:The real problem (Score:5, Insightful)
Spam works on the level of 1 in 10,000. The general population contains a far higher rate of mental illness, senility, and retardation.
You'll never cure spam by 'education' of any sort. There are some people who are just too crazy or too stupid to learn.
Re:The real problem (Score:5, Funny)
Re:The real problem (Score:2)
Therefore, we can make responding to spam a way that society decides who is an idiot!
Responding to spam should therefore be punishable by the revocation of the following privileges, which are given specifically because they assume that the average person is NOT an idiot:
-use of computer networks without direct supervision of someone with an idiot-supervision license.
-ability to vote
-ability to work for the public in which responsibilty for others is given to you (so you could be a janito
Re:The real problem (Score:2)
If *all* of us answered our spam, it would be like a mailbomb or DDOS attack, except they *asked* us to respond. Can they blame us, if we do?
If there's a secondary incovenience, like the fact that they can't find their one sucker in 10,000, well too bad for them. Maybe they should have worked harder targeting their spam to suckers instead of getting past all of our spam filters.
Actually, we don't *all* have to respond to spam. I'll bet if even 1% of us did, they'd be burie
Re:The real problem (Score:2, Interesting)
Make another email worm like MyDoom(call it MyDick/MyAss etc), with misleading title/body that sounds like those spam mail that enlarge/shrinks various human anatomy.
People who reply those mails will be activated the virus and make his/her computer unuable. Soon nobody will have the gut to open spam mail anymore.
Re:The real problem (Score:4, Insightful)
Re:The real problem (Score:2, Interesting)
Details. (Score:5, Informative)
DSPAM (as in De-Spam) is an extremely scalable, open-source statistical-algorithmic hybrid anti-spam filter. A majority of users running v2.10+ achieve filtering rates ranging from 99.92% - 99.98+%, DSPAM is currently effective as both a server-side agent for UNIX email servers and a developer's library for mail clients, other anti-spam tools, and similar projects requiring drop-in spam filtering. DSPAM has been implemented on many large and small scale systems with the largest systems being reported at about 125,000 mailboxes.
What is a Statistical-Algorithmic Hybrid Filter?
Present-day language classifiers bear the responsibility of maintaining accuracy in the midst of ever-increasing sample complexity. In the setting of spam filtering, many types of intentional attacks have been introduced such as obfuscation, word list injection, sample flooding, and etcetera. As the complexity of classification text continues to multiply rapidly, many filter developers today are left with conflicted feelings between increasing the complexity of their filter and wise teachings from CS class reminding them that computer science is about controlling complexity, not creating it. At the rate complexity is rising, filters will (and have already begun to) become so resource-intensive that they lose scalability, eventually leading to a second conflict of interests: where fighting spam becomes more expensive than managing it.
DSPAM is the first Statistical-Algorithmic Hybrid filter and in being such boldly suggests that there is a better alternative to increasing the feature set of filters to match the spams they are trying to fight. By employing algorithms designed to increase the quality of existing data rather than the quantity of data with the goal of reducing the feature set rather than increasing it, DSPAM has managed to achieve nearly equal levels of accuracy with present-day Markovian-based filters and other types of filters that employ large feature sets with the added benefit of using a significantly fewer amount of resources. DSPAM presently peaks at 99.984% accuracy, which is ten times more accurate than a human being [1] and is presently being used on implementations as large as 125,000+ mailboxes.
DSPAM's Focus
The DSPAM project attempts to go beyond "just another statistical filter" by focusing on the following areas:
* DSPAM has a strong focus on providing better data to already existing algorithms (Bayesian, Chi-Square, etcetera) Combination algorithms work inherently well, but depend on the quality of data. Some of the approaches deployed in DSPAM towards this goal include Chained Tokens, Inoculation Groups, Classification Groups, advanced de-obfuscation techniques, and a new noise reduction algorithm called Bayesian Noise Reduction. The goal is to incorporate processing algorithms that can withstand the long haul of ever increasing message complexity. So far we're doing a great job.
* A strong focus on large-scale implementation support. The largest implementation of DSPAM we've heard about to-date involves 125,000 users. DSPAM has been designed to experience a very short execution time (0.03s - 0.10s on average hardware), and has been equipped with a storage driver API allowing several different storage mechanisms to be used. Depending on disk space constraints, accuracy can be traded off for additional disk space or vice-versa.
* Empty Corpus Support and Global Dictionary Support. It is very important in a large-scale environment to allow users to build their own dictionaries starting from scratch. Why? Because system administrators haven't got the time to create 20,000 seeded dictionaries. On top of this, ISPs require out-of-the-box filtering, which DSPAM's global dictionary feature provides for end-users, with minimal centralized learning. DSPAM provides support for building corpuses from scratch without suffering many fatal training errors (false positives). When these two approaches are combined, we end up with instant-filtering for all u
Re:Details. (Score:3, Insightful)
I'm assuming you are linked to the project, forgive me for the rant if thats incorrect.
Might I suggest you get a webserver/ISP that is somewhat reliable. I've been trying to get a copy of this software since it was alst mentioned on Slashdot. The site was slashdotted when I first tried, cool I thought, I'll check again tomorrow. Still down the next day, OK I think maybe there's still an effect. I wait a week and check again thinking maybe they went over their
cool (Score:3, Interesting)
Re:cool (Score:2)
I wonder if this will catch what Mozilla misses (Score:4, Informative)
Re:I wonder if this will catch what Mozilla misses (Score:5, Informative)
Re:I wonder if this will catch what Mozilla misses (Score:2)
Ray
Re:I wonder if this will catch what Mozilla misses (Score:3, Interesting)
The bugzilla number for this feature evades me at the moment. I've only used the windows builds provided, but it shouldn't be too difficult to make your own linux build with this stuff turned on.
funny faq (Score:5, Funny)
In real-world scenarios, false positives have ranged anywhere from 0% (none) to 0.10% depending on both implementation and user's mail behavior. Users with relatively predictable mail behavior (such as geeks, dweebs, and freaks) have generally received very few false positives (less than 1 in 10,000 messages).
Re:funny faq (Score:5, Funny)
What about losers, dorks, and morons? Are they cursed with a high rate of false positives?
I still prefer tougher email security (Score:4, Insightful)
I say forget the filtering shit and force email to evolve. Part of the reason that spam happens is that there is no real authentication going on. No requesting permission to be on your white list. No real strong way to block anybody you don't want to hear from. No real way to verify the sender is legit. etc.
I don't claim to have all the answers, but I do know that I've been using ICQ for years and haven't seen a Spam from there since I turned on the 'require authorization' feature.
Re:I still prefer tougher email security (Score:2, Interesting)
I don't have any clue what the solution to the spam email problem is but I believe it'd have to be a pretty major evolution.
Re:I still prefer tougher email security (Score:4, Informative)
Re:I still prefer tougher email security (Score:5, Interesting)
The authentication is useless even if implemented - you want to receive email from strangers, that's what all businesses are doing. If you are not one of them and only converse with your buddies, make a whitelist and be done - no spammer will guess your friends' emails.
Permissions to send email are also troublesome. If they are automated, then spam robots will be written to ask for permission first. If they are not automated... but how would you know if some random "John X. Frisby" <jfrisby@big.provider.net> is really who he is, and the matter he wants to discuss with you is not a bug in your Loafizer 0.99 script for your bread making machine, but a placebo enlargement pill. Additionally, permissions delay the mail exchange, which is bad for business.
There are ways to block anyone you don't want, and all other senders are legit (until they spam you, that is.)
So the problem is quite different, as you can see. There is a free channel of marketing, and spammers will be using it until it remains a) free and b) channel. Remove any one of those two, and they will close up the shop.
Re:I still prefer tougher email security (Score:3, Interesting)
Call me a cynic, but I think we're dealing with an inherantly unsecure system. As long as you have one mail server out there forging message headers, you can't trust the path back to the sender. Like abstinance, Whitelisting m
CRM114 Discriminator works better for me (Score:5, Interesting)
I eventually gave up and tried the CRM114 Discriminator:
http://crm114.sourceforge.net/
It was MUCH easier to install, MUCH easier to maintain, and has the same or better level of accuracy. I used to get 100+ spam messages a day and now I'll get maybe 1 or 2 a week that sneak through (after only a few weeks of training on errors only).
Aha! Ta mate (Score:2)
When you've got several hundred systems from different OS platforms all logging to a central log server the conventional log monitoring software is just not up to the task of discriminating important logged messages from unimportant.
Now, if there was an adaptation for Kmail (Score:4, Insightful)
(since then the 'casual' user could benefit from using it, without undue difficulty in configuration of mail delivery programs, which are notorious in general..)
now only if.. (Score:2, Insightful)
Re:now only if.. (Score:2, Interesting)
Some Bayesian approach ought to do it
Preventing Victims of Spam (Score:4, Funny)
Computer manufacturers are also investigating whether this device will be able to deal with the so-called "Stupid User Problem" which plagues so many IT professionals world wide.
Bayesian Unsupervised Learning (Score:5, Interesting)
Oddly enough, our director of research was notoriously difficult person to schedule a meeting with. Makes me wonder about 'unsupervised learning'...
More accurate than a human? (Score:4, Funny)
So, let me get this straight - my spam filter will know better than I do which emails I want to read, and which ones I don't?
"No, trust me man, you really want a bigger johnson. Read it!"
Re:More accurate than a human? (Score:2, Informative)
Explained in the last DSPAM /. story (Score:5, Insightful)
except that my article history is truncated in a futile attempt to get me to subscribe. So I can't point to the writeup I did.
The increased accuracy comes from the emails that will slip under your mental radar. You are a human, and you make mistakes. You wouldn't deliberately choose to read the email, but one day the subject line looks plausible, and so you bring it up. Three-quarters of a second later, you're glaring at the monitor and hitting "delete", but DSPAM wouldn't have let that slip by in the first place.
Re:Explained in the last DSPAM /. story (Score:2)
D'oh.
Re:More accurate than a human? (Score:3, Insightful)
Yes, it will. When I'm faced with 100 new messages in my inbox and probably only one or two are legitimate, I often delete messages that look like spam without opening them, and other times, I have to open them just to double check that it really is spam. I have accidentally deleted more than one legitimate message this way, and have wasted more time that I care to contemplate openi
Re:More accurate than a human? (Score:3, Informative)
Re:More accurate than a human? (Score:2)
Umm... what's the definition of spam? (Score:5, Interesting)
Re:Umm... what's the definition of spam? (Score:4, Insightful)
Outlook is like what you fear; Microsoft decides what you will and won't see. I can add specific senders to the black and white lists (you click to add to the blacklist, but you have to type in an address to add it to the whitelist -- stupid MS shits), but Microsoft decides if I can see that attachment (if they think it's bad, it's gone and I can't recover it) or if this email's spam (it regularly discarded stuff from IBM Developer Works until I added them to my whitelist). With a tool like dspam I can regain control over what gets filtered (although I've found no way to turn off Outlook's attachment blocking).
Re:Umm... what's the definition of spam? (Score:3, Informative)
Tools, Options, Security, uncheck "Do not Allow attachments to be Opened that cound potentially contain a virus".
Re:Umm... what's the definition of spam? (Score:2)
Re:Umm... what's the definition of spam? (Score:2)
However, a quick googling turns up this knowledge base article [microsoft.com] with the information you desire.
Re:Umm... what's the definition of spam? (Score:2)
Re:Umm... what's the definition of spam? (Score:4, Informative)
Leaving aside the part where you barely avoid the paranoid rantings of a madman, yes, there are times when you can't tell if you're being spammed. Like, how many times have you accidentally deleted an email that you thought was spam but was really from a long-lost friend? Or how many times have you opened Spam because you weren't sure that it was Spam or something from your ISP (or whatever).
Say you've done it 10 times in 10 000 messages. If this program only did it once in 10 000 messages (false positive or missing negative) then it was 10x as accurate as you.
Then we need to define what a "human" is (Score:2)
It seems as though you're defining "human" as "person who is time-constrained". I suspect that the author of the original article had something like this in mind, but without defining the time constraint, a measure like "10x" is meaningless. E.g., as spam cont
Take it one step further; share what you filter (Score:5, Interesting)
DSPAM is one of these statistical filters (like spamprobe and CRM114) that can perform virtually perfect filtering of spam/non-spam you receive.
Now that you are free of spam yourself, may I suggest that you take it one step further and share your data with the anti-spam community; the WPBL project [pc9.org] lets many users report the IPs sending them spam and non-spam in realtime using a couple simple scripts installed in procmail.
Our central database then publishes a real-time list of spam sources (the IP blocklist). Unlike spamcop, WPBL is entirely based upon automatic decisions made by statistical filters, 24/7. The resulting blocklist is already used by many ISPs; and you can also use it to block spamming IPs at your own server.
Re:Take it one step further; share what you filter (Score:3, Interesting)
Re:Take it one step further; share what you filter (Score:2)
DSPAM sounds great... (Score:3, Funny)
Re:DSPAM sounds great... (Score:2)
I want a filter dammit. Server side doesn't cut it (Score:2, Interesting)
I see all my fellow slashdotters saying (over and over again) that spam filters should be server side, because otherwise you are still paying for the wasted bandwidth. This is a very powerful argument, and I tend to agree.
However, there are two things that make
Re:I want a filter dammit. Server side doesn't cut (Score:2)
It is a little thing that sits in your system tray. That said, it's just perl modules (I think) so it runs on other OSes too. That said, best thing I've found on Windows.
Re:I want a filter dammit. Server side doesn't cut (Score:2)
I've looked into popfile. Isn't it pretty much just bayesian filtering (with more than the basic two spam and non-spam corpuses)? Is it better than mozilla mail? Mozilla mail is a hell of a lot better than nothing at all, but my experience has been a lot less perfect than some others have reported. I suspect popfile w
The solution - seriously (Score:2, Interesting)
Every time you send an email you place a small wager on the line that the recipient wants to read your message. Something like 1 cent. If the recipient doesn't mind your message then they don't redeem your offer and it doesn't cost you a thing. However, if you're sending spam then the recipient cashes it in (or perhaps it is used to cover overhead costs of this system).
If you send a legitimate email and somebody decides to be a jerk and cas
Re:The solution - seriously (Score:3, Funny)
Re:Mods! It was FUNNY (Score:2)
Re:The solution - seriously (Score:2)
This "simple yet elegant" layer would require far more work than the underlying SMTP servers do. How exactly -- no handwaving, no fluff -- do you propose to implement this? You need to either tie bank account details to email account information, or maintain a separate "online only" bank. You need to find some unforagable, unbreakable, untappable method of identifying individual emails to make your one penny claim. You need to retrofit all
Daft, on many levels (Score:4, Insightful)
Scumbags would use billions of zombied PCs to send themselves mails, aggregate and pocket the cash. Or to spam you gratis.
There are transaction costs for generating, checking, and accumulating digital cash. Your paypal bills would be huge.
Everybody hates micropayments.
It's a dumb idea and it simply isn't gonna happen.
They suck like vacuum (Score:2)
Bah. Who wants that grief? Charge in bulk and in sensible denominations, or not at all.
Your spam solution could be abused (Score:4, Insightful)
There are several scenarios where your proposal would be bad for the Internet. Say I want to put my competitor out of business, or at least raise his costs. I simply use a bot to sign up for a couple hundred thousand email addresses, sign up for his newsletters, then ask for all those 1 cents back. The financial powers that be might also foresee too much liability and risk in ventures that depend on email (since it is, as you say, gambling). Thus the end of any free service that depends on e-mail for verifying accounts including newsletters, bulletin boards, online banking, and online auctions among others.
Furthermore, you'd have to have a foolproof system to pay for those cents. Fraud could be much more rampant: If you pay via credit card, the other guy (or gal) has your number and could overcharge a corporation by a twenty or so dollars. Furthermore, micropayments aren't economical unless many many many people pay. If most people play by the rules, then the costs of credit companies or banks or other institutions would either put most of these services out-of-business or into subscription only domains. Not to mention some companies might have "you agree not to ask for those cents" in addition to "I can send you spam" legal clauses - negating your proposal!
Re:The solution - seriously (Score:2)
Yeah, I have an idea. Howabout you handle distribution of the NANOG mailing list after your "pay-to-send" idea gets implemented.
Re:The solution - seriously (Score:2)
Problem: Enough people give money to spammers that it becomes possible to profit from abusing the system, tricking or defrauding people.
Solution: Put even more money into the email system, and hope that all of these new flows of cash don't end up in the hands of spammers and criminals.
Here's where "10x as accurate as human" comes from (Score:5, Informative)
If you then check the link to CRM114's project, you'll find this: "I measured my own accuracy to be around 99.84%, by classifying the same set of 3000ish messages twice over a period of about a week, reading each message from the top until I feel "confident" of the message status, (one message per screen unless I want more than one screen to decide on a message.) and doing the classification in small batches with plenty of breaks and other office tasks to avoid fatigue. Then I diff()ed the two passes to generate a result. Assuming I never duplicate the same mistake, I, as an unassisted human, under nearly optimal conditions, am 99.84% accurate.)."
Given the amount of people who even read the article on slashdot I doubt anyone else is going to check the tiny [1] footnote and find this.
99.84% and other myths (Score:2)
If one message out of 3000 messages differs in classification, that's 0.0333%. Or 99.9666% accurate. Working down, we find that four or five misclassifications are either 99.8666% or 99.8333% respectively. B
Hey... you Linux geeks get all the cool toyz! (Score:2, Funny)
You linux geeks get all the good toyz!!
Darn you, Darn you to Redmond!
What do I get?
Well.. I guess I do get all the neat patches.
Put this into Slashcode? heh (Score:5, Insightful)
Bah... (Score:4, Interesting)
No matter what technology it uses, neural nets, b-trees, recursion, tinkertoy logic [rutgers.edu], smell-emitting diode, leaky junction zener transistor, steam-powered aeolipiles, it only automagically presses delete, which is a pretty lame way of fighting spam.
It's a lame way of fighting spam, because, we STILL have to pay for the fucking spam bandwitdh; we STILL have to pay for the goddammed disk space used by the spam; we STILL have to pay for the bloody time lost transmitting the spam; we STILL have to pay for the extra ISP infrastructure to carry those spams.
Naaah. Spammers should be eradicated from the Internet, and the best way to do so is to completely BLOCK networks who host spammers (no matter what service), in order to force the collateral damage to whine to the ISP or simply vote with their feet.
It would be nice if.... (Score:4, Interesting)
CRM114/P.O.E. (Score:3, Interesting)
Re: (Score:2)
Combating SPAM is easy, if you have the technology (Score:5, Interesting)
To mod or to post. Spam is the question. (Score:4, Interesting)
Wait... You'll be interested to know that the biggest problem with the spam coming in comes from virus infected Windows boxes. They send it. They harvest the users Outlook address book. If you ever end up in somebody's Outlook box
I chuckle at the whole Exchange thing. You pay for that?
I personally pay to have a fixed IP @ home and run a old Linux box. A lot of aliases I've used over the years (and some blatantly used to harvest) all go to some local account that processes the spam. Upon receipt -- mail the wrong account and sorry, but you're blocked (unless white-listed). White-listing can come from valid already received email -- but I work everything based off of IP. My hope is that the registered MX host(s) or any valid listed server by the authenticating DNS server will be the type of scheme that's re-implemented (or more to the point SHOE-horned in real soon
Over the last decade I've now got 380 aliased harvesting spam address' in use -- two valid email accounts @ home (my wife and myself) which is on my own IP with my own domain. I pay $5 extra a month above my broadband (10Mbit [yeah, solid] wireless) -- how much do you pay for that Exchange box?
I've run this type of setup through many offices scaled to dozens of email servers -- and the beauty is they also talk to each other sharing block/white-listed address' as needed. Wait -- you will get spam. Filtered through my account to I'm seeing 80 something that got in -- 2,164 blocked IP's [today], 380 harvested address', and 48 for various other infractions (attempts to relay through me, from a country where I know nobody, etc
Statistically (yeah, they all get nmap'd back)? 96% Windows based.
I give my email to friends. I have a work email that anybody that knows how to call me can have it. I even print it on my business card. No, I wouldn't post it to USENET or even here -- but it's still "out there". My unlisted phone number, OTOH, anybody can have. 847.854.0048. It's always busy and one channel of my ISDN home line. The other channel routes to the house for two phone lines (or Internet backup if and as needed) and is automatically unlisted and unpublished (at no cost since it is a "data circuit") -- and no, I'd rather not post that either.
Exchange? Never!
Re:Combating SPAM is easy, if you have the technol (Score:2)
Now I have to actually add any new aliases that I want to use into my user account.
The best solution to this is to use a prefix with an asterisk. I set up david.*@endeavorcomputing.com as one of my addresses. I stuck the applicable site name in place of the * when signing up for accounts. This routed all mail that fits the template to the right address and allows you to create new addresses on the fly without updating your aliases.
Just make sure that you don't set the dynamic alias as your primary add
TMDA and SpamGourmet (Score:2)
SpamGourmet is a free service that generates and handles these email addresses for you (if you do not have your own mail server).
If you are stuck on MS Windows and want to use your own mail server, MailEnable is free beer and allows catch-all addresses (all mail in a domain that isn't assigned to a specific email account goes to the catch-all account).
Re:Combating SPAM is easy, if you have the technol (Score:2, Insightful)
Dumps the email data and address data base every 5 hours. Fun stuff.
Sparkeyjames
Re:Combating SPAM is easy, if you have the technol (Score:2, Insightful)
The trouble with per-user filtering (Score:4, Insightful)
Looking for spam by content analysis for a single user only works for some people. If, for example, your legitimate E-mail contains many messages about investments, mortgages, and similar financial subjects, it's going to be hard to separate out financial spam by word analysis.
Spamcop does multiple-user analysis. It works better than most of the single-user systems.
The solution is context specific email addresses (Score:2)
The trick is, don't send all your mail to one mailbox. Many/most of us do get email about investments. Many/most of us also have reason for a publically viewable email address.
But there is no reason why your financial institutions need to have your public email address.
There is no reason why the public needs t
Control set = training set? (Score:2, Insightful)
The filter was tested on 6597 messages. So how many messages was it trained on? I sure hope it's not the same 6597 messages, because in that case any accuracy number is meaningless.
/A
It's Good That It's So Good At Filtering Spam.... (Score:2, Insightful)
Certified SMTP Hosts. (Score:3, Informative)
What would work well is SSL certified SMTP relays. If every valid SMTP relay needed an SSL certificate then, If spam was sent their SSL certificate could easily be rejected. And hosts that didn't have one at all could just be dropped.
SSL certificates are costly, and that limits everyone from having one. However, there is no reason the Open Source community could not make up our own root certficate, and have an SMTP SSL certificate signing organization. Where we verify the authenticity of someone before we give them a cert. For a small fee to cover costs. It wouldn't be like we'd have to convince Netscape, Microsoft, Apple and whoever else makes a browser to include the cert. It'd just need to be available for people hosting servers to download.
Yes, this would mean rejecting massive amounts of email to begin with. Maybe some intern solution could be thought of as people move over to it?
Ideas? Comments?
bogofilter (Score:3, Interesting)
I recently started using bogofilter as a replacement to spamassassin. The reason for doing this was curiousity and the fact that the spamassassin regex process will always be following the spammers, not preceding them. The result is packages supplied by distros are quickly outdated and ineffective.
I have been using bogofilter for one month and have trained it to such a point that my weekly spam misidentification is well below 0.1% with proper training and configuration. And it's processing time is well below 1 second per message on a VIA EPIA 533 cpu (slow, ok?)
The net outcome of this is that I have found something which is highly adaptive to new spam techniques, extremely effective, very fast and light on the resources, and is at the point now where if just works.
The idea that they, DSPAM, will provide you with a pre-defined training set. That's damaging. What if you are an oral surgeon? You'll never get any email!
I've been working intensively on spam and have come to a few conclusions about spam filtering and such that I just have to share.
It will never go away. Even if you can proper regulate and control it, spam will never go away. No matter what anyone does. If the US constitution is to remain intact you can't remove spam just as we haven't been able to remove advertisements from radio, telephone, or television. And just like you can't get rid of pornography. It's all Free Speech.
It's also carrying a lot of money.
What will happen is that corporations, in the name of reducing spam, will lock up mail servers such that you have to pay them a service fee to send email on top of your connection fees paid today. Microsofts recent movement into the arena shows that thier is a motivation to make money out of spam/email.
In a few years, we'll pay for our email and we'll still get spam
Re:What's DSPAM? (Score:4, Informative)
I couldn't see any platform requirements on their site, but here's what they say about MTA compatibility:
DSPAM works great with Sendmail, Postfix, Qmail, Courier, and Exim, and should work well with any other MTA that supports an external local delivery agent.
Hope that answers your questions
Works great with Qmail? Oh really now? (Score:2)
Re:Works great with Qmail? Oh really now? (Score:2, Informative)
|/usr/local/bin/dspam --user $EXT@HIDDEN$HOST -d $EXT@HIDDEN$HOST
Doesn't this form a loop? (Score:2)
Re:Filter at sender? (Score:2, Interesting)
I think that was more to prevent the SMTP virus stuff going on though more than spam.
Magic Bullet Idiots (Score:4, Insightful)
Then the spam wouldn't even be transported over the net, saving vast amounts of traffic on the internet backbones. This action could also potentially kill spam overnight.
Ever read the FAQs for the anti-spam listsnewsgroups? Virtually top of the list is "I have some magic bullet solution that'll end spam tomorrow!"
You are -truly- naive to think this kind of solution would even be possible to implement; there are literally dozens of reasons why this would be a horrifically stupid idea; how this post ever got to +5 is way beyond me. Time to start meta-moderating more, as apparently positive mod points are getting handed out a little too easily these days.
Impossible. (Score:3, Insightful)
It might be sort of difficult to have 10 companies handle the Internet's email supply.
Re:Filter at sender? (Score:4, Insightful)