IETF to Look at Spam 211
m00nun1t writes "CNET has an article about the Internet Engineering Task Force (IETF) looking at what they can do about spam. According to the article, many of the proposals seems to "require changes in basic e-mail technology", which presumably means SMTP (and about time!). Maybe they are looking beyond just SMTP - anyone have any insights here?"
It wouldn't be adopted instantaneously. (Score:5, Insightful)
Re:What's the point (Score:3, Insightful)
Simply put, it's too easy to create spam over the SMTP protocol. The from, and reply to fields are completely free text, and have no requirement that they must be a reflection of the actual sender of the message.
However, if SMTP were to fall out of favor for a new protocol, that new protocol could start the rules over and require that the server that is named in the from field must confirm that the username provided actually sent the message. Spoofing for the use of spam would then become practically impossible.
Once we have a confirmed from address, it puts a responsiblity to stand behind each e-mail sent through a server. Moreover, once spam use has been detected a really reputable server operator could simply stop authenticating that sender's e-mail.. causing auto deletion before its presented to a user.
If you can't make it illegal, try to make it impossible...
Changing SMTP (Score:5, Insightful)
The article notes that one of the major problems is the filtering of genuine mail due to agressive spam filters necessitated by cleverer spammers. Consider this analogous to dropping some packets at the network layer. Just as the transport layer handles this problem, we can build a higher level protocol to handle filtered mail.
Note that having a mechanism to handle dropped mail allows us to employ agressive filtering: one that is sure to stop 100% of spam.
What I have in mind is as follows: when Bob receives a mail from Alice (i.e, it has passed through Bob's filter) the client software sends a confirmation mail back to the Alice. This is not a regular mail that the Alice will see in her inbox; it has a special header flag that marks it as a confirmation. Alice's client software keeps track of the confirmation messages; by looking at her "sent-mail" folder she can see which of her messages have not been confirmed (and are hence likely to have been mistaken for spam).
Finding that Bob has filtered her mail, Alice can either re-word it and send it again or do something like (assuming that Bob knows Alice): "Hi Bob, this is me, Alice. Your filter blocked this so I've rot13'd it to get past the filter. rot13 what follows to read my mail." Another option is to encrypt the mail with Bob's public key (assuming that spammers' scripts won't be clever enough to get your public key from your web page). Note that 99% of the time the mail is going to get through. You have to make that little effort to prove you are a human only once in a long while.
There is minor problem with requiring the receiver to send a confirmation message: Bob might check his mail only after a couple of days, during which time Alice may assume that her mail was blocked. There are 2 solutions: either Bob runs a script to filter his mail regularly, or else has his ISP implement his filter for him.
Note that this won't work if you have the receiver send a reply whenever the message did get blocked: the reply could itself get blocked etc. (This is called the red army - blue army problem in networking).
Keep it simple... (Score:5, Insightful)
World of Ends [worldofends.com], recently [slashdot.org] discussed [slashdot.org] on Slashdot, discusses why the simplicity (or stupidity) of the Internet is so useful. "The Internet isn't a thing. It's an agreement," they say.
That same argument applies to e-mail. Following their logic, it is best to leave SMTP alone. Simpler protocols are better. Leave the "value-added" pieces to the edge, and let the simple message transfer protocol alone.
Authentication... (Score:3, Insightful)
I know - I don't write code - and this probably sounds stupid. But I don't really see any way of forcing someone to quit sending you email. SMTP is short and sweet - but it can't continue to just hop mail. It has to be checked somehow. And it would slow down the mass emailers a lot. Hopefully someone a lot smarter than I in this area can come up with something.
Duke
Re:Move the onus from the recipient to the sender. (Score:4, Insightful)
This system has some flaws:
1) If a person sees an e-mail in their inbox, then they can read it, and they are happy. Can you imagine the hordes of people who would now see that they got an e-mail, but could not get it for one reason or another? This makes e-mail *seem* fragile. Please explain to my step-father why he can see that he has e-mail, but he cannot read it on the plane. This is not a technical issue, but a psychological one, which is much harder to program around :-)
2) By what criteria could you filter the email? If you have not received the e-mail, you probably won't have enough information to tell if it is spam or not. The only information that you could go on is what is in the "notice" message.
Nice try.
Power grab by TBTB (Score:3, Insightful)
But this doesn't help (Score:1, Insightful)
If I recieve an email from Foo@bar.com, how do I know that it's not from one of my friends who are using the Bar cybercafe or from someone on a mailing list I'm on? Most spam filters rely on actually reading the contents of the email. It would be nice if I could tell the Bar.com server, "please filter out this spam using this filter" but it ain't going to happen and if it does there's no way I'm going to trust them to do it.
So basically, the only way to be sure it's not spam, is if I filter it in my own system.
Backward compatibility (Score:5, Insightful)
Of members of the lay public who regularly use email as a means of communication do not have the level of technical comfort that most Slashdot readers take for granted.
Of people who use email, the percentage who know how to use a web browser is much less than 100%. The percentage who can google for information is much less than 100%. The percentage who can successful extract and decode an email attachment is much less than 100%. The percentage who can view a government form or a corporate brochure in PDF format and read it with Acrobat is much less than 100%.
And the average age of their computers and operating systems is much more than three years--and they're not likely to update their email programs.
Whatever is done needs to be 100% backward compatible with existing email clients, not requiring even simple upgrades, or an astonishing proportion of real-world Net users will be disenfranchised.
(And please, let's not have any facile expressions of contempt for AOL users or webtv clients or people who bought email appliances (that includes one of the retirees I mentioned).
Re:Spammers too smart for this. (Score:4, Insightful)
TEBJ LBHE CRAVF OL FVK VAPURF, VS LBH URYC ZR FZHTTYR ZBARL SEBZ AVTREVN.
(not to mention that return token let's them know that you at least look at your mail.)
*Barf* (Score:3, Insightful)
Disadvantage of the current internet (Score:2, Insightful)
Due to the original designs being not real secure, I'm quite sure that the spam problem can not be solved without fundamental changes in the way we use email nowadays. Perhaps the policy regarding blacklisting can be changed: at this moment most people accept mail from everybody, but not from a few blacklisted sites. It's likely that this will be changed: we don't accept your mail unless we know who you are. Unfortunately, even then there will always be people who will abuse it. Hopping from one account to another, or sue-ing every single ISP that has the guts to disconnect their connection after spamming. In short: it's not simply a technical matter, their will be a need of *globally equal* legislation too. Legislation alone won't do the trick either. No, it's time for Mr Geek to marry Miss LawAndOrder.
Don't forget that the IETF is not the first to attempt to find a solution. RIPE [ripe.net] has its anti-spam workgroup for example.
Re:RSS (Score:4, Insightful)
Instead of adding a band-aid solution to spam, let's sit down and list what we need for an a-mail server. Scalability, reliability, fault tolerance, expandability and distributed servers top my list. I'm sure that there are other better ones out there too. If you're going to revamp the protocol, try to get everything in the first time, and let's try to get it right.
Re:192.168.0.0/24 routing (Score:4, Insightful)
Computationally, that's a lot of public key encryption in action. For sites that process large amounts of email, this is going to hurt. But let's say we can throw money and CPU at that problem. And I suppose we can do the same for the problem of the tens of millions of key/address pairs we'll need to store centrally. Not so bad, then.
Socially, the existence of anonymous email may be important and valuable. But I suppose anonymous remailers could appear and use their own corporate keys to signed messages in your scheme.
Practically, you'd need a way to prevent denial of service attacks against someone's email by generating sufficient fradulent 'bad reports' to cause their key to be centrally revoked. This seems bad.
Nothing totally insurmountable, but still pretty annoying to deal with.
(P.S. No packet on the internet is truly identified by its source in most cases. IP addresses can be forged fairly trivially. This doesn't really bear on your proposal, but thought you should know.)
Re:It wouldn't be adopted instantaneously. (Score:3, Insightful)
Presumably AMTP servers (a name I'm making up, A for authenticated) would accept connections from legacy SMTP servers, but prefiltered with various ad-hoc spamblock techniques we use now (Bayesian filtering, limits on connection rates, etc.)
Re:It wouldn't be adopted instantaneously. (Score:2, Insightful)
Adopting a new protocol is very different than upgrading to a new version of an implementation of a protocol. In the case of a new protocol, there might be two different kinds of things going on at the same time, either with the same MTA, or different MTAs. In the case of Apache 2.0, you can't have the same web site available under the new version at the old version at the same time. With a new protocol, you can easily have a transition period because of the window of concurrency. With a new version of an implementation of the same protocol, deployed for a single instances of usage (e.g. a domain), it's basically one or the other. You can run Apache 2.0 on www.test-site.example.com while Apache 1.3 still runs www.example.com. But you can't have www.example.com running both very easily.
Re:Move the onus from the recipient to the sender. (Score:5, Insightful)
It's implementation could also be made rather interesting. Rather than a completely new protocol that is totally impractical (since it would require everyone upgrading simultaneously) this kind of scheme could be implemented in a completely backwardly compatible manner. Allow me to describe what I mean.
Your email server has been upgraded to the new system and you send an email. Your outgoing server store the email and forwards a very simple email message onto the recipients email server, this small email contains the appropriate subject line and an extra chunk (with appropriate mime type) containing the information necessary to retrieve the full email message (ie: Server details, email id and probably an authentication token of some sort). Your client software supports the new standard it receives the stub email and retrieves the full message appropriately. This stub email is not an extra compatibility thing; we are simply using the existing smtp infrastructure to tell the recipient that they have a piece of email.
But what if the recipient has not yet upgraded, here comes the clever bit. Html email works as an extra mime chunk that enabled clients automatically decode and show the reader, non enabled clients see the standard plain text version of the message that is also present in the message, this mechanic can be used to our advantage here, the normal text or html portion of the stub email contains a hyper link back to the sending server which a url designed to bring up a basic web mail page with the recipients message.
Using this implementation scheme it would be possible for the sender who upgraded from day one to send an email to anyone with the complete confidence that they would be able to read the full text of the message. The only proviso here is that the recipient had access to a web browser.
In addition I can see one other advantage to your proposed scheme that has not been mentioned, the email system becomes inherently more secure. Since the sending server must actively hand over the email it can record that this has been done and tell the recipient if the message had been read before. Although as with anything else strong cryptography would be required to ensure to ensure that nobody could get hold the authentication token (and thereby read the email) it would be possible for the sending server (providing you trust it) to tell you authoritatively that nobody else had retrieved the message contents.
Re:Spam can be avoided without protocol changes. (Score:3, Insightful)
No. Not all redundant mail is spam. There are plenty of legitimate distribution lists out there.
However highly localized approaches like personal mail filters will always fail due to the high variety of spam.
Yes. However, if a high enough % of people point at an email message and say 'spam' then it's spam. People are very good at spotting spam- so a mechanism that records what people think about the mail and not deliver it to anyone that hasn't already read it would be very successful. If 90% of the first few thousand people to read an email say it's spam, the other million need never see it at all.
Re:Move the onus from the recipient to the sender. (Score:3, Insightful)
When the checking-mail process begins, the client would go to the receive-side server to get the list of notifications received. It would first apply any local filter rules to strike out unacceptable notifications, then go one-by-one to the servers to confirm that they sent the message the notification claims, that the server is still offering the message, and than ask for the message itself.
If the message has been declared spam by the server operator, then the server will intentially pull the message from availablity and essentially vaporize it before it hits a majority of inboxes. Server owners have an incentive to do this... because it'd be extremely easy to add server owners who don't into a local blacklist.
Yeah, a verbose log file can be made available for the geeks that wanna know what happened under the hood, but the average end user wouldn't see the message pop into their Inbox until the message has been sucessfully cleared and transmitted. Once its in the Inbox, it's a local object that the user can do what they want to.
If they are reinventing SMTP, might as well... (Score:5, Insightful)
Essensially, what we need is forced authentication (Score:2, Insightful)
Now what happens to all the people that want to send mail from a different host? They need to auth with their "real" mail server. So if I want to send from otherdomain.com with a domain.com address, it'll have to go like this:
otherdomain.com -> domain.com -> recievingdomain.com
This would not stop spam per se, but it would stop fake from e-mail adresses, which is extremely annoying.
Kjella
Re:Backward compatibility (Score:3, Insightful)
Whatever is done needs to be 100% backward compatible with existing email clients, not requiring even simple upgrades, or an astonishing proportion of real-world Net users will be disenfranchised.
Whatever is done needs to be able to deliver email while effectively correcting the system of incentives that encourages spamming. Any other considerations, like it or not, can only addressed insofar as they don't interfere with acheiving that goal. The current email system is broken and an ever-increasing amount of noise is flooding into that system. The end result is that the delivery system (which, as you have pointed out, is important to so many people) is in the process of collapsing. You talk as if the choice is between a healthy email system and some new one that we don't really need, when it's really a choice between a system that will inevitably be rendered useless by spam volume and a new one. And that new one has to include whatever features are required to avoid a repeat performance.
If the best solution is all server-side (and some proposals are) they may be able to also get the kind of backward-compatibility that you feel is required. But, make no mistake, we aren't doing anyone any favors if we don't actually fix the current system, even if the fix does eventually require a client upgrade when the last parts of old system are finally phased out. If it is any comfort, I would expect that "AOL users or webtv clients or people who bought email appliances" will be the least-effected since their providers understand that market and have control of both ends of that particular client-server implementation.
Re:AOL, Earthlink, MSN, etc. (Score:4, Insightful)
Convincing "larger ISPs" to implement an alternative standard would also require prodigious effort.
Actually, the more mail your site receives, the more interested you tend to be in stopping the flow of spam. If you consider how much in resources they spend dealing with spam in terms of capacity (for storage, bandwidth, processing volume, and filtering) and user complaints, it isn't that surprising. If a workable implementation ever comes out of this, you can expect the larger ISPs to have test servers up pretty quickly.
Re:Move the onus from the recipient to the sender. (Score:4, Insightful)
The big problem I see with this is that it would work very well over robust, high-speed networks where all servers have 24/7 reliability. How well will it work over less robust or fast networks? The latency involved in querying and fetching 100 messages adds up pretty darn quick.
If the message has been declared spam by the server operator, then the server will intentially pull the message from availablity and essentially vaporize it before it hits a majority of inboxes. Server owners have an incentive to do this... because it'd be extremely easy to add server owners who don't into a local blacklist.
I think a much better option would be to stop it before it becomes submitted. But I see significant power issues involved with giving sysadmins the power to retroactively nuke messages by content. Yeah, it helps to stop spam but it also gives the sysadmin the power to nuke political content as well.
In addition, I can see how such a system can be technically circumvented by spammers. Set up a server to broadcast bogus notifications and just send a single file out. Blacklists are not effective then for the same reason they are not effective now, the costs of setting up on a new IP is trivial.
Yeah, a verbose log file can be made available for the geeks that wanna know what happened under the hood, but the average end user wouldn't see the message pop into their Inbox until the message has been sucessfully cleared and transmitted. Once its in the Inbox, it's a local object that the user can do what they want to.
Ok, the initial description just sounded like some kind of a distributed peer-to-peer imap where instead of storing the messages on the recipient server the messages are fetched as they are read. But I disagree that this process will be transparent to the user because of the added latency as the recipient server authenticates each individual messages. Checking my mail with IMAP, I know what is available within a second after I open a connection (using local mailboxes is even quicker). I don't see how a "pull" system that authenticates, verifies and fetches for each mail message can match that performance.
Chain of Trust - Problems (Score:3, Insightful)
What you are describing is commonly known as a chain of trust. A common usage of this is in public-key infrastructure deployments in global organizations such as the military and pharmaceutical companies or in chained SSL/TLS servers.
In those types of environments (internal to an organization) there is ultimately a trusted root server that acts as the certifying authority, telling all servers and users within a cicumscribed domain that you can trust servers X, Y and Z.
The problem for e-mail and spam is that in cross-domain environments (external to an organization), this type of infrastructure does not currently exist. How do I know that I should trust mail from domain XYZ.com or ABC.com unless I have already negotiated a trust agreement with those domains?
If we only trust the servers, we implicitly trust that the administrators of the servers have authenticated the users. This is usually not the case, especially in the case of spammers.
Authenticating users would likely have to be done through a certificate-based system which raises all kinds of privacy concerns.
Authenticating servers and even authenticating users would not thwart the use of authorized e-mail accounts to send spam. This was demonstrated when spammers paid Tufts University students $20/month to relay spam via their e-mail accounts [slashdot.org].
It's a challenging technical problem, but it's an even more challenging social problem.
Would paying really BE more expensive? (Score:3, Insightful)
However , of late, with other people making such gross abuse of the world's mail systems that I feel I am (and all of us are) paying anyway, it might be worth revisiting this question. I'm not 100% convinced that paying per piece of mail sent would be more expensive than, effectively, paying (in both time and in dollars spent on ISP infrastructure) per piece of mail received. I send a lot less mail than I receive, and I bet most people do.
Re:Dr. Jeffery Race's proposal on NANOG (Score:1, Insightful)
In communities small enough for everyone to know each other, it is extremely difficult to be a Bad Apple. The anonymity of large communities brings out the bad apple in borderline cases and therefore you see a higher apparent percentage.
The answer is to silently identify and block bad Apple behavior. In small communities, ostracism works well. Ostracism has been tried on the net with RBL et al but is difficult due to the anonymity (the bad guy simply moves on to a different IP leaving behind a bunch of blocked innocents).
Identify bad behavior and block it at the relay level. It is the only way to preserve the openness we have now.