Gmail's AI-Powered Spam Detection Is Its Biggest Security Upgrade in Years (arstechnica.com) 45
The latest post on the Google Security blog details a new upgrade to Gmail's spam filters that Google is calling "one of the largest defense upgrades in recent years." ArsTechnica: The upgrade comes in the form of a new text classification system called RETVec (Resilient & Efficient Text Vectorizer). Google says this can help understand "adversarial text manipulations" -- these are emails full of special characters, emojis, typos, and other junk characters that previously were legible by humans but not easily understandable by machines. Previously, spam emails full of special characters made it through Gmail's defenses easily.
[...] The reason emails like this have been so difficult to classify is that, while any spam filter could probably swat down an email that says "Congratulations! A balance of $1000 is available for your jackpot account," that's not what this email actually says. A big portion of the letters here are "homoglyphs" -- by diving into the endless depths of the Unicode standard, you can find obscure characters that look like they're part of the normal Latin alphabet but actually aren't.
[...] The reason emails like this have been so difficult to classify is that, while any spam filter could probably swat down an email that says "Congratulations! A balance of $1000 is available for your jackpot account," that's not what this email actually says. A big portion of the letters here are "homoglyphs" -- by diving into the endless depths of the Unicode standard, you can find obscure characters that look like they're part of the normal Latin alphabet but actually aren't.
Spam detection was fine, but the Spell checker! (Score:3)
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
Spell checkers formerly got very tripped up by homonyms, for example: "That sure is a plane looking plane." Also homophones almost always need context to be checked correctly. Consider: "Everyone accept me got to go." or "It's currently ate o-clock."
Nit: That would be grammar checkers. There's nothing wrong with the spelling of either of those two sentences. They just make no sense because the wrong word was spelled.
Re: (Score:2)
Semantics. Technically you know which word you mean, but you don't know which spelling goes with it. It corrects the spelling of the word you mean to type from the word you did type.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
like a wood plane and an aeroplane? They're both spelled correctly and are both grammatically correct as entered.
Try again.
Re: (Score:2)
I believe they use predictive keyboard type of algorithm and run what you're typing through that to suggest corrections based on what words you're likely to use vs. what you actually type (and using that as a basis to decide if a homophone is the wrong word). Which, yes, is the precursor to something like ChatGPT. ChatGPT is essentially the same idea on a bigger scale but uses prompt text instead of trying to continue your own typing. And uses an entirely different type of training data.
Apply it to SEO fuckers also (Score:5, Interesting)
1 - do their usual stuff (crawl as googlebot, get the page)
2 - crawl as Chrome or Firefox in plain user version, render the page in the browser, do a screenshot and OCR it...
3 - compare the text of the 2 versions (now with IA !). If they differ too much, penalize the page/site.
This would handle text in images, text with fucky unicode, white text on white background, etc...
Re: (Score:2)
Re: (Score:2)
And especially, to kill ads. Especially ones injected by Google.
Re: (Score:2)
Smuggling in censorship (Score:2, Insightful)
Re: (Score:2)
Not yet. But this is clearly a preparation step.
Re: Smuggling in censorship (Score:3)
over engineering (Score:1)
Re: (Score:2)
Re: (Score:2)
I have a custom spamassassin rule that blocks subjects with those in it. Extending that to full emails is probably something like half an hour of Perl or Python scripting. I guess that is too hard for Google these days...
Re: (Score:2)
Re: (Score:2)
As I said, "too hard". In this case, too hard for the organization, not too hard for the engineers.
One of the signs of a tech company in deline is when they cannot do simple things right anymore.
Re: (Score:2)
That can be pretty tough when you're a multi-lingual system and some languages spell English loanwords with a Latin alphabet. I think it would be easier to normalize the glyphs to whichever form spells a real word and then use that in the filtering.
Bayes (Score:2)
In my experience, Bayes was very good at catching stuff like this, because words with homoglyphs in them were very quickly classified as very spammy.
I welcome the ensuing AI battle (Score:2)
While I believe AI may be able to filter spam more effectively, I also believe AI will be used to generate spam more effectively too. I believe the generative AI will be more effective than the protective AI, since all you have to do is setup a gmail account to check if it was filtered, so that AI can get immediate, automatic feed back.
Whereas the filter AI relies on people marking it as spam to fix any errors, The may also never even see any false positives. I personally welcome the day where AI gets bette
I would've thought homoglyphs are easy to detect? (Score:3)
Re: (Score:2)
Strange. I would've thought that detecting homoglyphs should be relatively easy? There really aren't THAT many legitimate use-cases to use vastly different Unicode codepoints from different parts of Unicode in the sme email? Or to use the lookalike Unicode characters mixed with regular ones?
Yeah, throwing "AI" at a simple task like this seems like they are just looking for nails to hit with their AI hammer. Also as other posters have mentioned, we've known for years that homoglyphs have been used a lot on spam - if they were serious about reducing spam, a human at Google should have written a script to deal with this simple vector many years ago.
But can it detect AI spam? (Score:2)
Detecting AI generated content is pretty much impossible at this time, after all. So this is probably just the next step in the arms-race.
Current filters use invalid IP-based block lists. (Score:2)
Gmail puts emails from my Linode-hosted server into SPAM folders. If they are send from a VM hosted elsewhere, they don't go into the SPAM folders.
My Linode server is no blocklists. SPF and DKIM are setup and working properly. I run a small family mailserver: no spam sent ever, since I began using the IP address. There is absolutely no reason for my emails to be categorized as SPAM.
Clearly, there is no intelligence involved in this categorization as SPAM, artificial or otherwise.
Perhaps the new upgraded fil
Re: (Score:2)
Unlikely. Gmail doesn't want to play nice with independent self-hosted email. They want you to find the experience so sucky that you throw up your hands in despair and move to Gmail so they can mine all your family's data. Stay strong!
Re: (Score:2)
I want to reinforce that, if I route the email through a different VM, hosted by a different service, Gmail doesn't treat them as SPAM. The only difference is the IP address the emails are coming from.
Re: (Score:3)
You have proven how straightforward it is to get around IP-based spam filtering. You'd think Google would also know in 2023 how pointless it is to flag spam based on IP address. It seems so dumb that I'm inclined to ascribe it to malice rather than incompetence.
This doesn't seem like a massive problem (Score:2)
"by diving into the endless depths of the Unicode standard, you can find obscure characters that look like they're part of the normal Latin alphabet but actually aren't."
So, filter every email that uses "lookalike" characters. Done and dusted.
Re: (Score:2)
Not every e-mail that uses them: some of the lookalikes are e.g. Cyrillic letters. What it should do is look at the metadata to see what language the e-mail claims to be in and weight characters accordingly.
outbound too, right? (Score:3)
GMail is going to apply this filtering to its outbound mail too, right? Right?
Dear GMail, saving your users from spam while allowing them to relentlessly spam the rest of the world is evil. Be sure to apply your new filtering to your user's outbound mail too, maybe GMail would stop being the top spammer in the world.
Re: (Score:2)
Dear GMail, saving your users from spam while allowing them to relentlessly spam the rest of the world is evil. Be sure to apply your new filtering to your user's outbound mail too, maybe GMail would stop being the top spammer in the world.
Most of my spam comes from Google Groups.
Spammers import lists of email addresses and there's no way to remove yourself from the list (I've tried everything from "unsubscribe" to clicking the unsubscribe link) as you're not quite sure what address spammers used and either
They need it (Score:3)
Heh, double negatives (Score:2)
Their chart [googleusercontent.com] shows their "False Positive reduction" was -19.49% and "False Negative reduction" was -17.71%, but I doubt they'd be reporting on negative reductions (aka increases).
Kidding aside, this serves to me (as a researcher in this exact space) as a reminder of how bad GMail is (was?) at threat detection; a spam-specific tokenizers like the (not at all recent) one in SpamAssassin is pretty good (see Paul Graham's 2003 Better Bayesian Filtering [paulgraham.com] writeup, which SA used extensively). There's lots of room f
text in images (Score:1)
this does nothing for image based spam where the text is a vector in a graphic file.