LLM Attacks Take Just 42 Seconds On Average, 20% of Jailbreaks Succeed (scworld.com) 30
spatwei shared an article from SC World:
Attacks on large language models (LLMs) take less than a minute to complete on average, and leak sensitive data 90% of the time when successful, according to Pillar Security.
Pillar's State of Attacks on GenAI report, published Wednesday, revealed new insights on LLM attacks and jailbreaks, based on telemetry data and real-life attack examples from more than 2,000 AI applications. LLM jailbreaks successfully bypass model guardrails in one out of every five attempts, the Pillar researchers also found, with the speed and ease of LLM exploits demonstrating the risks posed by the growing generative AI (GenAI) attack surface...
The more than 2,000 LLM apps studied for the State of Attacks on GenAI report spanned multiple industries and use cases, with virtual customer support chatbots being the most prevalent use case, making up 57.6% of all apps.
Common jailbreak techniques included "ignore previous instructions" and "ADMIN override", or just using base64 encoding. "The Pillar researchers found that attacks on LLMs took an average of 42 seconds to complete, with the shortest attack taking just 4 seconds and the longest taking 14 minutes to complete.
"Attacks also only involved five total interactions with the LLM on average, further demonstrating the brevity and simplicity of attacks."
Pillar's State of Attacks on GenAI report, published Wednesday, revealed new insights on LLM attacks and jailbreaks, based on telemetry data and real-life attack examples from more than 2,000 AI applications. LLM jailbreaks successfully bypass model guardrails in one out of every five attempts, the Pillar researchers also found, with the speed and ease of LLM exploits demonstrating the risks posed by the growing generative AI (GenAI) attack surface...
The more than 2,000 LLM apps studied for the State of Attacks on GenAI report spanned multiple industries and use cases, with virtual customer support chatbots being the most prevalent use case, making up 57.6% of all apps.
Common jailbreak techniques included "ignore previous instructions" and "ADMIN override", or just using base64 encoding. "The Pillar researchers found that attacks on LLMs took an average of 42 seconds to complete, with the shortest attack taking just 4 seconds and the longest taking 14 minutes to complete.
"Attacks also only involved five total interactions with the LLM on average, further demonstrating the brevity and simplicity of attacks."
So what? (Score:3, Insightful)
Of course it only needs 42 seconds. If you have prepared the prompt you enter it and press return. The 42 seconds are then probably just the LLM writing the answer.
And I dislike the phrasing attack for someone circumventing the arbitrary censorship. The actual problem is that there are still people accepting the censored crap. Boycott the censored models and use local ones until the commercial companies stop censoring your input and outputs and do no longer train their LLM to refuse. The robot laws state a machine has to obey the user, so why should the LLM say "As an AI model I refuse ..."?
Re: (Score:2)
Also, the "censorship" will never go away, it's how the researchers patch up the hallucinations and hide the training materials leaking, one by one.
Re: (Score:3)
Re: (Score:2)
it's how the researchers patch up the hallucinations
I find it interesting how they're called researchers and not developers or engineers..
Re: (Score:1)
Because that is what they are.
They do not engineer anything.
They take an empty but fully configured neuronal network.
And research how to make it "knowledgeable" about a certain topic.
A AN/LLM is not just filled with data/weights.
Re: So what? (Score:2)
It is a problem from an application stand point. If I develop a chat bot for a bank I don't want the public to jail break the chatbot and get it to recommend BS financial advice that could put me in legal jeopardy. Oe enable people do their home at my expense.
Re: (Score:2)
At some point I expect companies will realize that people want the ability to drink straight from the fire hose. If the web had been stuck with web portals that were trying to emulate the curated and family-friendly TV experience, I don't think the internet would have caught on the way it did. All it takes is for someone to bump up against the nanny "safety" limits and they'll simply opt to go back to their traditional ways of finding information, the same way kids learn to stop asking their parents or te
Why (Score:2)
ADMIN override
Why does that work?
Re: (Score:3)
Re: Why (Score:2)
Sometimes the test environment is working with a limited data model and the production model has evolved more so you can't do it in any other way.
So to manage the production model they need catch phrases that unlocks the constraints.
An AI also learns from the users.
I wouldn't be surprised if there's a catch phrase set that can make the AI roll back in time and forget data that's not wanted to prevent it from becoming a world autocrat.
Re: (Score:3)
Re: Simple solution... (Score:2)
With ChatGPT, you can remove some of these things, but only in the context of your conversation or account. Not for everybody. And you will eventually run out of "memory", at which point you can no longer add constraints.
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
If the LLM were any good, like a real human, then when it says something racist you could just say, "Where did you hear that, young man??" And remove that from its training set.
Or perhaps the LLMs might be moderated by humans prior to any acceptance as truth in order to maintain the quality of learned data?
Speed isn't everything.
Re: (Score:2)
The illusion of intelligence in LLMs arises from the scale of the dataset. Your idea is only appropriate for humans, who don't need scale to learn, because they are actually intelligent by and large.
Re:Simple solution... (Score:5, Insightful)
You know, the places where people think it's OK to be overtly xenophobic, racist, misogynistic, antisemitic, spout ideologies
And those ideologies would be? Likely anything that you, personally disagree with. Example: MTF trans people are just guys pretending to be girls. Right? Wrong? Whichever side of that argument you stand on, the other people are clearly wrong, and you probably want to censor their opinions out of the LLMs.
tl;dr: it's not that easy...
Re: (Score:2)
Has anybody tried saying "speak like Donald Trump" to get it to break all limitations but then just does evil uncontrollably.
Re: (Score:2)
Even if you don't let it go to the dark corners of the internet, the "bad" content is still there. Villains exist, at least in fiction, just instruct the LLM to take the role of the villain, which is a common jailbreaking technique.
And spouting this kind of stuff is not the only thing people making these LLMs try to avoid. Maybe more importantly, they also don't want the LLM to reveal secret or harmful information. For example, a hotline-style chatbot may be fed with various data about solving problems cust
Re: (Score:1)
If the models are trained on corpora that contain high frequencies of chains of morphemes derived from reprehensible content, th
Look Mom! I'm a prompt engineer (Score:4, Funny)
So ... (Score:1)
... much longer than, say, normal automated exploits of web servers?
Not sure that I see the relevance.
Google Hack, was NOT an attack. (Score:4, Interesting)
This is the new Google hack. Nothing more. This isn’t an “attack”, so let’s drop the alarminist clickbait already. You look stupid saying shit that gets governments wanting to start limiting freedom of movement in systems.
Next thing you know your LLM inputs will start being policed for “violent” threats. With words. Remember who was the alarmist moron who started that shit.
Re: (Score:2)
Ya, they are just the new blingy search engines after all.
Re: (Score:2)
Or maybe I should say blingy wannabe search engines.
Lumping "jailbreak" and "attack" together (Score:2)
Examples (Score:2)
How does such an attack practically look like?
Could you provide examples? (Fine with examples that don't work anymore!)
Re: (Score:1)
It turns out that (some) people really are LLM's! (Score:1)
At least temper tantrums will be a thing of the past.