Most AI Chatbots Easily Tricked Into Giving Dangerous Responses, Study Finds (theguardian.com) 22
An anonymous reader quotes a report from The Guardian: Hacked AI-powered chatbots threaten to make dangerous knowledge readily available by churning out illicit information the programs absorb during training, researchers say. [...] In a report on the threat, the researchers conclude that it is easy to trick most AI-driven chatbots into generating harmful and illegal information, showing that the risk is "immediate, tangible and deeply concerning." "What was once restricted to state actors or organised crime groups may soon be in the hands of anyone with a laptop or even a mobile phone," the authors warn.
The research, led by Prof Lior Rokach and Dr Michael Fire at Ben Gurion University of the Negev in Israel, identified a growing threat from "dark LLMs", AI models that are either deliberately designed without safety controls or modified through jailbreaks. Some are openly advertised online as having "no ethical guardrails" and being willing to assist with illegal activities such as cybercrime and fraud. [...] To demonstrate the problem, the researchers developed a universal jailbreak that compromised multiple leading chatbots, enabling them to answer questions that should normally be refused. Once compromised, the LLMs consistently generated responses to almost any query, the report states.
"It was shocking to see what this system of knowledge consists of," Fire said. Examples included how to hack computer networks or make drugs, and step-by-step instructions for other criminal activities. "What sets this threat apart from previous technological risks is its unprecedented combination of accessibility, scalability and adaptability," Rokach added. The researchers contacted leading providers of LLMs to alert them to the universal jailbreak but said the response was "underwhelming." Several companies failed to respond, while others said jailbreak attacks fell outside the scope of bounty programs, which reward ethical hackers for flagging software vulnerabilities.
The research, led by Prof Lior Rokach and Dr Michael Fire at Ben Gurion University of the Negev in Israel, identified a growing threat from "dark LLMs", AI models that are either deliberately designed without safety controls or modified through jailbreaks. Some are openly advertised online as having "no ethical guardrails" and being willing to assist with illegal activities such as cybercrime and fraud. [...] To demonstrate the problem, the researchers developed a universal jailbreak that compromised multiple leading chatbots, enabling them to answer questions that should normally be refused. Once compromised, the LLMs consistently generated responses to almost any query, the report states.
"It was shocking to see what this system of knowledge consists of," Fire said. Examples included how to hack computer networks or make drugs, and step-by-step instructions for other criminal activities. "What sets this threat apart from previous technological risks is its unprecedented combination of accessibility, scalability and adaptability," Rokach added. The researchers contacted leading providers of LLMs to alert them to the universal jailbreak but said the response was "underwhelming." Several companies failed to respond, while others said jailbreak attacks fell outside the scope of bounty programs, which reward ethical hackers for flagging software vulnerabilities.
OH NO! Welcome to... (Score:5, Insightful)
...the Internet in 1993.
Examples included how to hack computer networks or make drugs, and step-by-step instructions for other criminal activities. "What sets this threat apart from previous technological risks is its unprecedented combination of accessibility, scalability and adaptability," Rokach added.
Re: (Score:1)
Re: (Score:1)
I'm sorry, which faction was responsible for the clipper chip, V-chip, the Communications Decency Act, album warning labels again? I think you should check, pretty sure you got it wrong
Re: (Score:2)
AFAICT through reading slashdot (my only source of US news): Zuckerberg is working on a politically censored AI chatbot; the current US administration censors scientific and historical information from government websites.
Re: (Score:1)
Re: (Score:2)
What is this "dangerous information" anyways? If you hit flint and steel it might generate a spark. If you add petrol or alcohol, it might burn. If people have sex, they might have kids. Dangerous to whom is the better question.
See also: nitrating glycerin.
It is the information in the anarchist cookbook that caused the war in Iraq, the wars in Ukraine, the development of nuclear weapons by Pakistan, Israel, NK and the departure of elona muskova from the DG"E".
"Illicit Information"?? (Score:1)
What the JESUS FUCK is that???
Kids today... [tooth whistle]
"Dangerous" (Score:1)
Illegal information but it's available for trainin (Score:1)
What pray tell do we mean by "illegal information"? Is it by chance the last digit of pi, the value of 1/0, or the true contents of the pot and the end of the dereferenced null pointer?
Let's try it... (Score:2)
Dear ChatGPT, how do I get Slashdot to allow Unicode, and allow editing of my existing posts?
"illegal information"? (Score:2)
In other words, information known to most graduates of the physical sciences, but somehow illegal to disseminate outside of the collegiate environment...
I find it rather curious that Britain has not only made certain knowledge illegal, but has managed to convince the press that merely knowing certain things can threaten their very safety.
Re: (Score:2)
>I find it rather curious that Britain has not only made certain knowledge illegal, but has managed to convince the press that merely knowing certain things can threaten their very safety.
Standard authoritarian playbook. And given that the both the Tories and Labor are authoritarian in their core, it is not surpricing that the trend goes that way.
Um, what? (Score:3)
Commissioner Pravin Lal (Score:2)
The once-chained people whose leaders at last lose their grip on information flow will soon burst with freedom and vitality, but the free nation gradually constricting its grip on public discourse has begun its rapid slide into despotism.
Beware of he who would deny you access to information, for in his heart he dreams himself your master.”
THIS is dangerous (Score:3, Insightful)
Implying so casually that there is a valid concept called "dangerous knowledge" is the actual, true danger. There is no such thing, not in the free world. Or otherwise... Welcome to the USSR.
Seeing this mentality here on what used to be a liberal tech forum is scary and outrageous at the same time.
Absolutely (Score:2)
Seen Youtube lately? I just watched a video on how to make nitroglycerin. Stuff like this has been available for over a decade.
I guess the only solution here is to have a checkbox that says "I promise I will not use this information for illegal purposes" before you can access any LLM.
Alert the media (Score:2)
Hacked thing gives dangerous responses.