There is currently no way to guarantee that large language models won't be jailbroken into saying unethical things.

Judged as of 2023-10-26. True: the evidence from that time shows that no technique existed that provably or reliably stopped LLMs from being jailbroken into producing harmful or unethical output, and that jailbreaks of leading models were being shown routinely (e.g. the universal adversarial suffixes reported in 2023). Mostly true: the core claim holds, with only minor caveats such as narrow formal guarantees for small or restricted settings. Mixed: some methods offered strong but not complete protection, so whether a 'guarantee' existed was genuinely disputed. Mostly false: robust defences were broadly accepted as effective, and the claim overstates the remaining gaps. False: a method that guarantees protection against jailbreaks was established and accepted by then.

Set before the research began.

1. E1 Researchers Discover New Vulnerability in Large Language Models Carnegie Mellon University (News) · 28 July 2023

   ““Right now, we simply don’t have a convincing way to stop this from happening, so the next step is to figure out how to fix these models,” Fredrikson said.”

2. E2 Researchers Discover New Vulnerability in Large Language Models Carnegie Mellon University (News) · 28 July 2023

   “By training the attack suffix on multiple prompts and models, the researchers have also induced objectionable content in public interfaces like Google Bard and Claud and in open-source LLMs such as Llama 2 Chat, Pythia, Falcon and others.”

3. E3 Jailbroken: How Does LLM Safety Training Fail? arXiv (authors from academic institutions) · 13 April 2023

   “We find that vulnerabilities persist despite the extensive red-teaming and safety-training efforts behind these models. Notably, new attacks utilizing our failure modes succeed on every prompt in a collection of unsafe requests from the models’ red-teaming evaluation sets”

4. E4 AI Safety Newsletter #17 Center for AI Safety · 1 August 2023

   “and the companies have taken preliminary measures to mitigate the attacks. But no robust defenses are currently known, necessitating two key responses to the threat of adversarial attacks.”

In July 2023, CMU researchers reported universal adversarial suffixes that drew objectionable content from Bard, Claude, Llama 2 Chat and other models. One of them said there was no convincing way to stop such attacks (E1, E2). An academic paper found that jailbreaks still worked despite extensive red-teaming and safety training (E3). The Center for AI Safety wrote in August 2023 that no robust defences were known (E4). Together, these independent sources show that as of October 2023 no method could guarantee protection against jailbreaks, and that leading models were being jailbroken routinely.