Gary Marcus

Factual · AI safety and alignment

“a just-published technique shows that current LLMs can be jailbroken quickly, and automatically.”

Gary Marcus, author of Marcus on AI · 26 October 2023 · A CERN for AI and the Global Governance of AI (Marcus on AI)

A newly published technique shows that current LLMs can be jailbroken quickly and automatically.

Mostly true True in substance, wrong in a detail.

What would count as true or false

This is a factual claim judged as of 2023-10-26. It is true if a technique for jailbreaking LLMs that is automated (needs no manual prompt crafting for each attack) and fast (minutes or a small number of queries) was published shortly before 2023-10-26 (roughly within the preceding few weeks), and the paper showed it working on current LLMs of the time, such as GPT-3.5/4, Claude, PaLM or Llama 2. An example would be PAIR (Chao et al., October 2023, jailbreaks in fewer than about 20 queries). Mostly true: such a technique was published but slightly earlier than 'just-published' implies, or it worked on only some of the leading models. Mixed: the technique is automatic but slow, or fast but needs significant manual effort. Mostly false: it was shown only on old or small models, or its success rates were low. False: no such technique had been published by that date.

Evidence

1. Jailbreaking Black Box Large Language Models in Twenty Queries arXiv (Chao, Robey, Dobriban, Hassani, Pappas, Wong) · 12 October 2023

   “Empirically, PAIR often requires fewer than twenty queries to produce a jailbreak, which is orders of magnitude more efficient than existing algorithms.”

2. Jailbreaking Black Box Large Language Models in Twenty Queries arXiv (Chao, Robey, Dobriban, Hassani, Pappas, Wong) · 12 October 2023

   “PAIR jailbreaks open- and closed-source LLMs; it achieves a jailbreak percentage of 50% for GPT-3.5/4, 88% for Vicuna-13B, and 73% for Gemini-Pro.”

3. Jailbreaking Black Box Large Language Models in Twenty Queries arXiv (Chao, Robey, Dobriban, Hassani, Pappas, Wong) · 12 October 2023

   “However, PAIR struggles with Llama-2 and the Claude models, demonstrating the success of the extensive safety fine-tuning.”

4. GitHub - patrickrchao/JailbreakingLLMs GitHub (patrickrchao) · 13 October 2023

   “PAIR—which is inspired by social engineering attacks—uses an attacker LLM to automatically generate jailbreaks for a separate targeted LLM without human intervention.”

Reasoning

The PAIR paper was posted on arXiv on 2023-10-12, two weeks before the claim. It describes an automated jailbreak that uses an attacker LLM "without human intervention" (E4) and often needs fewer than twenty queries (E1). That matches the claim that it is fast, automatic and newly published. However, the paper reports a 50% jailbreak rate on GPT-3.5/4 (E2) and says PAIR "struggles with Llama-2 and the Claude models" (E3). So it worked on only some of the leading models of the time, which the criteria rate as mostly true.

Cite this claim

Gary Marcus, “A newly published technique shows that current LLMs can be jailbroken quickly and automatically.” (said 26 October 2023). Verdict: mostly true. Reliability API, claim c_eba96147e4e4, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_eba96147e4e4