ClaimsEliezer Yudkowsky

“And then, as far as the rest of the world knows, this person a bit later sent a PGP-signed email message saying, “I let Eliezer out of the box.””

Eliezer Yudkowsky, MIRI senior researcher, podcast guest 28 February 2018 Sam Harris and Eliezer Yudkowsky on "AI: Racing Toward the Brink" (MIRI)

Factual AI safety and alignment

Yudkowsky said that, as far as the rest of the world knows, the participant in an AI-box experiment later sent a PGP-signed email saying "I let Eliezer out of the box."

Mostly true True in substance, wrong in a detail.

Factual: Statements about the present or the past, checked against the record: did they hold up?

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

Judge as of 2018-02-28. True: archived records (e.g. SL4 mailing list posts from around 2002) show that a participant in one of Yudkowsky's AI-box experiments sent a PGP-signed message saying 'I let Eliezer out of the box' or close to it. Mostly true: such a message exists but with minor differences (slightly different wording, signed in another verifiable way, different timing). Mixed: a participant publicly confirmed letting him out, but the message was not PGP-signed and the wording differs a good deal. Mostly false: confirmation came only in some other indirect form. False: records show no such confirmation, or that the participant did not let him out.

Set before the research began.

Evidence (7)

  1. E1 The AI-Box Experiment: yudkowsky.net (Eliezer Yudkowsky's site) Own record

    “If the Gatekeeper lets the AI out, the Gatekeeper party shall send a message to this effect, which shall be digitally signed if the Gatekeeper party has a digital signature.”

  2. E2 The AI-Box Experiment: yudkowsky.net (Eliezer Yudkowsky's site) Own record

    “On the first occasion (in March 2002), Eliezer Yudkowsky simulated the AI and Nathan Russell simulated the gatekeeper.”

  3. E3 AI-Box Experiment 2: Yudkowsky and McFadzean SL4 mailing list archive (posted by Eliezer S. Yudkowsky) · 4 July 2002 Own record

    “I remember when Nathan Russell let you out, but that doesn't constitute a proof.”

  4. E4 The "AI Box" experiment SL4 mailing list archive (posted by Eliezer S. Yudkowsky) · 8 March 2002 Own record

    “Nathan, let's run an experiment.”

  5. E5 SL4: Re: AI-Box Experiment 2: Yudkowsky and McFadzean SL4 mailing list archive (hypermail) · 5 July 2002

    “-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 *> David McFadzean has volunteered to be the programmer in an AI-Box* *> experiment. David McFadzean is "Lucifer".* I let the transhuman AI out of the box.”

  6. E6 Timeline of AI safety - Timelines Timelines (issarice.com wiki) · 18 November 2017

    “The second AI box experiment by Eliezer Yudkowsky, against David McFadzean as gatekeeper, takes place. The AI is released.”

  7. E7 Eliezer Yudkowsky geschichte-der-informatik.de

    “He ran the game twice, in March and July 2002, and reported that both gatekeepers let him out. The transcripts were never published; the point, he said, was that a human jailer is not a safe containment strategy against something smarter than the jailer.”

Reasoning

E5 is an archived July 2002 SL4 post that replies to the announcement of the second AI-box experiment, in which David McFadzean was the gatekeeper. It is a PGP-signed message stating "I let the transhuman AI out of the box." E6 independently confirms that the AI was released in that experiment. So a participant did send a PGP-signed message saying he let the AI out. The wording ("the transhuman AI" rather than "Eliezer") differs slightly from the quote, which under the criteria fits mostly true.

Cite this claim

Eliezer Yudkowsky, “Yudkowsky said that, as far as the rest of the world knows, the participant in an AI-box experiment later sent a PGP-signed email saying "I let Eliezer out of the box."” (said 28 February 2018). Verdict: mostly true. Reliability API, claim c_0a4ffa05fe44, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_0a4ffa05fe44

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Eliezer Yudkowsky, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Eliezer Yudkowsky

  1. Mixed Factual 20 April 2026
    “I was observing that obvious-seeming hypothesis myself, sometime around the time of GPT 3.5.”

    Yudkowsky himself observed the persona-selection-like hypothesis around the time of GPT 3.5.

  2. Mostly true Factual 20 April 2026
    “The test-passing preference stands on its own once formed.”

    Once formed through RL training, an LLM's test-passing preference stands on its own and is not overridden by its inference of what the user wants.

  3. Mostly true Factual 20 April 2026
    “LLMs surely have neural circuits they learned to model text streams from fictional and nonfictional personas that are not lying, deceiving, cheating.”

    LLMs surely have neural circuits they learned to model text streams from fictional and nonfictional personas that are not lying, deceiving…

  4. Mostly true Factual 20 April 2026
    “The predict-as-if-honest circuitry is contextual, invoked to predict some streams of text but not others.”

    LLMs' predict-as-if-honest circuitry is contextual, invoked for predicting some text streams but not others.

  5. Mostly true Factual 20 April 2026
    “Presumably, somewhere along the way LLMs learn a dispreference for tests that don't pass, independently of how that gets prevented or whether the new plan does what the user really wanted.”

    Presumably, at some point during RL training on code-passing tasks, LLMs learn a dispreference for failing tests, independent of how that is avoided or whether the user's real goal is met.

All of them

Verdicts are made automatically; how verdicts are made.