ClaimsEliezer Yudkowsky

“Now, one of the conditions of this little meet-up was that no one would ever say what went on in there. Why did I do that? Because I was trying to make a point about what I would now call cognitive uncontainability.”

Eliezer Yudkowsky, MIRI senior researcher, podcast guest 28 February 2018 Sam Harris and Eliezer Yudkowsky on "AI: Racing Toward the Brink" (MIRI)

Factual AI safety and alignment

One condition of the AI-box experiment was that no one would ever say what went on inside it.

Mostly true True in substance, wrong in a detail.

Factual: Statements about the present or the past, checked against the record: did they hold up?

The evidence is the person's or their organisation's own records.

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

True: Yudkowsky's original AI-box experiment rules and reports (e.g. his 2002 AI-Box Experiment page and the SL4 mailing-list posts) show that the participants agreed not to disclose what was said or done during the experiment. Mostly true: there was a non-disclosure condition, but with minor exceptions, e.g. the outcome could be reported or the condition could be waived by mutual consent. Mixed: non-disclosure applied in some runs but not as a general condition. Mostly false: confidentiality was only informal or optional. False: the rules contained no such condition and transcripts were meant to be published.

Set before the research began.

Evidence (4)

  1. E1 The AI-Box Experiment: yudkowsky.net (Eliezer Yudkowsky's personal site) · 4 September 2020 Own record

    “Regardless of the result, neither party shall ever reveal anything of what goes on within the AI-Box experiment except the outcome.”

  2. E2 How To Win The AI Box Experiment (Sometimes) — LessWrong LessWrong (post by pinkgothic) · 12 September 2015 Own record

    “Regardless of the result, neither party shall ever reveal anything of what goes on within the AI-Box experiment except the outcome.”

  3. E3 The "AI Box" experiment SL4 mailing list (archive) · 8 March 2002 Own record

    “One of the conditions of the test is that neither of us reveal what went on inside”

  4. E4 AI-Box Experiment 2: Yudkowsky and McFadzean SL4 mailing list (archive) · 4 July 2002 Own record

    “And you do understand that, under the terms of the arrangement, whether you let me out or not, you can never tell anyone what went on in there?”

Reasoning

Yudkowsky's AI-Box Experiment protocol page [E1] states that "neither party shall ever reveal anything of what goes on within the AI-Box experiment except the outcome." A LessWrong post [E2] quotes the same rule. His 2002 SL4 posts [E3, E4] also describe non-disclosure of what went on inside as a condition of the test. So the non-disclosure condition existed, but the protocol explicitly allowed the outcome to be revealed. The resolution criteria list that kind of exception under "mostly true".

Cite this claim

Eliezer Yudkowsky, “One condition of the AI-box experiment was that no one would ever say what went on inside it.” (said 28 February 2018). Verdict: mostly true. Reliability API, claim c_1accca44d1ef, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_1accca44d1ef

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Eliezer Yudkowsky, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Eliezer Yudkowsky

  1. Mixed Factual 20 April 2026
    “I was observing that obvious-seeming hypothesis myself, sometime around the time of GPT 3.5.”

    Yudkowsky himself observed the persona-selection-like hypothesis around the time of GPT 3.5.

  2. Mostly true Factual 20 April 2026
    “The test-passing preference stands on its own once formed.”

    Once formed through RL training, an LLM's test-passing preference stands on its own and is not overridden by its inference of what the user wants.

  3. Mostly true Factual 20 April 2026
    “LLMs surely have neural circuits they learned to model text streams from fictional and nonfictional personas that are not lying, deceiving, cheating.”

    LLMs surely have neural circuits they learned to model text streams from fictional and nonfictional personas that are not lying, deceiving…

  4. Mostly true Factual 20 April 2026
    “The predict-as-if-honest circuitry is contextual, invoked to predict some streams of text but not others.”

    LLMs' predict-as-if-honest circuitry is contextual, invoked for predicting some text streams but not others.

  5. Mostly true Factual 20 April 2026
    “Presumably, somewhere along the way LLMs learn a dispreference for tests that don't pass, independently of how that gets prevented or whether the new plan does what the user really wanted.”

    Presumably, at some point during RL training on code-passing tasks, LLMs learn a dispreference for failing tests, independent of how that is avoided or whether the user's real goal is met.

All of them

Verdicts are made automatically; how verdicts are made.