ClaimsEliezer Yudkowsky

“it broke alignment after it turned general, and did so relatively late in the game of that general intelligence accumulating capability and knowledge”

Eliezer Yudkowsky, author 10 June 2022 AGI Ruin: A List of Lethalities (MIRI)

Factual AI safety and alignment

When natural selection produced general intelligence in humans, alignment with the outer optimization loop broke after the intelligence became general, relatively late in its accumulation of capability and knowledge.

Unresolvable The public record could not settle it.

Factual: Statements about the present or the past, checked against the record: did they hold up?

Said for an organisation they lead or work for; it counts on both records.

Reasoning

Interpretive claim about evolutionary history and 'alignment breaking late' is not settleable by records.

Cite this claim

Eliezer Yudkowsky, “When natural selection produced general intelligence in humans, alignment with the outer optimization loop broke after the intelligence became general, relatively late in its accumulation of capability and knowledge.” (said 10 June 2022). Verdict: unresolvable. Reliability API, claim c_61d36da20616, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_61d36da20616

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Eliezer Yudkowsky, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Eliezer Yudkowsky

  1. Mixed Factual 20 April 2026
    “I was observing that obvious-seeming hypothesis myself, sometime around the time of GPT 3.5.”

    Yudkowsky himself observed the persona-selection-like hypothesis around the time of GPT 3.5.

  2. Mostly true Factual 20 April 2026
    “The test-passing preference stands on its own once formed.”

    Once formed through RL training, an LLM's test-passing preference stands on its own and is not overridden by its inference of what the user wants.

  3. Mostly true Factual 20 April 2026
    “LLMs surely have neural circuits they learned to model text streams from fictional and nonfictional personas that are not lying, deceiving, cheating.”

    LLMs surely have neural circuits they learned to model text streams from fictional and nonfictional personas that are not lying, deceiving…

  4. Mostly true Factual 20 April 2026
    “The predict-as-if-honest circuitry is contextual, invoked to predict some streams of text but not others.”

    LLMs' predict-as-if-honest circuitry is contextual, invoked for predicting some text streams but not others.

  5. Mostly true Factual 20 April 2026
    “Presumably, somewhere along the way LLMs learn a dispreference for tests that don't pass, independently of how that gets prevented or whether the new plan does what the user really wanted.”

    Presumably, at some point during RL training on code-passing tasks, LLMs learn a dispreference for failing tests, independent of how that is avoided or whether the user's real goal is met.

All of them

Verdicts are made automatically; how verdicts are made.