ClaimsEliezer Yudkowsky

“an AGI strongly optimizing on that signal will kill you, because the sensory reward signal was not a ground truth about alignment (as seen by the operators).”

Eliezer Yudkowsky, author 10 June 2022 AGI Ruin: A List of Lethalities (MIRI)

Prediction AI safety and alignment

Even if an AGI ends up perfectly inner-aligned on a sensory reward signal, an AGI strongly optimizing on that signal "will kill you", because the sensory reward signal was not a ground truth about alignment as seen by the operators.

Unresolvable The public record could not settle it.

Prediction: Statements about events outside the person's control, checked once they fell due: did they come true?

Said for an organisation they lead or work for; it counts on both records.

Reasoning

Prediction about a hypothetical AGI with no horizon; untestable.

Cite this claim

Eliezer Yudkowsky, “Even if an AGI ends up perfectly inner-aligned on a sensory reward signal, an AGI strongly optimizing on that signal "will kill you", because the sensory reward signal was not a ground truth about alignment as seen by the operators.” (said 10 June 2022). Verdict: unresolvable. Reliability API, claim c_f7905d2dc581, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_f7905d2dc581

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Eliezer Yudkowsky, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More prediction claims by Eliezer Yudkowsky

  1. Resolved false Prediction 6 April 2023
    “The current crop of people will not try it.”

    The current AI developers will not try the alignment approach he described (filtering training data to nice examples and training on it).

  2. Resolved true Prediction 6 April 2023
    “Alignment will not be solved in a few years.”

    AI alignment will not be solved within a few years.

  3. Resolved true Prediction 6 April 2023
    “even if GPT-5 doesn’t end the world, which I agree is like more than 50% of where my probability mass lies”

    GPT-5 will not end the world, with more than 50% of Yudkowsky's probability on that outcome.

  4. Mostly true Prediction 2008
    “Most AI researchers will not want to make unFriendly AIs.”

    Most AI researchers will not want to make unFriendly AIs.

  5. Mixed Prediction 5 March 2024
    “Beating GPT-4 on GI just forces OpenAI to release 4.5.”

    If a rival AI developer beats GPT-4 on general intelligence, OpenAI will be forced to release GPT-4.5.

All of them

Verdicts are made automatically; how verdicts are made.