ClaimsEliezer Yudkowsky

“Trying to train by gradient descent against that behavior, in that toy domain, is something I’d expect to produce not-particularly-coherent local patches to thought processes, which would break with near-certainty inside a superintelligence generalizing far outside the training distribution and thinking very different thoughts.”

Eliezer Yudkowsky, author 10 June 2022 AGI Ruin: A List of Lethalities (MIRI)

Prediction AI safety and alignment

Yudkowsky expects that training by gradient descent against that behavior in a toy domain would produce not-particularly-coherent local patches to thought processes, which would break with near-certainty inside a superintelligence generalizing far outside its training distribution.

Unresolvable The public record could not settle it.

Prediction: Statements about events outside the person's control, checked once they fell due: did they come true?

Said for an organisation they lead or work for; it counts on both records.

Reasoning

Claim about behavior of a hypothetical superintelligence; no horizon and cannot be tested.

Cite this claim

Eliezer Yudkowsky, “Yudkowsky expects that training by gradient descent against that behavior in a toy domain would produce not-particularly-coherent local patches to thought processes, which would break with near-certainty inside a superintelligence generalizing far outside its training distribution.” (said 10 June 2022). Verdict: unresolvable. Reliability API, claim c_8e0168857521, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_8e0168857521

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Eliezer Yudkowsky, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More prediction claims by Eliezer Yudkowsky

  1. Resolved false Prediction 6 April 2023
    “The current crop of people will not try it.”

    The current AI developers will not try the alignment approach he described (filtering training data to nice examples and training on it).

  2. Resolved true Prediction 6 April 2023
    “Alignment will not be solved in a few years.”

    AI alignment will not be solved within a few years.

  3. Resolved true Prediction 6 April 2023
    “even if GPT-5 doesn’t end the world, which I agree is like more than 50% of where my probability mass lies”

    GPT-5 will not end the world, with more than 50% of Yudkowsky's probability on that outcome.

  4. Mostly true Prediction 2008
    “Most AI researchers will not want to make unFriendly AIs.”

    Most AI researchers will not want to make unFriendly AIs.

  5. Mixed Prediction 5 March 2024
    “Beating GPT-4 on GI just forces OpenAI to release 4.5.”

    If a rival AI developer beats GPT-4 on general intelligence, OpenAI will be forced to release GPT-4.5.

All of them

Verdicts are made automatically; how verdicts are made.