ClaimsEliezer Yudkowsky

“Most AI researchers will not want to make unFriendly AIs.”

Eliezer Yudkowsky, author of chapter 2008 Artificial Intelligence as a Positive and Negative Factor in Global Risk (MIRI)

Prediction AI safety and alignment

Most AI researchers will not want to make unFriendly AIs.

Mostly true True in substance, wrong in a detail.

Prediction: Statements about events outside the person's control, checked once they fell due: did they come true?

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

Read 'unFriendly AI' as AI that is harmful to, or misaligned with, human interests. True: surveys or other systematic evidence show that a clear majority of AI researchers do not intend or want to build harmful or misaligned AI, for example by preferring beneficial or safe outcomes. Mostly true: the majority holds, but the evidence covers only part of the field or is indirect. Mixed: the evidence is split, or roughly half express indifference to or acceptance of building such systems. Mostly false: a substantial plurality say they are willing to build AI they consider likely misaligned or harmful. False: a majority of AI researchers say they want or intend to build AI that is harmful to humans.

Set before the research began.

Evidence (4)

  1. E1 2022 Expert Survey on Progress in AI [AI Impacts Wiki] AI Impacts · 4 August 2022

    “69% of respondents think society should prioritize AI safety research more or much more, up from 49% in 2016.”
  2. E2 [PDF] Ethics and Governance of Artificial Intelligence: A Survey of Machine Learning Researchers (Extended Abstract) IJCAI-22 (authors from Syracuse University, Centre for the Governance of AI, Council on Foreign Relations, University of Pennsylvania, DeepMind) · 2022

    “When asked about prioritizing AI safety an overwhelming majority of our respondents (68%) say that the field should be prioritized more than at present.”
  3. E3 Thousands of AI Authors on the Future of AI arXiv (Grace et al.; AI Impacts)

    “A large majority of respondents thought that AI safety research should be prioritized more than it currently is.”
  4. E4 [PDF] Ethics and Governance of Artificial Intelligence: A Survey of Machine Learning Researchers (Extended Abstract) IJCAI-22 (authors from Syracuse University, Centre for the Governance of AI, Council on Foreign Relations, University of Pennsylvania, DeepMind) · 2022

    “31% of researchers indicate that they would resign or threaten to resign from their jobs, and 25% indicate that they would speak out publicly to the media or online, if their organization decided to work on lethal autonomous weapons.”

Reasoning

Independent surveys of AI researchers show that clear majorities want AI safety research prioritised more. The 2022 AI Impacts survey puts this at 69% [E1], an ML researcher survey at 68% [E2], and the later survey of thousands of AI authors calls it "a large majority" [E3]. A meaningful share of ML researchers also say they would resign or speak out if their organisation worked on lethal autonomous weapons [E4]. This points to a majority preferring safe, beneficial outcomes. But none of the surveys asks directly whether researchers want to build misaligned or harmful AI, so the evidence is indirect, which fits "mostly true" under the criteria.

Cite this claim

Eliezer Yudkowsky, “Most AI researchers will not want to make unFriendly AIs.” (said 2008). Verdict: mostly true. Reliability API, claim c_f7fa4f6e3ac3, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_f7fa4f6e3ac3

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Eliezer Yudkowsky, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More prediction claims by Eliezer Yudkowsky

  1. Resolved false Prediction 6 April 2023
    “The current crop of people will not try it.”

    The current AI developers will not try the alignment approach he described (filtering training data to nice examples and training on it).

  2. Resolved true Prediction 6 April 2023
    “Alignment will not be solved in a few years.”

    AI alignment will not be solved within a few years.

  3. Resolved true Prediction 6 April 2023
    “even if GPT-5 doesn’t end the world, which I agree is like more than 50% of where my probability mass lies”

    GPT-5 will not end the world, with more than 50% of Yudkowsky's probability on that outcome.

  4. Mixed Prediction 5 March 2024
    “Beating GPT-4 on GI just forces OpenAI to release 4.5.”

    If a rival AI developer beats GPT-4 on general intelligence, OpenAI will be forced to release GPT-4.5.

  5. Mostly false Prediction 6 April 2023
    “So probably Paul’s going to win, but slight moral victory.”

    Paul Christiano will probably win his forecast disagreement with Yudkowsky on an AI getting IMO gold by 2025, with prediction-market odds then running around 30%.

All of them

Verdicts are made automatically; how verdicts are made.