ClaimsDario Amodei

“And I would suspect that if we can get to 90, 95% that it will represent ability to autonomously do a significant fraction of software engineering tasks.”

Dario Amodei, CEO of Anthropic 11 November 2024 Transcript for Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity (Lex Fridman Podcast)

Prediction AI capabilities

Dario Amodei said he would suspect that if AI models reach 90-95% on the software-engineering benchmark, that will represent the ability to autonomously do a significant fraction of software engineering tasks.

Unresolvable The public record could not settle it.

Prediction: Statements about events outside the person's control, checked once they fell due: did they come true?

Reasoning

A conditional interpretive claim about what a benchmark score represents is not testable against a record.

Cite this claim

Dario Amodei, “Dario Amodei said he would suspect that if AI models reach 90-95% on the software-engineering benchmark, that will represent the ability to autonomously do a significant fraction of software engineering tasks.” (said 11 November 2024). Verdict: unresolvable. Reliability API, claim c_c9d89e277af8, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_c9d89e277af8

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Dario Amodei, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More prediction claims by Dario Amodei

  1. Mostly false Prediction 23 September 2026
    “We believe AI for biology is on a similar exponential trend.”

    Amodei said that "we" believe AI for biology is on an exponential trend similar to the one he described for AI in mathematics.

  2. Resolved true Prediction 28 February 2025
    “for coding, we're going to see very serious things by the end of 2025,”

    By the end of 2025, AI will do very serious things in coding.

  3. Resolved false Prediction 11 November 2024
    “But I would guess that in another 10 months, we’ll probably get pretty close. We’ll be at least 90%.”

    Amodei guessed that within about 10 months AI models would probably get pretty close on typical real-world programming tasks, reaching at least 90%.

  4. Mostly false Prediction 11 November 2024
    “in 10 months we’ve gone from 3% to 50% on this task. And I think in another year we’ll probably be at 90%.”

    Amodei said that, after progress from 3% to 50% on SWE-bench in 10 months, state-of-the-art models would probably be at about 90% in another year, while adding that he did not know and it might be less.

  5. Mostly true Prediction 11 November 2024
    “I would guess a year from now, the models can do this very, very reliably.”

    Amodei guessed that by about November 2025, AI models would be able to perform computer-use tasks very, very reliably.

All of them

Verdicts are made automatically; how verdicts are made.