Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
76 69 82 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
56 42 70 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 4 promises resolved, 5 needed for a score.
All types combined: 72 (likely 66 to 78, high confidence), from 93 claims: 76 factual claims, 17 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Dario Amodei's claims
4 claims
| Verdict | Kind | Said | Claim | Topic |
|---|---|---|---|---|
| Resolved false | Prediction | 10 Mar 2025 | Within three to six months of March 2025, AI will be writing 90 percent of code. Prediction · 10 Mar 2025 | AI timelines and forecasting |
| Resolved false | Prediction | 10 Mar 2025 | Within twelve months of March 2025, AI may be writing essentially all of the code. Prediction · 10 Mar 2025 | AI timelines and forecasting |
| Resolved false | Prediction | 11 Nov 2024 | Amodei guessed that within about 10 months AI models would probably get pretty close on typical real-world programming tasks, reaching at least 90%. Prediction · 11 Nov 2024 | AI capabilities |
| Resolved false | Prediction | 13 Oct 2022 | The situation where several companies can produce good reasoning models will rapidly cease as everyone moves further up the scaling curve. Prediction · 13 Oct 2022 | Economics and markets |
“But I would guess that in another 10 months, we’ll probably get pretty close. We’ll be at least 90%.”
Dario Amodei, Anthropic CEO, interviewee 11 November 2024 Transcript for Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity (Lex Fridman Podcast)
Amodei guessed that within about 10 months AI models would probably get pretty close on typical real-world programming tasks, reaching at least 90%.
Resolved false The record shows it did not happen or was not so.
What would count as true or false
In context, the measure of 'typical real-world programming tasks' is SWE-bench Verified (or the SWE-bench variant he discussed). The deadline is September 2025, about 10 months after 11 November 2024. TRUE: by 30 September 2025, an AI model or agent system has a published or officially verified score of 90% or more on SWE-bench Verified (lab announcement, the official SWE-bench leaderboard or a model system card). MOSTLY TRUE: a score of 90% or more is reached by the deadline, but only under a minor qualifier the claim did not depend on, such as an extensive-sampling or parallel-compute setting reported by the lab. MOSTLY FALSE: the best score by the deadline is between 88.2% and just under 90% (within 2% of the bound), or 90% is reached only after September 2025. FALSE: the best score by 30 September 2025 is below 88.2%.
Evidence
- Why we no longer evaluate SWE-bench Verified OpenAI · 23 February 2026
“from 74.9% to 80.9% in the last 6 months”
- Why SWE-bench Verified no longer measures frontier coding capabilities OpenAI · 23 February 2026
“After initial leaps, state-of-the-art progress on SWE-bench Verified has slowed, improving(opens in a new window) from 74.9% to 80.9% in the last 6 months.”
- Introducing Claude Sonnet 4.5 Anthropic · 29 September 2025 Own record
“We report 77.2%, which was averaged over 10 trials, no test-time compute, and 200K thinking budget on the full 500-problem SWE-bench Verified dataset.”
- Introducing Claude Sonnet 4.5 Anthropic · 29 September 2025 Own record
“This results in a score of 82.0% for Sonnet 4.5.”
- Claude Sonnet 4.5: Best coding model or just more marketing? R&D World · 30 September 2025
“In terms of claimed benchmarks, Sonnet 4.5 leads Anthropic’s own chart on SWE-bench Verified (n=500), posting 77.2% accuracy in standard runs and 82.0% with parallel test-time compute (the asterisked figure).”
- Introducing GPT-5 for developers OpenAI · 7 August 2025
“GPT‑5 is state-of-the-art (SOTA) across key coding benchmarks, scoring 74.9% on SWE-bench Verified and 88% on Aider polyglot.”
Reasoning
An OpenAI review from February 2026 says state-of-the-art progress on SWE-bench Verified slowed, rising only from 74.9% to 80.9% over the previous six months (E1, E2). None of the reported scores reaches or comes near 90% by September 2025. OpenAI puts state-of-the-art progress at 74.9% rising to 80.9% in the six months before February 2026 [E2], and Anthropic's highest reported figure for Sonnet 4.5 in September 2025 is 82.0% [E4]. Near the deadline, the top reported figures were GPT-5 at 74.9% (E6) and Claude Sonnet 4.5 at 77.2% in standard runs. Sonnet 4.5 reached 82.0% only with parallel test-time compute (E3, E4, E5). The best score by 30 September 2025 was well below the 88.2% threshold, so the claim resolves false.
Cite this claim
Dario Amodei, “Amodei guessed that within about 10 months AI models would probably get pretty close on typical real-world programming tasks, reaching at least 90%.” (said 11 November 2024). Verdict: resolved false. Reliability API, claim c_775bda437f17, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_775bda437f17
Claims without a verdict
- Unresolvable 157
- Not yet due 40
- Own records only 43
- Not researched 134
- Not checkable 864
Is a verdict here wrong, or do you speak for Dario Amodei? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.