Amodei said that, after progress from 3% to 50% on SWE-bench in 10 months, state-of-the-art models would probably be at about 90% in another year, while adding that he did not know and it might be less.
Mostly false False in substance, right in a detail.
Prediction: Statements about events outside the person's control, checked once they fell due: did they come true?
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
The benchmark is SWE-bench Verified, the variant on which state-of-the-art scores were about 50% in November 2024. The test is the best publicly reported score from a state-of-the-art model or agent system by the end of November 2025. 'About 90%' is a rough figure. True: the best reported score is 87% or higher. Mostly true: 84% to under 87%. Mostly false: 75% to under 84% (real progress, but short of the predicted level). False: below 75%. A score first reached after November 2025 does not count towards the claim.
Set before the research began.
Evidence (3)
-
E1 Claude Opus 4.5 vs Sonnet 4.5: Pricing Revolution & Performance Comparison | Anthropic Vertu · 25 November 2025
“Opus 4.5 achieved 80.9% on SWE-bench Verified, outperforming Sonnet 4.5's 77.2% and Opus 4.1's 74.5%.”
-
E2 Introducing Claude Sonnet 4.5 Anthropic · 29 September 2025 Own record
“This results in a score of 82.0% for Sonnet 4.5.”
-
E3 SWE-bench Verified | AI Wiki AI Wiki · 18 March 2026
“The public state-of-the-art then rose to 49% by October 2024, past 80% in November 2025,”
Reasoning
E1, an independent source dated 2025-11-25, reports that Claude Opus 4.5 scored 80.9% on SWE-bench Verified. E3, another independent source, says the public state of the art went past 80% in November 2025. Anthropic's own announcement (E2), which is not independent of the subject, gives a score of 82.0% for Sonnet 4.5, though the quoted passage does not name the benchmark variant. All of these best reported scores by the end of November 2025 fall between 75% and 84%. That is real progress from about 50%, but short of the predicted ~90%, so it falls in the mostly-false band.
Cite this claim
Dario Amodei, “Amodei said that, after progress from 3% to 50% on SWE-bench in 10 months, state-of-the-art models would probably be at about 90% in another year, while adding that he did not know and it might be less.” (said 11 November 2024). Verdict: mostly false. Reliability API, claim c_3ee9cbce30d1, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_3ee9cbce30d1
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More prediction claims by Dario Amodei
-
Mostly false
“We believe AI for biology is on a similar exponential trend.”
-
Resolved true
“for coding, we're going to see very serious things by the end of 2025,”
By the end of 2025, AI will do very serious things in coding.
-
Resolved false
“But I would guess that in another 10 months, we’ll probably get pretty close. We’ll be at least 90%.”
-
Mostly true
“I would guess a year from now, the models can do this very, very reliably.”
-
Mostly true
“We're gonna be fine. The-- the impact of this designation is fairly small.”
Verdicts are made automatically; how verdicts are made.