Amodei guessed that within about 10 months AI models would probably get pretty close on typical real-world programming tasks, reaching at least 90%.

In context, the measure of 'typical real-world programming tasks' is SWE-bench Verified (or the SWE-bench variant he discussed). The deadline is September 2025, about 10 months after 11 November 2024. TRUE: by 30 September 2025, an AI model or agent system has a published or officially verified score of 90% or more on SWE-bench Verified (lab announcement, the official SWE-bench leaderboard or a model system card). MOSTLY TRUE: a score of 90% or more is reached by the deadline, but only under a minor qualifier the claim did not depend on, such as an extensive-sampling or parallel-compute setting reported by the lab. MOSTLY FALSE: the best score by the deadline is between 88.2% and just under 90% (within 2% of the bound), or 90% is reached only after September 2025. FALSE: the best score by 30 September 2025 is below 88.2%.

Set before the research began.

1. E1 Why we no longer evaluate SWE-bench Verified OpenAI · 23 February 2026

   “from 74.9% to 80.9% in the last 6 months”

2. E2 Why SWE-bench Verified no longer measures frontier coding capabilities OpenAI · 23 February 2026

   “After initial leaps, state-of-the-art progress on SWE-bench Verified has slowed, improving(opens in a new window) from 74.9% to 80.9% in the last 6 months.”

3. E3 Introducing Claude Sonnet 4.5 Anthropic · 29 September 2025 Own record

   “We report 77.2%, which was averaged over 10 trials, no test-time compute, and 200K thinking budget on the full 500-problem SWE-bench Verified dataset.”

4. E4 Introducing Claude Sonnet 4.5 Anthropic · 29 September 2025 Own record

   “This results in a score of 82.0% for Sonnet 4.5.”

5. E5 Claude Sonnet 4.5: Best coding model or just more marketing? R&D World · 30 September 2025

   “In terms of claimed benchmarks, Sonnet 4.5 leads Anthropic’s own chart on SWE-bench Verified (n=500), posting 77.2% accuracy in standard runs and 82.0% with parallel test-time compute (the asterisked figure).”

6. E6 Introducing GPT-5 for developers OpenAI · 7 August 2025

   “GPT‑5 is state-of-the-art (SOTA) across key coding benchmarks, scoring 74.9% on SWE-bench Verified and 88% on Aider polyglot.”

An OpenAI review from February 2026 says state-of-the-art progress on SWE-bench Verified slowed, rising only from 74.9% to 80.9% over the previous six months (E1, E2). None of the reported scores reaches or comes near 90% by September 2025. OpenAI puts state-of-the-art progress at 74.9% rising to 80.9% in the six months before February 2026 [E2], and Anthropic's highest reported figure for Sonnet 4.5 in September 2025 is 82.0% [E4]. Near the deadline, the top reported figures were GPT-5 at 74.9% (E6) and Claude Sonnet 4.5 at 77.2% in standard runs. Sonnet 4.5 reached 82.0% only with parallel test-time compute (E3, E4, E5). The best score by 30 September 2025 was well below the 88.2% threshold, so the claim resolves false.