As of 2023, the model can be very good at coding but can't quite prove the prime number theorem yet.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judge as of 2023-08-08 against the leading AI models publicly known then. The claim has two parts: (a) such models were very good at coding, and (b) no AI model could yet prove the prime number theorem. True: evidence from that time shows leading models (e.g. GPT-4, Claude 2) performed strongly on coding benchmarks or in practical coding use, and no AI system had produced a proof of the prime number theorem, whether autonomously or as a formal proof generated mainly by AI. Mostly true: both parts hold broadly, with a minor qualification, such as coding ability being strong but uneven. Mixed: one part holds and the other is clearly contradicted. Mostly false: the coding part is a significant overstatement for models of that time AND there is a substantial qualification to the second part. False: evidence shows an AI system had already proved the prime number theorem by August 2023 and models were not good at coding. Human-written formalisations of the theorem in proof assistants do not count as an AI proving it.
Set before the research began.
Evidence (4)
-
E1 Claude 2 Anthropic · 11 July 2023 Own record
“Claude 2 scored a 71.2% up from 56.0% on the Codex HumanEval, a Python coding test.”
-
E2 Model Card and Evaluations for Claude Models - Anthropic Anthropic Own record
“Previous models lagged behind the state-of-the-art on coding tasks.”
-
E3 GPT-4 OpenAI · 14 March 2023
“We’ve also been using GPT-4 internally, with great impact on functions like support, sales, content moderation, and programming.”
-
E4 List of mathematical discoveries made with artificial intelligence - Wikipedia Wikipedia · 13 August 2026
“In September 2025, Math Inc.'s Gauss agent formalized the strong prime number theorem in Lean, producing approximately 25,000 lines of code and formally proving more than 1,000 theorems.”
Reasoning
The evidence supports the coding part, with GPT-4 used for programming (E3) and Claude 2's HumanEval score (E1). On the prime number theorem, E4 records only an AI formalisation in September 2025 and does not show whether any AI proof existed by August 2023, so the second part is not settled.
Cite this claim
Dario Amodei, “As of 2023, the model can be very good at coding but can't quite prove the prime number theorem yet.” (said 8 August 2023). Verdict: unresolvable. Reliability API, claim c_1c512934aa79, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_1c512934aa79
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Dario Amodei
-
Mixed
“It’s easy to dismiss this as a one-off or curiosity, but we’ve repeatedly seen a pattern where AI performance in new intellectual domains goes from weak to superhuman in a matter of a few years.”
-
Mixed
“we were measuring these trends back when models cost $1,000 or $10,000 to train”
Amodei's group measured AI scaling trends when models cost $1,000 or $10,000 to train.
-
Mixed
“a lot of the other reasoning models in the market are trained primarily on math and competition coding,”
-
Mostly true
“I do think AI coding is moving the fastest of all the other areas.”
Amodei said he thinks AI coding is moving the fastest of all the areas of AI.
-
Mostly true
“Because when it knows that it doesn't get any gain out of thinking further, it doesn't think for longer.”
Verdicts are made automatically; how verdicts are made.