Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
76 70 82 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
60 42 78
All types combined: 74 (likely 68 to 80, high confidence), from 79 claims: 71 factual claims, 8 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Eliezer Yudkowsky's claims
“Beating GPT-4 on GI just forces OpenAI to release 4.5.”
Eliezer Yudkowsky, writing on X 5 March 2024 Post by @ESYudkowsky on X, 2024-03-05 (X)
If a rival AI developer beats GPT-4 on general intelligence, OpenAI will be forced to release GPT-4.5.
Mixed Partly true and partly false in roughly equal measure.
What would count as true or false
This is a conditional prediction. The condition is met when a rival developer releases a model widely judged (by benchmark leaderboards or broad expert or press consensus) to be better than GPT-4 at general capability. True: the condition is met and OpenAI then releases a model branded GPT-4.5, with the release reported or reasonably presented as a response to losing the lead. Mostly true: OpenAI answers the rival with a clearly upgraded GPT-4 successor under a different name, or releases GPT-4.5 but the causal link to the rival is only weakly shown. Mixed: GPT-4.5 is released eventually, but long after the condition was met, with no evident link to competitive pressure. Mostly false: OpenAI releases no GPT-4.5 and no prompt competitive successor for a long period while rivals stay ahead. False: rivals clearly surpass GPT-4, and OpenAI never releases a GPT-4.5 and makes no competitive release in response. If no rival ever surpasses GPT-4, the claim is unresolvable.
Evidence
- The AI wars heat up with Claude 3, claimed to have “near-human” abilities Ars Technica · 4 March 2024
“According to Anthropic, Claude 3 Opus beats GPT-4 on 10 AI benchmarks, including MMLU (undergraduate level knowledge), GSM8K (grade school math), HumanEval (coding), and the colorfully named HellaSwag (common knowledge).”
- Simon Willison on gpt-4 Simon Willison's Weblog
““The king is dead”—Claude 3 surpasses GPT-4 on Chatbot Arena for the first time.”
- Introducing GPT-4.5 OpenAI · 27 February 2025
“GPT‑4.5 is an example of scaling unsupervised learning by scaling up compute and data, along with architecture and optimization innovations.”
- OpenAI launches GPT-4.5, its largest model to date | TechCrunch TechCrunch · 1 March 2025
“GPT-4.5 is OpenAI’s largest model to date, trained using more computing power and data than any of the company’s previous releases.”
Reasoning
The condition appears to have been met: Ars Technica reported that Claude 3 Opus beat GPT-4 on 10 benchmarks (E1), and Simon Willison's weblog noted that Claude 3 surpassed GPT-4 on Chatbot Arena (E2). OpenAI released a model branded GPT-4.5 on 2025-02-27 (E3, E4), nearly a year after the condition was met. The quoted passages from OpenAI's announcement and TechCrunch's coverage describe GPT-4.5 in terms of scaling compute and data and do not mention a rival model or a competitive response (E3, E4). That matches the criteria's "mixed" case: an eventual release with no evident link to competitive pressure.
Cite this claim
Eliezer Yudkowsky, “If a rival AI developer beats GPT-4 on general intelligence, OpenAI will be forced to release GPT-4.5.” (said 5 March 2024). Verdict: mixed. Reliability API, claim c_51a36d84a2e5, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_51a36d84a2e5
Claims without a verdict
Is a verdict here wrong, or do you speak for Eliezer Yudkowsky? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.