Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
83 77 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
69 49 87 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
69 49 87
All types combined: 81 (likely 76 to 86, high confidence), from 87 claims: 77 factual claims, 5 predictions, 5 promises. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Sam Altman's claims
5 claims
| Verdict | Kind | Said | Claim | Topic |
|---|---|---|---|---|
| Mostly true | Prediction | 9 Aug 2025 | After GPT-5 launches there will be another big spike in demand that OpenAI will not be able to serve. Prediction · 9 Aug 2025 | AI compute, hardware and data |
| Mostly true | Prediction | 21 Dec 2024 | On many coding tasks, OpenAI's o3-mini will outperform o1 at a massive cost reduction. Prediction · 21 Dec 2024 | AI capabilities |
| Mostly true | Prediction | 5 Jan 2019 | By 2025, AGI will feel within reach to many people in the AI industry. Prediction · 5 Jan 2019 | AI timelines and forecasting |
| Resolved true | Prediction | 5 Jan 2019 | By 2025, net-gain nuclear fusion will be working at prototype scale. Prediction · 5 Jan 2019 | Technology |
| Resolved true | Prediction | 2017 | Agrees with Kellyanne Conway that Trump is a man of action who is just getting started. Prediction · 2017 | Other |
“on many coding tasks, o3-mini will outperform o1 at a massive cost reduction!”
Sam Altman, OpenAI CEO, on X 21 December 2024 Post by @sama on X, 2024-12-21 (X)
On many coding tasks, OpenAI's o3-mini will outperform o1 at a massive cost reduction.
Mostly true True in substance, wrong in a detail.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judge at o3-mini's public release (early 2025) and its first few months of evaluation. The two parts are: (a) on many coding tasks, o3-mini (at some reasoning-effort setting) scores better than o1; (b) it costs much less. True: OpenAI's published evaluations and independent coding benchmarks (for example Codeforces, SWE-bench Verified, LiveBench coding, Aider) show o3-mini beating o1 on several coding benchmarks, and o3-mini's API price or cost per task is several times lower (around 5x or more cheaper). Mostly true: both hold, but o3-mini beats o1 on only some coding benchmarks or the cost reduction is smaller than "massive" (for example 2–5x). Mixed: only one of (a) and (b) holds. Mostly false: o3-mini matches but rarely beats o1 on coding, even though it is cheaper. False: o3-mini does worse than o1 on most coding benchmarks and is not substantially cheaper, or o3-mini was never released.
Evidence
- OpenAI o3-mini, now available in LLM Simon Willison's Weblog · 31 January 2025
“o3-mini is priced at $1.10/million input tokens, $4.40/million output tokens—less than half the price of GPT-4o (currently $2.50/$10) and massively cheaper than o1 ($15/$60).”
- OpenAI Releases Reasoning Model o3-mini, Faster and More Accurate Than o1 InfoQ · 11 February 2025
“At high reasoning level, o3-mini outperforms the full o1 model on several benchmarks, including AIME 2024, GPQA Diamond, and SWE-bench Verified.”
- OpenAI o3-mini OpenAI · 31 January 2025 Own record
“o3‑mini with medium reasoning effort matches o1’s performance in math, coding, and science, while delivering faster responses.”
- OpenAI o3-mini, now available in LLM Simon Willison's Weblog · 31 January 2025
“the benchmarks in the o3-mini system card (PDF) aren’t a universal win for o3-mini across all categories.”
Reasoning
E1, an independent source, lists o3-mini's API price at $1.10/$4.40 per million input/output tokens against o1's $15/$60. That is about 13 times cheaper, which meets the "massive" cost-reduction part. On coding, E2 (independent) says o3-mini at high reasoning effort beat full o1 on SWE-bench Verified, but that is the only coding benchmark the evidence names as a win. E3 says o3-mini at medium effort only matches o1 on coding, and E4 says the system-card benchmarks are not a universal win for o3-mini. The cost part fully holds, but only one coding-benchmark win is documented rather than several, which fits the criteria's "mostly true".
Cite this claim
Sam Altman, “On many coding tasks, OpenAI's o3-mini will outperform o1 at a massive cost reduction.” (said 21 December 2024). Verdict: mostly true. Reliability API, claim c_40c3f94b025c, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_40c3f94b025c
Claims without a verdict
Is a verdict here wrong, or do you speak for Sam Altman? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.