Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
79 72 85 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
80 72 88 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Gary Marcus's claims
52 claims
“But will we see a mindblowing GPT this year? I doubt it.”
Gary Marcus, author, Marcus on AI 31 March 2024 When Will the GenAI Bubble Burst? (Marcus on AI)
Marcus doubted that a mindblowing GPT model would be released in 2024.
Mostly true True in substance, wrong in a detail.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
The claim is that no 'mindblowing' GPT model would be released during calendar 2024. A GPT model means one from OpenAI's GPT or successor line (for example GPT-4o or o1). 'Mindblowing' means broadly treated by mainstream tech press and AI experts as a step-change comparable to the jump from GPT-3.5 to GPT-4 or larger, not just an incremental improvement. True: no model released in 2024 was broadly characterised that way, and 2024 releases were widely described as incremental. Mostly true: a 2024 release drew some 'breakthrough' framing, but the prevailing assessment was incremental or narrow. Mixed: credible opinion is substantially split on whether a 2024 release was such a leap. Mostly false: a 2024 release was widely, though not universally, regarded as a major leap. False: a GPT model released in 2024 was broadly hailed as a GPT-4-scale leap. A model announced but not released by 2024-12-31 does not count.
Evidence
- Hello GPT-4o OpenAI · 13 May 2024
“As measured on traditional benchmarks, GPT‑4o achieves GPT‑4 Turbo-level performance on text, reasoning, and coding intelligence, while setting new high watermarks on multilingual, audio, and vision capabilities.”
- 12 days of OpenAI: The Ars Technica recap Ars Technica · 12 December 2024
“The company reported that the model operates faster than its preview version and reduces major errors by 34 percent on complex real-world questions.”
- OpenAI's o3 model aced a test of AI reasoning – but it's still not AGI | New Scientist New Scientist · 20 December 2024
“The o3 model’s high score comes as the tech industry and AI researchers have been reckoning with a slower pace of progress in the latest AI models for 2024, compared with the initial explosive developments of 2023.”
- OpenAI o3 Breakthrough High Score on ARC-AGI-Pub | ARC Prize ARC Prize · 20 December 2024
“This is a surprising and important step-function increase in AI capabilities, showing novel task adaptation ability never seen before in the GPT-family models.”
Reasoning
OpenAI's own launch post for GPT-4o says its text, reasoning and coding performance is at the GPT-4 Turbo level, with gains in multilingual, audio and vision (E1). That describes an incremental step, not a GPT-4-scale leap. Ars Technica reports the company said the fully released model is faster than its preview version and reduces major errors by 34 percent on complex real-world questions (E2). New Scientist says the industry and AI researchers were "reckoning with a slower pace of progress" in 2024's latest models (E3). The ARC Prize calls an o3 result a "step-function increase in AI capabilities" (E4). New Scientist's report on the same result says o3 is still not AGI and notes a slower pace of progress in 2024's AI models (E3). This is breakthrough framing from one source alongside a more measured assessment. The evidence I have on how the press broadly received 2024 releases is thin, so I rate this mostly true rather than fully true.
Cite this claim
Gary Marcus, “Marcus doubted that a mindblowing GPT model would be released in 2024.” (said 31 March 2024). Verdict: mostly true. Reliability API, claim c_54be66ed1c03, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_54be66ed1c03
Claims without a verdict
Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.