Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
79 72 85 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
80 72 88 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Gary Marcus's claims
42 claims
“No single system will solve more than 4 of the AI 2027 Marcus-Brundage tasks by the end of 2025.”
Gary Marcus, Substack author 1 January 2025 25 AI Predictions for 2025, from Marcus on AI (Marcus on AI)
No single AI system will solve more than 4 of the AI 2027 Marcus-Brundage tasks by the end of 2025.
Resolved true The record shows it happened or was so.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judged at the end of 2025. The tasks are the ten AI-2027 tasks from the Marcus-Brundage bet. True: as of 31 December 2025, no single AI system had credibly completed 5 or more of those tasks. False: some single system had credibly completed 5 or more by that date. Where it is disputed whether a task was solved, follow the assessment of Marcus and Brundage if they agree, and otherwise follow the weight of independent assessment. Only if the fifth task's status remains genuinely contested is mostly false or mostly true appropriate, leaning on the stronger evidence.
Evidence
- Where will AI be at the end of 2027? A bet Gary Marcus (Substack) · 30 December 2024 Own record
“We, Gary Marcus, author, scientist, and noted skeptic of generative AI, and Miles Brundage, an independent AI policy researcher who recently left OpenAI and is bullish on AI progress, have agreed to the following bet”
- Gary Marcus prediction: No system will solv… — Correct | SiPoly SiPoly · 8 October 2026
“No single AI system reliably performs 5+ of the 10 tasks by 31 Dec 2025.”
- Gary Marcus prediction: No system will solv… — Correct | SiPoly SiPoly · 8 October 2026
“LessWrong "2025 in AI predictions": evaluated "Correct"”
- Gary Marcus's AI Predictions: 19 tracked | Forecast Fools Forecast Fools · 28 July 2025
“No AI system reliably solved more than four of the Marcus-Brundage tasks during 2025; Marcus's year-end review and a public tracking market both settled this against the breakthrough scenario.”
Reasoning
Two independent prediction trackers report that no single AI system reliably completed five or more of the ten Marcus-Brundage tasks by 31 December 2025. SiPoly marks the claim correct, citing a LessWrong '2025 in AI predictions' evaluation (E2, E3). Forecast Fools says no system reliably solved more than four tasks in 2025, citing Marcus's year-end review and a public tracking market (E4). Both are secondary aggregators and do not reproduce the sources they cite, but they agree, and no evidence suggests any system completed five or more of the tasks.
Cite this claim
Gary Marcus, “No single AI system will solve more than 4 of the AI 2027 Marcus-Brundage tasks by the end of 2025.” (said 1 January 2025). Verdict: resolved true. Reliability API, claim c_d492a505b02c, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_d492a505b02c
Claims without a verdict
Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.