Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
79 72 85 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
80 72 88 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Gary Marcus's claims
52 claims
“GPT-5 will surely be better, a lot better than GPT-4.”
Gary Marcus, author 27 July 2025 What to Expect When You're Expecting … GPT-5 (Marcus on AI)
GPT-5 will be considerably better than GPT-4.
Mostly true True in substance, wrong in a detail.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judge at GPT-5's release (OpenAI released GPT-5 in August 2025) and the evaluations soon after. True: GPT-5 substantially outperformed GPT-4 (original 2023 model) across major benchmarks (reasoning, coding, maths, factuality) and in general assessments, by margins most observers would call large. Mostly true: GPT-5 was clearly better than GPT-4 on most measures, but the gain was moderate on some important measures or it regressed in a minor area. Mixed: improvement was large on some major dimensions and negligible or negative on others of similar weight. Mostly false: GPT-5 was only marginally better than GPT-4 overall. False: GPT-5 was not better, or was worse, than GPT-4, or no model named GPT-5 was released. Compare against GPT-4 itself, not GPT-4o or o3, as the claim does.
Evidence
- GPT-5 and GPT-4 were both major leaps in benchmarks from the previous generation Epoch AI · 29 August 2025
“GPT-5 achieved a 25x improvement in 50% time horizon from GPT-4, compared to GPT-4’s 36x improvement on GPT-3, and both improvements are consistent with the longer-term trend.”
- Considering ChatGPT-5 aiandacademia (Substack) · 18 August 2025
“According to Vectara, GPT-5 has a grounded hallucination rate of 1.4%, compared to 1.8% for GPT-4, and 1.69% for GPT-4 turbo and 4o mini”
- GPT-5 is no slowdown Transformer · 8 August 2025
“But observers quickly noticed that despite the years of anticipation, GPT-5 isn’t much better than anything else already on the market.”
- Introducing GPT-5 OpenAI · 7 August 2025
“GPT‑5 is much smarter across the board, as reflected by its performance on academic and human-evaluated benchmarks, particularly in math, coding, visual perception, and health.”
Reasoning
Epoch AI [E1] measures a 25x improvement in 50% time horizon from GPT-4 to GPT-5 and calls both it and GPT-4's 36x gain over GPT-3 major leaps consistent with the longer-term trend. OpenAI's launch post [E4] reports broad gains in math, coding, visual perception and health, though OpenAI makes the model. On factuality, the Vectara figures reported in [E2] show only a modest gain: a grounded hallucination rate of 1.4% for GPT-5 against 1.8% for GPT-4. [E3] compares GPT-5 with other models on the market, not with the original GPT-4. GPT-5 showed a large gain over GPT-4 on Epoch AI's time-horizon measure [E1] and a modest gain on one factuality measure [E2], with broader gains claimed by OpenAI itself [E4], which fits the mostly-true criterion.
Cite this claim
Gary Marcus, “GPT-5 will be considerably better than GPT-4.” (said 27 July 2025). Verdict: mostly true. Reliability API, claim c_f664959e6d26, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_f664959e6d26
Claims without a verdict
Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.