ClaimsGary Marcus

“GPT-5 will surely be better, a lot better than GPT-4.”

Gary Marcus, author 27 July 2025 What to Expect When You're Expecting … GPT-5 (Marcus on AI) archived copy

Prediction AI capabilities

GPT-5 will be considerably better than GPT-4.

Mostly true True in substance, wrong in a detail.

Prediction: Statements about events outside the person's control, checked once they fell due: did they come true?

The second judge would not resolve this claim; the verdict shown is the first judge's.

What would count as true or false

Judge at GPT-5's release (OpenAI released GPT-5 in August 2025) and the evaluations soon after. True: GPT-5 substantially outperformed GPT-4 (original 2023 model) across major benchmarks (reasoning, coding, maths, factuality) and in general assessments, by margins most observers would call large. Mostly true: GPT-5 was clearly better than GPT-4 on most measures, but the gain was moderate on some important measures or it regressed in a minor area. Mixed: improvement was large on some major dimensions and negligible or negative on others of similar weight. Mostly false: GPT-5 was only marginally better than GPT-4 overall. False: GPT-5 was not better, or was worse, than GPT-4, or no model named GPT-5 was released. Compare against GPT-4 itself, not GPT-4o or o3, as the claim does.

Set before the research began.

Evidence (4)

  1. E1 GPT-5 and GPT-4 were both major leaps in benchmarks from the previous generation Epoch AI · 29 August 2025

    “GPT-5 achieved a 25x improvement in 50% time horizon from GPT-4, compared to GPT-4’s 36x improvement on GPT-3, and both improvements are consistent with the longer-term trend.”
  2. E2 Considering ChatGPT-5 aiandacademia (Substack) · 18 August 2025

    “According to Vectara, GPT-5 has a grounded hallucination rate of 1.4%, compared to 1.8% for GPT-4, and 1.69% for GPT-4 turbo and 4o mini”
  3. E3 GPT-5 is no slowdown Transformer · 8 August 2025

    “But observers quickly noticed that despite the years of anticipation, GPT-5 isn’t much better than anything else already on the market.”
  4. E4 Introducing GPT-5 OpenAI · 7 August 2025

    “GPT‑5 is much smarter across the board, as reflected by its performance on academic and human-evaluated benchmarks, particularly in math, coding, visual perception, and health.”

Reasoning

Epoch AI [E1] measures a 25x improvement in 50% time horizon from GPT-4 to GPT-5 and calls both it and GPT-4's 36x gain over GPT-3 major leaps consistent with the longer-term trend. OpenAI's launch post [E4] reports broad gains in math, coding, visual perception and health, though OpenAI makes the model. On factuality, the Vectara figures reported in [E2] show only a modest gain: a grounded hallucination rate of 1.4% for GPT-5 against 1.8% for GPT-4. [E3] compares GPT-5 with other models on the market, not with the original GPT-4. GPT-5 showed a large gain over GPT-4 on Epoch AI's time-horizon measure [E1] and a modest gain on one factuality measure [E2], with broader gains claimed by OpenAI itself [E4], which fits the mostly-true criterion.

Cite this claim

Gary Marcus, “GPT-5 will be considerably better than GPT-4.” (said 27 July 2025). Verdict: mostly true. Reliability API, claim c_f664959e6d26, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_f664959e6d26

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Gary Marcus, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More prediction claims by Gary Marcus

  1. Mostly true Prediction 27 July 2025
    “Some of its advice will be head-scratchingly bad.”

    Some of GPT-5's advice will be very bad.

  2. Resolved true Prediction 27 July 2025
    “GPT-5 really is about to drop, no foolin’.”

    OpenAI will release GPT-5 imminently.

  3. Resolved true Prediction 27 July 2025
    “It will still make a significant number of shake-your-head stupid errors, in ways that are hard to fully predict.”

    GPT-5 will still make a significant number of obvious errors that are hard to predict.

  4. Mostly true Prediction 1 January 2025
    “Neurosymbolic AI will become much more prominent.”

    Neurosymbolic AI will become much more prominent in 2025.

  5. Mostly true Prediction 1 January 2025
    “Sora will continue to have trouble with physics.”

    OpenAI's Sora will continue to have trouble with physics in 2025.

All of them

Verdicts are made automatically; how verdicts are made.