ClaimsGary Marcus

“As far as I can tell from rumors, GPT-4 is architecturally essentially the same as GPT-3.”

Gary Marcus, author 25 December 2022 What to Expect When You're Expecting … GPT-4 (Marcus on AI)

Factual AI capabilities

GPT-4 is architecturally essentially the same as GPT-3, according to rumors.

Unresolvable The public record could not settle it.

Factual: Statements about the present or the past, checked against the record: did they hold up?

The second judge would not resolve this claim; the verdict shown is the first judge's.

What would count as true or false

This is a hedged claim about what rumors said, judged as of 2022-12-25. True: credible rumors or reporting circulating before that date described GPT-4 as essentially the same architecture as GPT-3 (a dense decoder-only transformer, mainly scaled or retrained), and later disclosures (e.g. OpenAI's GPT-4 technical report or credible leaks) do not show a fundamentally different architecture. Mostly true: the rumors said this, but later credible information shows notable but non-fundamental differences, e.g. a mixture-of-experts transformer, which is still a transformer LLM. Mixed: the rumors were split, or later information shows significant architectural changes that make "essentially the same" debatable. Mostly false: hardly any rumors before that date said this, though GPT-4 did turn out to be a transformer LLM. False: rumors at the time described a substantially different architecture and later evidence confirms it was fundamentally different.

Set before the research began.

Evidence (12)

  1. E1 ChatGPT is just a taste of a "monster" GPT-4 says Gary Marcus THE DECODER · 26 December 2022

    “When it comes to its architecture, Marcus doesn't expect any significant differences and therefore assumes that GPT-4 will have the same weaknesses as GPT-3 or ChatGPT”
  2. E2 ChatGPT is just a taste of a "monster" GPT-4 says Gary Marcus THE DECODER · 26 December 2022

    “Rumors about GPT-4 have been floating around the web for weeks, and they have two things in common: GPT-4 is supposed to outperform GPT-3 and ChatGPT significantly and be released relatively soon in the spring.”
  3. E3 Proprietary models - EcoLogits EcoLogits · 12 October 2026

    “The model architecture of GPT-4 was leaked1 and estimated to be a sparse mixture of experts model with a total of 1.8 trillion parameters and 16 experts of 111 billion parameters each.”
  4. E4 GPT-4 (2022) LifeArchitect.ai (Alan D. Thompson)

    “‘Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.’”
  5. E5 GPT-4 (2022) Lifearchitect (Alan D. Thompson) · March 2023

    “Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.”
  6. E6 GPT-4 (2022) Lifearchitect (Alan D. Thompson) · 28 October 2023

    “OpenAI was able to keep costs reasonable by utilizing a mixture of experts (MoE) model.”
  7. E7 GPT-4 (2022) Lifearchitect (Alan D. Thompson) · 12 December 2022

    “GPT3: 175b params for $4.6M. GPT4: 100t params for $2.6B. This trend line can't go up much further.”
  8. E8 GPT-4 (2022) Lifearchitect (Alan D. Thompson) · 24 August 2021

    ““From talking to OpenAI, GPT-4 will be about 100 trillion parameters…””
  9. E9 Rumor: GPT-4 to be 500x larger, available in a few years OpenAI Community forum (user discussion thread) · 13 September 2021

    “Altman said in the interview that contrary to popular belief, GPT-4 will not be any bigger than GPT-3 but will use more compute resources.”
  10. E10 GPT-4 architecture, datasets, costs and more leaked THE DECODER · 28 June 2023

    “GPT-4 is rumored to be based on eight models, each with 220 billion parameters, which are linked in the Mixture of Experts (MoE) architecture.”
  11. E11 GPT-4 Architecture, Infrastructure, Training Dataset, Costs, Vision, MoE SemiAnalysis · 10 July 2023

    “A dense transformer is the model architecture that OpenAI GPT-3, Google PaLM, Meta LLAMA, TII Falcon, MosaicML MPT, etc use.”
  12. E12 GPT‑4 - OpenAI OpenAI · 14 March 2023

    “GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that, while less capable than humans in many real-world scenarios, exhibits human-level performance on various professional and academic benchmarks.”

Reasoning

The pre-claim rumours in the evidence (E2, E7, E8, E9) concern GPT-4's size, compute, performance and timing, and none states that GPT-4 would share GPT-3's architecture. The criteria turn on what the rumours said about architecture, so the evidence does not settle the claim. Later reports of a mixture-of-experts design (E3, E6, E10) are described as leaked, estimated or rumoured.

Cite this claim

Gary Marcus, “GPT-4 is architecturally essentially the same as GPT-3, according to rumors.” (said 25 December 2022). Verdict: unresolvable. Reliability API, claim c_9f21b3b05a25, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_9f21b3b05a25

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Gary Marcus, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Gary Marcus

  1. Mostly true Factual January 2026
    “Reviews by Joanna Stern and Marques Brownle of one early prototype were damning;”

    Reviews by Joanna Stern and Marques Brownlee of one early humanoid robot prototype were damning.

  2. Mostly true Factual January 2026
    “Many would agree we have reached a point of diminishing returns for scaling; faith in scaling as a route to AGI has dissipated.”

    Many would agree the AI field has reached a point of diminishing returns from scaling, and faith in scaling as a route to AGI has dissipated.

  3. Mostly true Factual January 2026
    “Despite all the hype, agents didn’t turn out to be reliable.”

    AI agents did not turn out to be reliable in 2025.

  4. Mixed Factual 2025
    “but the systems don’t have the knowledge graph to do that.”

    Current AI systems lack the knowledge graph needed to handle easy facts like those on the first page of Wikipedia.

  5. Resolved true Factual 2025
    “Hasn’t worked so far. That’s for sure.”

    Bolting extra components onto an LLM and scaling it up has so far not produced human-level AGI.

All of them

Verdicts are made automatically; how verdicts are made.