ClaimsPeople

Gary Marcus

Gary Marcus is an author and AI entrepreneur and professor emeritus at NYU; he has written six books, including Rebooting AI and Taming Silicon Valley, founded Geometric Intelligence (bought by Uber), testified at the May 2023 US Senate AI hearing, and writes a newsletter with over 121,000 subscribers.

712 claims collected, 213 researched, 103 resolved. Report of 4 November 2026, method 0.1.0+c876030c.

Scores by kind of claim

Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.

  • Factual claims

    Statements about the present or the past, checked against the record: did they hold up?

    79 72 85

    High confidence, 64 factual claims resolved. From 18 documents, 2019 to 2026; 19 documents sampled from 37 candidates found.

    Not scored: 32 unresolvable, 4 checkable only against their own records.

  • Predictions

    Statements about events outside the person's control, checked once they fell due: did they come true?

    80 72 88

    Medium confidence, 38 predictions resolved. From 7 documents, 2019 to 2025; 19 documents sampled from 37 candidates found.

    Not scored: 60 unresolvable. 13 more fall due later.

    Does not yet allow for how likely each prediction was when made.

  • Promises

    Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?

    Insufficient record: 1 promise resolved, 5 needed for a score.

    Not scored: 1 unresolvable.

All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.

Gary Marcus's claims

5 claims

5 claims, sorted by newest said. Choose a claim to read it in full.
VerdictKindSaidClaimTopic
Mixed Factual 2 Jun 2026 Companies like Anthropic are ending their all-you-can-eat pricing, which will make many customers think twice. Factual · 2 Jun 2026 Economics and markets
Mixed Factual 2 Jun 2026 AI companies are building essentially the same technical solution with essentially the same data, so there is no moat. Factual · 2 Jun 2026 Economics and markets
Mixed Factual 2025 Current AI systems lack the knowledge graph needed to handle easy facts like those on the first page of Wikipedia. Factual · 2025 AI capabilities
Mixed Prediction 26 Oct 2023 With more than 60 elections around the world in 2024, the chance that at least one would be swayed by deepfakes is high. Prediction · 26 Oct 2023 Other global risks
Mixed Factual 14 Nov 2019 AI currently lacks a way of building reusable skills. Factual · 14 Nov 2019 AI capabilities
Back to the list

Gary Marcus

Open its page

Factual AI capabilities

“but the systems don’t have the knowledge graph to do that.”

Gary Marcus, speaking in conversation 2025 Dr. Ben Goertzel's Conversation with Dr. Gary Marcus at the World AI Summit 2025 (SingularityNET)

Current AI systems lack the knowledge graph needed to handle easy facts like those on the first page of Wikipedia.

Mixed Partly true and partly false in roughly equal measure.

The second judge said mostly false; the verdict shown is the first judge's.

What would count as true or false

Judged as of 2025, about the leading general-purpose AI systems of that time (frontier LLM chatbots). True: evidence shows these systems lacked an explicit, structured knowledge graph (or equivalent store of facts) and that this caused regular failures on easy, well-documented facts like those on the first page of a Wikipedia article. Mostly true: the systems lacked an explicit knowledge graph and made such errors noticeably often, but with small differences in detail, such as being reliable on most simple facts and failing mainly on some types. Mixed: the systems had no explicit knowledge graph, but by 2025 they usually answered easy Wikipedia-level facts correctly (often with retrieval or search), so the claimed practical shortfall only partly holds. Mostly false: the claim stretches a real but narrow weakness (occasional factual hallucination) into a general inability, or ignores widely deployed retrieval or knowledge-graph integration that handled such facts. False: benchmarks or evaluations from 2025 show leading systems answering easy, first-page Wikipedia facts with near-perfect reliability, or show that they did use knowledge graphs for this.

Evidence

  1. 'Flawed from the start': Accuracy of OpenAI's newest GPT questioned ABC News (Australia) · 20 March 2025
    “Then OpenAI ran GPT-4.5 through the quiz, finding it hallucinated 37 per cent of the time.”
  2. 'Flawed from the start': Accuracy of OpenAI's newest GPT questioned ABC News (Australia) · 20 March 2025
    “The next most recent GPT model, GPT-4o, hallucinated 62 per cent of the time.”
  3. SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge arXiv (Google DeepMind authors) · 4 January 2025
    “On this new benchmark, Gemini 2.5 Pro achieves a state-of-the-art F1-score of 55.6, outperforming other frontier models, including GPT-5.”
  4. 'Flawed from the start': Accuracy of OpenAI's newest GPT questioned ABC News (Australia) · 20 March 2025
    “And if you do have a simple query, SimpleQA's error rate shows you're better off using a search engine.”
  5. SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge arXiv (authors from Google DeepMind's Gemini Factuality team, per acknowledgments) · 4 January 2025
    “On this new benchmark, Gemini 2.5 Pro achieves a state-of-the-art F1-score of 55.6, outperforming other frontier models, including GPT-5.”
  6. SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge arXiv (authors from Google DeepMind's Gemini Factuality team, per acknowledgments) · 4 January 2025
    “Enabling tools on SimpleQA Verified results in near perfect performance, emphasizing that SimpleQA Verified should be employed for measuring parametric factuality only.”
  7. Introducing GPT-4.5 | OpenAI OpenAI · 27 February 2025
    “GPT‑4.5 has access to the latest up-to-date information with search, supports file and image uploads, and can use canvas to work on writing and code.”
  8. When Trivia Is Not Trivial: Everyday Knowledge Failures in Multilingual LLMs arXiv · 1 May 2026
    “Our analysis shows that models perform strongly on topics typically covered in widely available encyclopedic and educational sources, such as scientific and historical knowledge, while being less reliable on long-tail and everyday culture-specific topics.”
  9. Why DO large language models hallucinate? Gary Marcus (Substack newsletter) · 5 May 2025 Own record
    “Three paragraphs and a box, and Google AI Overviews couldn’t get it right.”
  10. Why DO large language models hallucinate? Gary Marcus (Substack newsletter) · 5 May 2025 Own record
    “They don’t have a database of records like any proper business would (which would be a strong basis to solve the problem)”
  11. SimpleQA AI Wiki (aiwiki.ai)
    “Challenging: Questions were adversarially collected against GPT-4o and GPT-3.5 responses.”
  12. SimpleQA AI Wiki (aiwiki.ai)
    “As tool-augmented scores diverge sharply from closed-book scores, evaluation suites are increasingly reporting both”

Reasoning

GPT-4.5 and GPT-4o had high hallucination rates on SimpleQA (E1, E2), and Gemini 2.5 Pro scored an F1 of 55.6 without tools (E3). However, SimpleQA's questions were collected adversarially against GPT-4o and GPT-3.5 responses, so the benchmark is not a direct test of easy first-page facts (E11). On SimpleQA Verified, enabling tools such as search produced near-perfect performance (E6), tool-augmented scores diverge sharply from closed-book scores (E12), and GPT-4.5 shipped with search (E7). A later study found models perform strongly on widely documented encyclopedic topics and are weaker on long-tail ones (E8). Together, this means the claimed practical failure on easy Wikipedia-level facts only partly holds, which matches the "mixed" criterion.

Cite this claim

Gary Marcus, “Current AI systems lack the knowledge graph needed to handle easy facts like those on the first page of Wikipedia.” (said 2025). Verdict: mixed. Reliability API, claim c_b78ae3ab6443, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_b78ae3ab6443

Claims without a verdict

Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.