ClaimsPeople

Gary Marcus

Gary Marcus is an author and AI entrepreneur and professor emeritus at NYU; he has written six books, including Rebooting AI and Taming Silicon Valley, founded Geometric Intelligence (bought by Uber), testified at the May 2023 US Senate AI hearing, and writes a newsletter with over 121,000 subscribers.

712 claims collected, 213 researched, 103 resolved. Report of 4 November 2026, method 0.1.0+c876030c.

Scores by kind of claim

Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.

  • Factual claims

    Statements about the present or the past, checked against the record: did they hold up?

    79 72 85

    High confidence, 64 factual claims resolved. From 18 documents, 2019 to 2026; 19 documents sampled from 37 candidates found.

    Not scored: 32 unresolvable, 4 checkable only against their own records.

  • Predictions

    Statements about events outside the person's control, checked once they fell due: did they come true?

    80 72 88

    Medium confidence, 38 predictions resolved. From 7 documents, 2019 to 2025; 19 documents sampled from 37 candidates found.

    Not scored: 60 unresolvable. 13 more fall due later.

    Does not yet allow for how likely each prediction was when made.

  • Promises

    Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?

    Insufficient record: 1 promise resolved, 5 needed for a score.

    Not scored: 1 unresolvable.

All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.

Gary Marcus's claims

213 claims, sorted by newest said. Choose a claim to read it in full.
VerdictKindSaidClaimTopic
Mostly true Factual 20 Sep 2019 Deep learning does more than linear regression but is a more sophisticated form of statistical correlation. Factual · 20 Sep 2019 AI capabilities
Mostly true Factual 20 Sep 2019 Bad AI is leading to fatalities and undermining democracy. Factual · 20 Sep 2019 AI policy and governance
Resolved true Prediction 20 Sep 2019 Marcus said he did not think machines would reach world-class scientific reasoning within the next five years (by 2024). Prediction · 20 Sep 2019 AI timelines and forecasting
Unresolvable Prediction 20 Sep 2019 Marcus said he thinks it is inevitable that society will adopt universal basic income, and that the open question is whether this happens gradually or through social unrest. Prediction · 20 Sep 2019 AI, the economy and work
Resolved true Factual 16 Feb 2019 The system has no idea that a unicorn has one horn and asserts that it has four. Factual · 16 Feb 2019 AI capabilities
Unresolvable Factual 16 Feb 2019 OpenAI's new system is state of the art on some question-answering tests but is no match for humans on story understanding. Factual · 16 Feb 2019 AI capabilities
Mostly false Factual 17 Jan 2019 According to Giuliani's disclosure on 2019-01-17, every tweet and statement like the one referenced was a lie, and Trump's campaign colluded. Factual · 17 Jan 2019 Other
Mostly false Factual 17 Jan 2019 A linked report confirms that Donald Trump took talking points from Vladimir Putin. Factual · 17 Jan 2019 Other
Unresolvable Prediction 15 Jan 2019 William Barr may become the person who most subverts American democracy, more than Donald Trump. Prediction · 15 Jan 2019 AI policy and governance
Unresolvable Factual 9 Jan 2019 The Kindle edition of The Algebraic Mind: Integrating Connectionism and Cognitive Science is available as of 2019-01-09. Factual · 9 Jan 2019 Their own work or organisation
Resolved true Factual 2 Jan 2019 Gary Marcus posted "Deep Learning: A Critical Appraisal" on 2018-01-02, asking whether deep learning might be approaching a wall and be just one tool among many needed for AGI. Factual · 2 Jan 2019 AI capabilities
Unresolvable Prediction 2019 The deep learning approach to language would never work, as argued in a chapter of Rebooting AI. Prediction · 2019 Their own work or organisation
Unresolvable Factual 2017 Some AI problems might be too complex for individual labs and not of enough financial interest to large AI companies like Google and Meta, which he described as driven by profits rather than humanity. Factual · 2017 AI policy and governance
Back to the list

Gary Marcus

Open its page

Factual AI capabilities

“@OpenAI is SoTA on some QA tests, but no match for humans on story understanding.”

Gary Marcus, writing on X 16 February 2019 Post by @GaryMarcus on X, 2019-02-16 (X)

OpenAI's new system is state of the art on some question-answering tests but is no match for humans on story understanding.

Unresolvable The public record could not settle it.

The second judge would not resolve this claim; the verdict shown is the first judge's.

What would count as true or false

Judge as of 2019-02-16 for OpenAI's GPT-2, released that week. The claim has two parts: (a) GPT-2 achieved state-of-the-art results on some question-answering or reading-comprehension benchmarks; (b) it was well short of human performance on story understanding. True: both hold. That would be shown by OpenAI's GPT-2 paper or blog reporting SOTA on at least one QA-type test (e.g., a zero-shot reading-comprehension or commonsense benchmark), together with evidence such as Marcus's tests, benchmark scores or contemporaneous evaluations that GPT-2 fell clearly below humans on story or narrative comprehension. Mostly true: both broadly hold, but the SOTA results were on language-modelling or related tasks rather than strictly QA, or held only in a narrow setting such as zero-shot. Mixed: one part clearly holds and the other does not. Mostly false: GPT-2 was not SOTA on any QA test, and the human-gap point rests only on anecdote. False: both parts are contradicted.

Evidence

  1. Better language models and their implications OpenAI · 14 February 2019
    “On other language tasks like question answering, reading comprehension, summarization, and translation, we are able to get surprising results without any fine-tuning of our models, simply by prompting the trained model in the right way (see below for examples of how we do this), though we do still fall short of state-of-the-art for specialized systems.”
  2. Language Models are Unsupervised Multitask Learners OpenAI (Radford, Wu, Child, Luan, Amodei, Sutskever)
    “GPT-2 achieves new state of the art results of 93.3% on common nouns and 89.1% on named entities.”
  3. Language Models are Unsupervised Multitask Learners OpenAI (Radford, Wu, Child, Luan, Amodei, Sutskever)
    “On reading comprehension the performance of GPT-2 is competitive with supervised baselines in a zero-shot setting.”
  4. Language Models are Unsupervised Multitask Learners OpenAI (Radford, Wu, Child, Luan, Amodei, Sutskever)
    “As seen in Figure 2 performance steadily improves as model size is increased and closes the majority of the gap to human performance on this test.”

Reasoning

E2 reports state-of-the-art results on common nouns and named entities but does not show they are on a question-answering test, and E1 says GPT-2 fell short of state of the art on question answering. E4 indicates some remaining gap to human performance on an unnamed test but does not show it concerns story understanding or that the gap was large, so neither part of the claim is clearly settled.

Cite this claim

Gary Marcus, “OpenAI's new system is state of the art on some question-answering tests but is no match for humans on story understanding.” (said 16 February 2019). Verdict: unresolvable. Reliability API, claim c_d9cf9266e3b9, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_d9cf9266e3b9

Claims without a verdict

Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.