Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
79 72 85 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
80 72 88 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Gary Marcus's claims
52 claims
“Central to the argument was that pure scaling would not solve hallucinations or abstraction; I concluded that “there are serious holes in the scaling argument.””
Gary Marcus, author March 2022 CONFIRMED: LLMs have indeed reached a point of diminishing returns (Marcus on AI)
Gary Marcus's March 2022 article argued pure scaling would not solve hallucinations or abstraction and concluded there are serious holes in the scaling argument.
Mostly true True in substance, wrong in a detail.
The second judge said resolved true; the verdict shown is the first judge's.
What would count as true or false
Judge against the text of Marcus's March 2022 article (e.g., 'Deep Learning Is Hitting a Wall', Nautilus, March 2022). True: the article argues that scaling alone would not solve hallucinations or unreliability and problems of abstraction or reasoning, and contains the words 'there are serious holes in the scaling argument' or near-identical wording. Mostly true: the article makes both arguments, but the quoted sentence differs slightly in wording, or appears as one point rather than as the conclusion. Mixed: the article makes only one of the two arguments (hallucinations or abstraction), or makes the argument without anything like the quoted conclusion. Mostly false: the article criticises scaling only in general terms, without connecting it to hallucinations or abstraction, and lacks the quoted line. False: no such March 2022 article exists, or it does not argue against scaling.
Evidence
- Deep Learning Is Hitting a Wall - Nautilus Nautilus · 10 March 2022 Own record
“There are serious holes in the scaling argument. To begin with, the measures that have scaled have not captured what we desperately need to improve: genuine comprehension.”
- Deep Learning Is Hitting a Wall - Nautilus Nautilus · 10 March 2022 Own record
“In the last several months, research from DeepMind and elsewhere on models even larger than GPT-3 have shown that scaling starts to falter on some measures, such as toxicity, truthfulness, reasoning, and common sense.”
- Deep Learning Is Hitting a Wall - Nautilus Nautilus · 10 March 2022 Own record
“Deep-learning systems are outstanding at interpolating between specific examples they have seen before, but frequently stumble when confronted with novelty.”
- Dialogue Origin: “Deep Learning Is Hitting a Wall” “There Is No Wall” Quote Investigator · 4 January 2025
“In conclusion, the phrase “Deep Learning Is Hitting a Wall” appeared as the title of an article by Gary Marcus in March 2022.”
Reasoning
E4, an independent source, confirms that Marcus published "Deep Learning Is Hitting a Wall" in March 2022. The article itself contains the exact sentence "There are serious holes in the scaling argument" (E1). It argues that scaling falters on truthfulness, reasoning and common sense (E2), and that deep learning stumbles when faced with novelty (E3). So both arguments the claim names appear, in substance, under the criteria's "unreliability" and "reasoning" wording. However, none of the quoted passages from the article uses the word "hallucinations". The quoted sentence opens a section of the argument rather than serving as its conclusion, which fits the "mostly true" criterion.
Cite this claim
Gary Marcus, “Gary Marcus's March 2022 article argued pure scaling would not solve hallucinations or abstraction and concluded there are serious holes in the scaling argument.” (said March 2022). Verdict: mostly true. Reliability API, claim c_3657e0b3d486, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_3657e0b3d486
Claims without a verdict
Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.