Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
79 72 85 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
80 72 88 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Gary Marcus's claims
42 claims
“For years I have been warning that “scaling” — eeking out improvements in AI by adding more data and more compute, without making fundamental architectural changes — would not continue forever.”
Gary Marcus, author 9 November 2024 CONFIRMED: LLMs have indeed reached a point of diminishing returns (Marcus on AI)
Gary Marcus has for years warned that AI scaling by adding data and compute without architectural changes would not continue forever.
Resolved true The record shows it happened or was so.
The evidence is the person's or their organisation's own records.
What would count as true or false
Judged as of 2024-11-09. True: dated writings or talks by Marcus from several years before November 2024 (for example 2022's 'Deep Learning Is Hitting a Wall', or earlier pieces) warn that improving AI by adding data and compute alone would hit limits or would not continue indefinitely. Mostly true: such warnings exist but go back only about one to two years, or are framed somewhat differently, for example as limits of deep learning in general. Mostly false: his earlier writings criticise deep learning but do not really address the limits of scaling. False: there are no such warnings before 2024.
Evidence
- Deep Learning Is Hitting a Wall - Nautilus Nautilus · 10 March 2022 Own record
“What’s more, the so-called scaling laws aren’t universal laws like gravity but rather mere observations that might not hold forever, much like Moore’s law, a trend in computer chip production that held for decades but arguably began to slow a decade ago.”
- Deep Learning Is Hitting a Wall - Nautilus Nautilus · 10 March 2022 Own record
“Indeed, we may already be running into scaling limits in deep learning, perhaps already approaching a point of diminishing returns.”
- Two years later, deep learning is still faced with the same fundamental challenges Gary Marcus Substack (Marcus's own newsletter) · 10 March 2024 Own record
“On the other hand, that’s not really what the paper was about. The article was about obstacles to general intelligence and why scaling wouldn’t be enough.”
Reasoning
Marcus's March 2022 Nautilus essay "Deep Learning Is Hitting a Wall" (E1, E2) says scaling laws are "mere observations that might not hold forever." It also says deep learning may already be running into scaling limits and diminishing returns. That is a dated warning, made more than two and a half years before the November 2024 claim, that gains from scaling would not continue indefinitely. The resolution criteria name this exact essay as an example that qualifies for "true." E3 is Marcus's 2024 summary of the essay's argument that "scaling wouldn't be enough"; it is consistent with the earlier warning but adds little independent weight.
Cite this claim
Gary Marcus, “Gary Marcus has for years warned that AI scaling by adding data and compute without architectural changes would not continue forever.” (said 9 November 2024). Verdict: resolved true. Reliability API, claim c_fba7655c74b4, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_fba7655c74b4
Claims without a verdict
Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.