Claims

Factual · Their own work or organisation

“For years I have been warning that “scaling” — eeking out improvements in AI by adding more data and more compute, without making fundamental architectural changes — would not continue forever.”

Gary Marcus, author · 9 November 2024 · CONFIRMED: LLMs have indeed reached a point of diminishing returns (Marcus on AI)

Gary Marcus has for years warned that AI scaling by adding data and compute without architectural changes would not continue forever.

Resolved true The record shows it happened or was so.

The evidence is the person's or their organisation's own records.

What would count as true or false

Judged as of 2024-11-09. True: dated writings or talks by Marcus from several years before November 2024 (for example 2022's 'Deep Learning Is Hitting a Wall', or earlier pieces) warn that improving AI by adding data and compute alone would hit limits or would not continue indefinitely. Mostly true: such warnings exist but go back only about one to two years, or are framed somewhat differently, for example as limits of deep learning in general. Mostly false: his earlier writings criticise deep learning but do not really address the limits of scaling. False: there are no such warnings before 2024.

Evidence

1. Deep Learning Is Hitting a Wall - Nautilus Nautilus · 10 March 2022 Own record

   “What’s more, the so-called scaling laws aren’t universal laws like gravity but rather mere observations that might not hold forever, much like Moore’s law, a trend in computer chip production that held for decades but arguably began to slow a decade ago.”

2. Deep Learning Is Hitting a Wall - Nautilus Nautilus · 10 March 2022 Own record

   “Indeed, we may already be running into scaling limits in deep learning, perhaps already approaching a point of diminishing returns.”

3. Two years later, deep learning is still faced with the same fundamental challenges Gary Marcus Substack (Marcus's own newsletter) · 10 March 2024 Own record

   “On the other hand, that’s not really what the paper was about. The article was about obstacles to general intelligence and why scaling wouldn’t be enough.”

Reasoning

Marcus's March 2022 Nautilus essay "Deep Learning Is Hitting a Wall" (E1, E2) says scaling laws are "mere observations that might not hold forever." It also says deep learning may already be running into scaling limits and diminishing returns. That is a dated warning, made more than two and a half years before the November 2024 claim, that gains from scaling would not continue indefinitely. The resolution criteria name this exact essay as an example that qualifies for "true." E3 is Marcus's 2024 summary of the essay's argument that "scaling wouldn't be enough"; it is consistent with the earlier warning but adds little independent weight.

Cite this claim

Gary Marcus, “Gary Marcus has for years warned that AI scaling by adding data and compute without architectural changes would not continue forever.” (said 9 November 2024). Verdict: resolved true. Reliability API, claim c_fba7655c74b4, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_fba7655c74b4