The shortcuts that LLMs use work about 80% of the time but not 100%.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
Reasoning
The 80% figure is an approximate estimate ("like 80%") with no defined measure.
Cite this claim
Gary Marcus, “The shortcuts that LLMs use work about 80% of the time but not 100%.” (said 2025). Verdict: unresolvable. Reliability API, claim c_4c7053a6a8a3, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_4c7053a6a8a3
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Gary Marcus
-
Mostly true
“Reviews by Joanna Stern and Marques Brownle of one early prototype were damning;”
Reviews by Joanna Stern and Marques Brownlee of one early humanoid robot prototype were damning.
-
Mostly true
“Many would agree we have reached a point of diminishing returns for scaling; faith in scaling as a route to AGI has dissipated.”
-
Mostly true
“Despite all the hype, agents didn’t turn out to be reliable.”
-
Mixed
“but the systems don’t have the knowledge graph to do that.”
-
Resolved true
“Hasn’t worked so far. That’s for sure.”
Bolting extra components onto an LLM and scaling it up has so far not produced human-level AGI.
Verdicts are made automatically; how verdicts are made.