Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
76 70 82 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
60 42 78
All types combined: 74 (likely 68 to 80, high confidence), from 79 claims: 71 factual claims, 8 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Eliezer Yudkowsky's claims
“I was a passionate noticer of the human brain not being homogenous, and was therefore unpleasantly surprised by transformers scaling as well as they did.”
Eliezer Yudkowsky, writing on X 25 March 2024 Post by @ESYudkowsky on X, 2024-03-25 (X)
Yudkowsky was unpleasantly surprised by how well transformers scaled, having believed the human brain is not homogeneous.
Mostly true True in substance, wrong in a detail.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
This is a claim about Yudkowsky's own past beliefs and reaction. True: his earlier writings show he stressed that the brain is not homogeneous (e.g. it has specialised, complex architecture) and, before or around the scaling of transformers, expected that simple uniform architectures would not scale as well as they did; and his record from that period (or a credible account of it) is consistent with him being surprised. Mostly true: the earlier writings show the non-homogeneity view, but there is little from that period to show the surprise beyond his 2024 statement, and nothing contradicts it. Mixed: the record is ambiguous, with some earlier statements anticipating scaling success and others not. Mostly false: his earlier statements mostly predicted that large homogeneous architectures would scale well, so the claimed surprise misdescribes his past views, although he did stress the brain's non-homogeneity. False: contemporaneous record shows he expected transformer scaling to succeed roughly as it did, or that he did not hold the stated view about the brain.
Evidence
- Dialogue with Eliezer Yudkowsky on Neural Networks — LessWrong LessWrong (linkpost from aashishreddy.substack.com) Own record
“Still no general intelligence. But, somehow, everyone outside the field knows that neural networks are the Dominant-Paradigm-Overthrowing New Idea, ever since backpropagation was invented in the 1970s.”
- Analogies and General Priors on Intelligence — AI Alignment Forum AI Alignment Forum · 20 August 2021
“The idea that you could create human-level intelligence by just feeding raw data into the right math-inspired architecture is pure fantasy.”
- Analogies and General Priors on Intelligence — AI Alignment Forum AI Alignment Forum · 20 August 2021
“By contrast, I think Yudkowsky was saying that AGI source code would be relatively simple and coherent and plausibly written by one team, and then that relatively simple source code would give rise to lots of different skills via learning.”
- Dialogue with Eliezer Yudkowsky on Neural Networks — LessWrong LessWrong (post by aashish, with replies by Eliezer Yudkowsky) · 2 October 2026 Own record
“One said: If you wire enough neurons together, it will emergently start thinking just like a human brain does. They usually did not use gradient descent, since gradient descent would never work on that many “neurons”.”
- Dialogue with Eliezer Yudkowsky on Neural Networks — LessWrong LessWrong (post by aashish, with replies by Eliezer Yudkowsky) · 2 October 2026 Own record
“There is to be clear still a big damn postmortem from my perspective on two actual Bad Predictions made for Bad Reasons, which are from my perspective something like:”
- Dialogue with Eliezer Yudkowsky on Neural Networks — LessWrong LessWrong (post by aashish, with replies by Eliezer Yudkowsky) · 2 October 2026 Own record
“This cached thought has been around for three decades. Still no general intelligence.”
- Analogies and General Priors on Intelligence — AI Alignment Forum AI Alignment Forum (riceissa, Sammy Martin; comment by Steven Byrnes) · 20 August 2021
“By contrast, I think Yudkowsky was saying that AGI source code would be relatively simple and coherent and plausibly written by one team, and then that relatively simple source code would give rise to lots of different skills via learning.”
- Eliezer Yudkowsky: Track Record LongtermWiki · 1 February 2026
“Yudkowsky's own assessment: "Reality was far to the Eliezer side of Eliezer on the Eliezer-Robin axis, and things like GPT-3 were built with less architectural complexity and generalized more than I was arguing to Robin that complex architectures should generalize over domains."”
- Eliezer Yudkowsky: Track Record LongtermWiki · 1 February 2026
“| Scaling | "Stack more layers" won't work | Updated that it went further than expected | GPT-4 |”
- Eliezer Yudkowsky: Track Record LongtermWiki · 1 February 2026
“LLMs are "a bit smarter than I thought this technology was going to scale to"”
- How Intelligible is Intelligence? Machine Intelligence Research Institute (Anna Salamon, Stephen Rayhawk, János Kramár) · 2010 Own record
“We discuss theoretical and empirical results bearing on the intelligibility of intelligence, and argue that they modestly favor a relatively high estimate.”
Reasoning
An independent track-record page quotes Yudkowsky saying GPT-3 was built with less architectural complexity, and generalized further, than he had argued to Robin Hanson that complex architectures would (E9). The same page lists his earlier scaling position as "'Stack more layers' won't work" (E10) and quotes him saying LLMs went further than he expected (E11). This fits him having favoured complex, non-uniform architectures and being surprised. However, the surprise itself is shown mostly by his own later statements, and no quote from that period directly shows him stressing that the brain is not homogeneous. One commenter reads his foom-debate position as expecting relatively simple AGI code (E4/E8), but alongside E9 that shows a difference of degree rather than a contradiction.
Cite this claim
Eliezer Yudkowsky, “Yudkowsky was unpleasantly surprised by how well transformers scaled, having believed the human brain is not homogeneous.” (said 25 March 2024). Verdict: mostly true. Reliability API, claim c_e12fd543dc03, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_e12fd543dc03
Claims without a verdict
Is a verdict here wrong, or do you speak for Eliezer Yudkowsky? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.