Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
80 72 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
70 62 78 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 2 promises resolved, 5 needed for a score.
All types combined: 75 (likely 69 to 80, high confidence), from 79 claims: 35 factual claims, 44 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Scott Alexander's claims
39 claims
“I think the Alignment community did not really expect LLMs.”
Scott Alexander, interviewee 3 April 2025 AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo (Dwarkesh Podcast)
Scott Alexander said he thinks the AI alignment community did not really expect LLMs.
Resolved true The record shows it happened or was so.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
The proposition is that Scott Alexander said this, so it is tested first on attribution and then on substance. True: a transcript or recording from around 2025-04-03 (for example the Dwarkesh Patel podcast with Daniel Kokotajlo) shows Scott Alexander saying, in substance, that he thinks the alignment community did not really expect LLMs. The evidence also has to show the view is reasonable: most prominent alignment writing before about 2019 (MIRI, LessWrong, Bostrom and similar) focused on agentic, reinforcement-learning or other non-language-model routes to AGI and did not centre on large language models. Mostly true: he made the statement, but the words differ in a way that slightly changes the emphasis (for example he said 'some of the community' or 'the timing of LLMs'). Mixed: he made the statement, but the record shows that a substantial, prominent part of the alignment community did explicitly anticipate LLM-like systems as the main path. Mostly false: the statement is attributed to the wrong speaker, or he said something materially different, such as 'expected LLMs but not their capabilities'. False: the transcript shows he did not make the statement, or that he said the opposite.
Evidence
- AI 2027: Dwarkesh's Podcast with Daniel Kokotajlo and Scott Alexander Zvi Mowshowitz (Don't Worry About the Vase, Substack) · 7 April 2025
“As Scott says, the alignment community did not expect LLMs, but also we are moving back towards RL-shaped things.”
- A newcomer’s guide to the technical AI safety field — AI Alignment Forum AI Alignment Forum (post by zeshen) · 4 November 2022
“Most of the research and publications at the time were from MIRI. They tend to focus on understanding the nature of intelligence and agency i.e. agent foundations, with much of their research work around embedded agency, decision theory, and value learning.”
- Links For April 2023 Astral Codex Ten (Scott Alexander) · 20 April 2023 Own record
“I personally am pretty surprised it was just “GPT-4 got released and was very good”.”
Reasoning
E1 is an independent write-up of the Dwarkesh Patel podcast with Daniel Kokotajlo and Scott Alexander. It credits the view to him directly: "As Scott says, the alignment community did not expect LLMs." On substance, E2, an independent guide on the AI Alignment Forum, says that most early alignment research came from MIRI and focused on agency, agent foundations, decision theory and value learning, not language models. That supports the view as reasonable. E3 is Scott Alexander's own post, so it adds only weak context about his surprise at GPT-4. We have no transcript to check his exact words, but E1's wording matches the claim closely.
Cite this claim
Scott Alexander, “Scott Alexander said he thinks the AI alignment community did not really expect LLMs.” (said 3 April 2025). Verdict: resolved true. Reliability API, claim c_e8d7cc72cf56, method 0.1.0+8bea457b. https://epistacklabs.com/claims/c_e8d7cc72cf56
Claims without a verdict
Is a verdict here wrong, or do you speak for Scott Alexander? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.