Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
76 69 82 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
56 42 70 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 4 promises resolved, 5 needed for a score.
All types combined: 72 (likely 66 to 78, high confidence), from 93 claims: 76 factual claims, 17 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Dario Amodei's claims
49 claims
“We were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue.”
Dario Amodei, Anthropic CEO, essay on personal site 18 February 2025 The Urgency of Interpretability (darioamodei.com)
Amodei said that "we" were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue; the surrounding text does not say who "we" refers to.
Mostly true True in substance, wrong in a detail.
The evidence is the person's or their organisation's own records.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judge as of 2025-02-18, reading 'we' as Anthropic and/or its interpretability researchers (including Amodei's collaborators). True: published work by those researchers by that date (e.g. 'Toy Models of Superposition', 2022) gives a theoretical or toy-model basis for the superposition hypothesis that motivated pursuing it (e.g. through dictionary learning or sparse autoencoders). Mostly true: such work exists, but it is mainly empirical or toy-model rather than a 'strong theoretical basis', or it was written partly by people not then at Anthropic. Mixed: the superposition idea was central to the work, but the theoretical basis came mainly from others. Mostly false: the group's role was minor, or the hypothesis came almost entirely from others. False: no such work by the group exists.
Evidence
- Toy models of superposition Anthropic · 14 September 2022 Own record
“In this paper, we use toy models — small ReLU networks trained on synthetic data with sparse input features — to investigate how and when models represent more features than they have dimensions. We call this phenomenon superposition.”
- Towards monosemanticity Anthropic · 5 October 2023 Own record
“we outline evidence that there are better units of analysis than individual neurons, and we have built machinery that lets us find these units in small transformer models.”
Reasoning
E1 is Anthropic's 2022 'Toy Models of Superposition' paper. It used small ReLU networks trained on sparse synthetic features to study how models represent more features than they have dimensions, a phenomenon it names superposition. E2, Anthropic's 2023 'Towards Monosemanticity' paper, reports building tools to find units of analysis better than individual neurons, which is consistent with pursuing that hypothesis. Both items are the group's own published work, and the basis they show comes from toy models rather than formal theory. That fits the 'mostly true' tier better than a 'strong theoretical basis'.
Cite this claim
Dario Amodei, “Amodei said that "we" were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue; the surrounding text does not say who "we" refers to.” (said 18 February 2025). Verdict: mostly true. Reliability API, claim c_61ce3575598b, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_61ce3575598b
Claims without a verdict
- Unresolvable 157
- Not yet due 40
- Own records only 43
- Not researched 134
- Not checkable 864
Is a verdict here wrong, or do you speak for Dario Amodei? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.