Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
76 69 82 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
56 42 70 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 4 promises resolved, 5 needed for a score.
All types combined: 72 (likely 66 to 78, high confidence), from 93 claims: 76 factual claims, 17 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Dario Amodei's claims
49 claims
“A property of AI — which I and my co-founders were among the first to document back when we worked at OpenAI — is that all else equal, scaling up the training of AI systems leads to smoothly better results on a range of cognitive tasks, across the board.”
Dario Amodei, essay author 13 October 2022 On DeepSeek and Export Controls (darioamodei.com)
Dario Amodei and his co-founders were among the first to document, while at OpenAI, that scaling up AI training smoothly improves results across a range of cognitive tasks.
Mostly true True in substance, wrong in a detail.
What would count as true or false
Judge as of 2022-10-13. True: Amodei and one or more later Anthropic co-founders (e.g. Jared Kaplan, Sam McCandlish, Tom Brown, Jack Clark, Chris Olah) wrote, while at OpenAI, early, widely cited work documenting smooth performance improvements from scaling, such as 'Scaling Laws for Neural Language Models' (Kaplan et al., 2020) and the GPT-3 paper, and this work was among the first such documentation. Mostly true: they wrote such work at OpenAI, but earlier work (e.g. Hestness et al. 2017 at Baidu) documented scaling trends, making 'among the first' a modest stretch, or the documented improvements were in loss rather than broadly 'across the board' cognitive tasks. Mixed: they contributed key work, but substantial earlier documentation by others makes the priority claim doubtful. Mostly false: the co-founders' scaling work was not done at OpenAI, or their involvement was marginal. False: they did not write such work.
Evidence
- Scaling laws for neural language models - OpenAI OpenAI · 23 January 2020
“Jared Kaplan, Sam McCandlish, Tom Henighan, Tom Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei”
- Scaling laws for neural language models - OpenAI OpenAI · 23 January 2020
“The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of magnitude.”
Reasoning
OpenAI's own page for 'Scaling Laws for Neural Language Models' (2020-01-23) lists Dario Amodei as an author. It also lists later Anthropic co-founders Jared Kaplan, Sam McCandlish and Tom Brown, which shows they did scaling work together at OpenAI (E1). The paper's abstract describes how loss changes smoothly as a power law with model size, data and compute (E2). That is a measure of loss, not results "across the board" on a range of cognitive tasks. Under the criteria, findings about loss rather than broad task performance make the claim a modest stretch, and the evidence does not settle the "among the first" point either way.
Cite this claim
Dario Amodei, “Dario Amodei and his co-founders were among the first to document, while at OpenAI, that scaling up AI training smoothly improves results across a range of cognitive tasks.” (said 13 October 2022). Verdict: mostly true. Reliability API, claim c_c346ae04d3c2, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_c346ae04d3c2
Claims without a verdict
- Unresolvable 157
- Not yet due 40
- Own records only 43
- Not researched 134
- Not checkable 864
Is a verdict here wrong, or do you speak for Dario Amodei? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.