Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
76 69 82 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
56 42 70 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 4 promises resolved, 5 needed for a score.
All types combined: 72 (likely 66 to 78, high confidence), from 93 claims: 76 factual claims, 17 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Dario Amodei's claims
49 claims
“We had this blog post out about security where we said we have a two key system for access to the model weights.”
Dario Amodei, Anthropic CEO 8 August 2023 Dario Amodei (Anthropic CEO) - $10 Billion Models, OpenAI, Scaling, & AGI in 2 years (Dwarkesh Podcast)
Anthropic published a blog post on security stating it has a two-key system for access to model weights.
Mostly true True in substance, wrong in a detail.
The second judge said mixed; the verdict shown is the first judge's.
The evidence is the person's or their organisation's own records.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judge as of 2023-08-08. True: an Anthropic blog post or official post on security, published on or before 2023-08-08, says Anthropic uses a two-party/two-key (multi-party authorization) control for access to model weights or the systems that hold them. Mostly true: such a post exists and describes two-person or multi-party access control over weights, but with slightly different wording (e.g. 'two-party control' or 'multi-party authorization') or applied to production systems that include the weights. Mixed: the post describes multi-party controls but does not clearly apply them to model weights. Mostly false: Anthropic published a security post, but it does not describe a two-key or multi-party access system, or the description appeared only after 2023-08-08. False: Anthropic's security publications before that date contain nothing of the kind.
Evidence
- Frontier model security Anthropic · 25 July 2023 Own record
“Anthropic is implementing two-party controls, SSDF, SLSA, and other cybersecurity best practices.”
- Frontier model security Anthropic · 25 July 2023 Own record
“This pattern should be applied to all systems involved in the development, training, hosting, and deployment of frontier AI models.”
- Frontier model security Anthropic · 25 July 2023 Own record
“two people with two keys are needed to open the most secure vaults, and multi-party review patterns have been applied in manufacturing”
Reasoning
Anthropic's 'Frontier model security' post, dated 2023-07-25, states that Anthropic is 'implementing two-party controls' (E1). It says this pattern should apply to all systems involved in developing, training, hosting and deploying frontier AI models [E2]. In the quoted text, 'two keys' appears as an analogy [E3], and the quoted passages do not name model weights directly. This matches the 'mostly true' criterion: different wording, applied to systems that include the weights.
Cite this claim
Dario Amodei, “Anthropic published a blog post on security stating it has a two-key system for access to model weights.” (said 8 August 2023). Verdict: mostly true. Reliability API, claim c_35811f4f9e16, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_35811f4f9e16
Claims without a verdict
- Unresolvable 157
- Not yet due 40
- Own records only 43
- Not researched 134
- Not checkable 864
Is a verdict here wrong, or do you speak for Dario Amodei? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.