Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
76 69 82 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
56 42 70 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 4 promises resolved, 5 needed for a score.
All types combined: 72 (likely 66 to 78, high confidence), from 93 claims: 76 factual claims, 17 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Dario Amodei's claims
49 claims
“Anthropic is unilaterally committing to this step now.”
Dario Amodei, CEO, writing for Anthropic 5 June 2025 We Must Pace the Frontier (darioamodei.com)
Anthropic is unilaterally committing now to the step of embedding third-party evaluators, which it describes as the key step for verifying any pacing commitments, comparable to regulatory supervisors embedded alongside employees in banking.
Mostly true True in substance, wrong in a detail.
The second judge said resolved true; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
This is a promise that Anthropic will give a team of embedded third-party evaluators (such as METR) ongoing, employee-like access to verify its safety practices. It is judged on delivery, not on the announcement. TRUE: independent evidence shows that such a team is embedded and has ongoing, employee-like access to Anthropic's systems and safety processes. Acceptable evidence includes a statement from the evaluator, a published agreement, or an independent report. MOSTLY TRUE: the arrangement is in place, but a minor detail differs (for example the evaluator is a different organisation, or access starts slightly narrower but ongoing). MOSTLY FALSE: only one-off or pre-deployment evaluations occur, or access is clearly short of employee-like. Also mostly false if the arrangement is only announced or in a pilot phase. FALSE: dated reports show the commitment was not implemented, was withdrawn, or that the evaluators named say they have no such access. Anthropic's own claim of delivery is not enough on its own.
Evidence
- Anthropic update on independent evaluators Anthropic · 15 October 2026 Own record
“We’ve onboarded six METR employees with company hardware, office space, and access to internal tools and systems that are roughly on par with what our own risk assessment teams have access to.”
- Anthropic gives outside AI watchdog “employee-like” access Ars Technica · 18 October 2026
“METR, a nonprofit that has worked with leading labs including OpenAI, Anthropic, Google, and Meta, now has a team embedded in the Claude maker’s office with access to its hardware and internal systems.”
- Review of the "Misalignment in high-stakes settings" section in Anthropic's Risk Report (August 2026) METR · 20 October 2026
“The authors of this review were engaged prior to the “An update on independent evaluators” announcement and did not have the level of access to information described in that post.”
- September 2026 progress report METR · 28 October 2026
“METR proposes using deeper company engagements to better inform the public about loss-of-control risks, which we will discuss in more detail in the next progress report.”
Reasoning
An independent outlet reports that a METR team is now embedded in Anthropic's office with access to its hardware and internal systems, described as employee-like access (E2). This matches Anthropic's account of onboarding six METR staff with access roughly on par with its own risk assessment teams (E1). METR's own review notes that earlier reviewers lacked this level of access, which is consistent with the arrangement being new (E3). Because the arrangement is new, the evidence cannot yet show sustained, ongoing access, and the passage from METR's progress report does not itself describe the arrangement (E4), so the verdict is mostly true rather than true.
Cite this claim
Dario Amodei, “Anthropic is unilaterally committing now to the step of embedding third-party evaluators, which it describes as the key step for verifying any pacing commitments, comparable to regulatory supervisors embedded alongside employees in banking.” (said 5 June 2025). Verdict: mostly true. Reliability API, claim c_d238d09e0a53, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_d238d09e0a53
Claims without a verdict
- Unresolvable 157
- Not yet due 40
- Own records only 43
- Not researched 134
- Not checkable 864
Is a verdict here wrong, or do you speak for Dario Amodei? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.