Anthropic is unilaterally committing now to the step of embedding third-party evaluators, which it describes as the key step for verifying any pacing commitments, comparable to regulatory supervisors embedded alongside employees in banking.
Mostly true True in substance, wrong in a detail.
Promise: Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
The second judge said resolved true; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
This is a promise that Anthropic will give a team of embedded third-party evaluators (such as METR) ongoing, employee-like access to verify its safety practices. It is judged on delivery, not on the announcement. TRUE: independent evidence shows that such a team is embedded and has ongoing, employee-like access to Anthropic's systems and safety processes. Acceptable evidence includes a statement from the evaluator, a published agreement, or an independent report. MOSTLY TRUE: the arrangement is in place, but a minor detail differs (for example the evaluator is a different organisation, or access starts slightly narrower but ongoing). MOSTLY FALSE: only one-off or pre-deployment evaluations occur, or access is clearly short of employee-like. Also mostly false if the arrangement is only announced or in a pilot phase. FALSE: dated reports show the commitment was not implemented, was withdrawn, or that the evaluators named say they have no such access. Anthropic's own claim of delivery is not enough on its own.
Set before the research began.
Evidence (4)
-
E1 Anthropic update on independent evaluators Anthropic · 15 October 2026 Own record
“We’ve onboarded six METR employees with company hardware, office space, and access to internal tools and systems that are roughly on par with what our own risk assessment teams have access to.”
-
E2 Anthropic gives outside AI watchdog “employee-like” access Ars Technica · 18 October 2026
“METR, a nonprofit that has worked with leading labs including OpenAI, Anthropic, Google, and Meta, now has a team embedded in the Claude maker’s office with access to its hardware and internal systems.”
-
E3 Review of the "Misalignment in high-stakes settings" section in Anthropic's Risk Report (August 2026) METR · 20 October 2026
“The authors of this review were engaged prior to the “An update on independent evaluators” announcement and did not have the level of access to information described in that post.”
-
E4 September 2026 progress report METR · 28 October 2026
“METR proposes using deeper company engagements to better inform the public about loss-of-control risks, which we will discuss in more detail in the next progress report.”
Reasoning
An independent outlet reports that a METR team is now embedded in Anthropic's office with access to its hardware and internal systems, described as employee-like access (E2). This matches Anthropic's account of onboarding six METR staff with access roughly on par with its own risk assessment teams (E1). METR's own review notes that earlier reviewers lacked this level of access, which is consistent with the arrangement being new (E3). Because the arrangement is new, the evidence cannot yet show sustained, ongoing access, and the passage from METR's progress report does not itself describe the arrangement (E4), so the verdict is mostly true rather than true.
Cite this claim
Dario Amodei, “Anthropic is unilaterally committing now to the step of embedding third-party evaluators, which it describes as the key step for verifying any pacing commitments, comparable to regulatory supervisors embedded alongside employees in banking.” (said 5 June 2025). Verdict: mostly true. Reliability API, claim c_d238d09e0a53, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_d238d09e0a53
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More promise claims by Dario Amodei
-
Mostly true
“Anthropic is unilaterally committing to the first of these steps.”
-
Resolved true
“When we receive some kind of formal action, we will look at it, we will understand it, and we will challenge it in court.”
-
Delivered on time
“We will have web search very soon.”
Verdicts are made automatically; how verdicts are made.