ClaimsDario Amodei

“They're not yet at the stage where we think that there is a real and meaningful increase in the threat end to end,”

Dario Amodei, Anthropic CEO, interviewed 28 February 2025 Hard Fork - Anthropic's C.E.O. Dario Amodei on Surviving the A.I. Endgame Transcript (Podscripts (Hard Fork)) archived copy

Factual AI safety and alignment

Anthropic assessed that Claude 3.7 Sonnet is not yet at a stage giving a real and meaningful end-to-end increase in dangerous threat capability.

Resolved true The record shows it happened or was so.

Factual: Statements about the present or the past, checked against the record: did they hold up?

The evidence is the person's or their organisation's own records.

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

True: Anthropic's official Claude 3.7 Sonnet system card or RSP capability assessment (Feb 2025) concludes that the model does not meaningfully increase end-to-end dangerous-threat capability (e.g., it does not need ASL-3 protections for CBRN/catastrophic uplift) while it may show some partial uplift. Mostly true: Anthropic assessed it as below the threshold, with minor wording differences (e.g., 'cannot rule out' language or a precautionary note). Mixed: Anthropic's assessment was inconclusive or found meaningful uplift in some threat domains but not others. Mostly false: Anthropic's documents show it found meaningful uplift but chose not to apply higher safeguards for other reasons. False: Anthropic's assessment concluded that Claude 3.7 Sonnet did provide a real, meaningful end-to-end increase in threat capability (e.g., designated it ASL-3).

Set before the research began.

Evidence (4)

  1. E1 Claude 3.7 Sonnet System Card - www-cdn.anthropic.com Anthropic Own record

    “Based on our assessments, we’ve concluded that Claude 3.7 Sonnet is released under the ASL-2 standard.”

  2. E2 Claude 3.7 Sonnet System Card - www-cdn.anthropic.com Anthropic Own record

    “Altogether, we believe that Claude 3.7 Sonnet continues to be sufficiently far away from the ASL-3 capability thresholds, and therefore, we find that ASL-2 safeguards remain appropriate. However, the results from our evaluations suggest improved performance in all domains, including some uplift in CBRN evaluations.”

  3. E3 Claude 3.7 Sonnet System Card - www-cdn.anthropic.com Anthropic Own record

    “Under the most conservative approach to evaluation scoring, Claude 3.7 Sonnet achieved a score of 69.7% on this evaluation, suggesting that while this evaluation alone cannot conclusively rule out ASL-3, it cannot conclusively rule it in either.”

  4. E4 Claude 3.7 Sonnet System Card - www-cdn.anthropic.com Anthropic Own record

    “Consistent with our RSP, the RSO and CEO made the ultimate determination on the model’s ASL.”

Reasoning

The Claude 3.7 Sonnet System Card states the model was released under the ASL-2 standard, a determination made by the RSO and CEO (E1, E4). It concludes the model "continues to be sufficiently far away from the ASL-3 capability thresholds," while noting improved performance and some uplift in CBRN evaluations (E2), which matches the true criterion of no meaningful end-to-end uplift alongside partial uplift. One evaluation was inconclusive on its own (E3), but the overall conclusion is stated firmly.

Cite this claim

Dario Amodei, “Anthropic assessed that Claude 3.7 Sonnet is not yet at a stage giving a real and meaningful end-to-end increase in dangerous threat capability.” (said 28 February 2025). Verdict: resolved true. Reliability API, claim c_7e1a7071c18d, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_7e1a7071c18d

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Dario Amodei, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Dario Amodei

  1. Mostly true Factual 28 February 2025
    “So we did this experiment where we basically train the model to be all the good thing, helpful, honest, harmless, friendly.”

    Anthropic ran an experiment training a model to be helpful, honest, harmless and friendly.

  2. Resolved true Factual 28 February 2025
    “10 to 25% referred to the chance of civilization getting substantially derailed, right?”

    Amodei's earlier 10 to 25% figure referred to the chance of civilization getting substantially derailed.

  3. Mostly true Factual 18 February 2025
    “this lack of understanding is essentially unprecedented in the history of technology.”

    The lack of understanding of how AI systems work is essentially unprecedented in the history of technology.

  4. Mostly true Factual 18 February 2025
    “We were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue.”

    Amodei said that "we" were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue; the surrounding text does not say who "we" refers to.

  5. Mostly true Factual 18 February 2025
    “What changed at this time was that it became clear this was going to be a central problem for language models, much worse than in vision.”

    It became clear that superposition would be a central problem for language models, much worse than in vision models.

All of them

Verdicts are made automatically; how verdicts are made.