ClaimsDario Amodei

“It's not anywhere near up to the task yet.”

Dario Amodei, Anthropic CEO 8 August 2023 Dario Amodei (Anthropic CEO) - $10 Billion Models, OpenAI, Scaling, & AGI in 2 years (Dwarkesh Podcast)

Factual AI safety and alignment

As of August 2023, mechanistic interpretability is not nearly capable enough to verify that a model is aligned.

Mostly true True in substance, wrong in a detail.

Factual: Statements about the present or the past, checked against the record: did they hold up?

The second judge would not resolve this claim; the verdict shown is the first judge's.

The evidence is the person's or their organisation's own records.

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

Judge as of August 2023. True: the state of the field at that time (published interpretability research, expert surveys and assessments, including from Anthropic, DeepMind and independent researchers) shows that mechanistic interpretability could not verify alignment of frontier-scale models, i.e. it was far from that capability. Mostly true: the field clearly could not do this, but some evidence shows meaningful partial capability (for example verification of limited properties in small models) that makes 'not anywhere near' a slight overstatement. Mixed: credible expert assessments of the time were split roughly evenly on whether interpretability was close to verifying alignment. Mostly false: contemporaneous evidence shows interpretability could already verify significant alignment-relevant properties of real models, so the claim greatly understates progress. False: evidence shows that interpretability methods in August 2023 could verify that frontier models were aligned.

Set before the research began.

Evidence (4)

  1. E1 arxiv-2209.11895.md HuggingFace dataset page summarising Olsson et al., 'In-context Learning and Induction Heads' (arXiv:2209.11895) · 2022 Own record

    “For small attention-only models, we present strong, causal evidence; for larger models with MLPs, we present correlational evidence”
  2. E2 arxiv-2209.11895.md HuggingFace dataset page summarising Olsson et al., 'In-context Learning and Induction Heads' (arXiv:2209.11895) · 2022 Own record

    “For 2-layer attention-only models, we were able to show precisely that induction heads implement this pattern copying behavior.”
  3. E3 Superposition, memorization, and double descent Anthropic · 5 January 2023 Own record

    “Despite overfitting being a central problem in machine learning, we have little mechanistic understanding of what exactly is going on when deep learning models overfit or memorize examples.”
  4. E4 Towards monosemanticity Anthropic · 5 October 2023 Own record

    “we have built machinery that lets us find these units in small transformer models.”

Reasoning

Anthropic-linked research from around that time reports its strongest mechanistic results on small models, with only correlational evidence for larger ones [E1][E4]. E1 says causal evidence covered only small attention-only models, and evidence for larger models was only correlational. E3 notes that the field had "little mechanistic understanding" of basic phenomena such as memorization, and E4 (October 2023) still describes the tools as working in "small transformer models". E2 shows real partial capability, a precise mechanistic explanation of one behaviour in 2-layer models, which fits the criteria's "mostly true" case, though nothing in the evidence suggests that alignment of frontier-scale models could be verified.

Cite this claim

Dario Amodei, “As of August 2023, mechanistic interpretability is not nearly capable enough to verify that a model is aligned.” (said 8 August 2023). Verdict: mostly true. Reliability API, claim c_5787ac923a7b, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_5787ac923a7b

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Dario Amodei, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Dario Amodei

  1. Mostly true Factual 28 February 2025
    “So we did this experiment where we basically train the model to be all the good thing, helpful, honest, harmless, friendly.”

    Anthropic ran an experiment training a model to be helpful, honest, harmless and friendly.

  2. Resolved true Factual 28 February 2025
    “They're not yet at the stage where we think that there is a real and meaningful increase in the threat end to end,”

    Anthropic assessed that Claude 3.7 Sonnet is not yet at a stage giving a real and meaningful end-to-end increase in dangerous threat capability.

  3. Resolved true Factual 28 February 2025
    “10 to 25% referred to the chance of civilization getting substantially derailed, right?”

    Amodei's earlier 10 to 25% figure referred to the chance of civilization getting substantially derailed.

  4. Mostly true Factual 18 February 2025
    “this lack of understanding is essentially unprecedented in the history of technology.”

    The lack of understanding of how AI systems work is essentially unprecedented in the history of technology.

  5. Mostly true Factual 18 February 2025
    “We were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue.”

    Amodei said that "we" were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue; the surrounding text does not say who "we" refers to.

All of them

Verdicts are made automatically; how verdicts are made.