As of August 2023, mechanistic interpretability is not nearly capable enough to verify that a model is aligned.
Mostly true True in substance, wrong in a detail.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
The evidence is the person's or their organisation's own records.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judge as of August 2023. True: the state of the field at that time (published interpretability research, expert surveys and assessments, including from Anthropic, DeepMind and independent researchers) shows that mechanistic interpretability could not verify alignment of frontier-scale models, i.e. it was far from that capability. Mostly true: the field clearly could not do this, but some evidence shows meaningful partial capability (for example verification of limited properties in small models) that makes 'not anywhere near' a slight overstatement. Mixed: credible expert assessments of the time were split roughly evenly on whether interpretability was close to verifying alignment. Mostly false: contemporaneous evidence shows interpretability could already verify significant alignment-relevant properties of real models, so the claim greatly understates progress. False: evidence shows that interpretability methods in August 2023 could verify that frontier models were aligned.
Set before the research began.
Evidence (4)
-
E1 arxiv-2209.11895.md HuggingFace dataset page summarising Olsson et al., 'In-context Learning and Induction Heads' (arXiv:2209.11895) · 2022 Own record
“For small attention-only models, we present strong, causal evidence; for larger models with MLPs, we present correlational evidence”
-
E2 arxiv-2209.11895.md HuggingFace dataset page summarising Olsson et al., 'In-context Learning and Induction Heads' (arXiv:2209.11895) · 2022 Own record
“For 2-layer attention-only models, we were able to show precisely that induction heads implement this pattern copying behavior.”
-
E3 Superposition, memorization, and double descent Anthropic · 5 January 2023 Own record
“Despite overfitting being a central problem in machine learning, we have little mechanistic understanding of what exactly is going on when deep learning models overfit or memorize examples.”
-
E4 Towards monosemanticity Anthropic · 5 October 2023 Own record
“we have built machinery that lets us find these units in small transformer models.”
Reasoning
Anthropic-linked research from around that time reports its strongest mechanistic results on small models, with only correlational evidence for larger ones [E1][E4]. E1 says causal evidence covered only small attention-only models, and evidence for larger models was only correlational. E3 notes that the field had "little mechanistic understanding" of basic phenomena such as memorization, and E4 (October 2023) still describes the tools as working in "small transformer models". E2 shows real partial capability, a precise mechanistic explanation of one behaviour in 2-layer models, which fits the criteria's "mostly true" case, though nothing in the evidence suggests that alignment of frontier-scale models could be verified.
Cite this claim
Dario Amodei, “As of August 2023, mechanistic interpretability is not nearly capable enough to verify that a model is aligned.” (said 8 August 2023). Verdict: mostly true. Reliability API, claim c_5787ac923a7b, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_5787ac923a7b
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Dario Amodei
-
Mostly true
“So we did this experiment where we basically train the model to be all the good thing, helpful, honest, harmless, friendly.”
Anthropic ran an experiment training a model to be helpful, honest, harmless and friendly.
-
Resolved true
“They're not yet at the stage where we think that there is a real and meaningful increase in the threat end to end,”
-
Resolved true
“10 to 25% referred to the chance of civilization getting substantially derailed, right?”
-
Mostly true
“this lack of understanding is essentially unprecedented in the history of technology.”
-
Mostly true
“We were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue.”
Verdicts are made automatically; how verdicts are made.