Anthropic conducts interpretability research aimed at seeing inside language models and understanding their workings.

Judge as of 2022-03-04. True if by that date Anthropic was conducting research aimed at understanding the internal workings of language models, shown by published papers, research posts or team descriptions (e.g. mechanistic interpretability work on transformers). Mostly true if such research was under way but its focus was slightly different from 'language models' (e.g. related neural network models). Mostly false if interpretability was only planned or announced with no work under way by that date. False if Anthropic had no interpretability research programme at that time.

Set before the research began.

1. E1 A Mathematical Framework for Transformer Circuits Transformer Circuits Thread (Anthropic) · 22 December 2021 Own record

   “Transformer language models are an emerging technology that is gaining increasingly broad real-world use, for example in systems like GPT-3 , LaMDA , Codex , Meena , Gopher , and similar models.”

2. E2 A mathematical framework for Transformer Circuits Anthropic · 22 December 2021 Own record

   “Reverse-engineering small attention-only transformers reveals how induction heads form via head composition.”

3. E3 Chris Olah Wiki: Co-founder at Anthropic | AI People Wiki AI People Wiki · 22 December 2021

   “At Anthropic, Olah is the corresponding author of "A Mathematical Framework for Transformer Circuits" (2021), last-listed author of "Toy Models of Superposition" (2022)”

E1 and E2 are Anthropic's published paper "A Mathematical Framework for Transformer Circuits," dated 2021-12-22, before the judge-as-of date of 2022-03-04. In it Anthropic reverse-engineers small attention-only transformers to explain how induction heads form, and it frames the work around transformer language models. E3, an independent wiki, attributes this 2021 paper to Anthropic. Together they show interpretability research into the internals of language models was under way by the claim date.