It became clear that superposition would be a central problem for language models, much worse than in vision models.

True: interpretability research, such as Anthropic's 'Toy Models of Superposition' (2022) and related work, documents a recognition that superposition is a central obstacle to interpreting language models and is more severe there than in vision models (for example fewer monosemantic neurons in LMs). Mostly true: superposition is documented as a central LM problem, but the comparison with vision is only implied or is less clear-cut. Mixed: the research supports one part only, either that it is central for LMs or that it is worse than in vision. Mostly false: superposition was recognised as an issue, but the literature does not treat it as central for LMs, or it treats it as comparably severe in vision. False: the research contradicts the claim.

Set before the research began.

1. E1 Towards Monosemanticity: Decomposing Language Models With Dictionary Learning Transformer Circuits Thread (Anthropic) · 4 October 2023 Own record

   “This is because we believe that the MLP layer likely uses superposition to represent more features than it has neurons (and correspondingly do more useful non-linear work!).”

2. E2 Polysemanticity | AI Wiki AI Wiki (aiwiki.ai) · 18 May 2026

   “Polysemanticity has been documented to be especially severe in transformer language models.”

3. E3 Toy models of superposition Anthropic · 14 September 2022 Own record

   “In this paper, we use toy models — small ReLU networks trained on synthetic data with sparse input features — to investigate how and when models represent more features than they have dimensions. We call this phenomenon superposition.”

4. E4 Zoom In: An Introduction to Circuits Distill · 10 March 2020

   “Despite this, polysemantic neurons are a major challenge for the circuits agenda, significantly limiting our ability to reason about neural networks.”

E1, Anthropic's 2023 dictionary-learning paper on language models, says the MLP layer is believed to use superposition to represent more features than it has neurons. E3 sets out the superposition phenomenon that research went on to study. E2, an independent secondary encyclopedia entry, states that polysemanticity is especially severe in transformer language models, which bears on the claim but does not explicitly compare language models with vision models. However, E4 shows polysemantic neurons were already called a major challenge for the circuits agenda in 2020, and no primary source here directly says superposition is much worse in language models than in vision, so the comparison is only partly supported.