Interpretability work on language models, such as induction heads, suggests much of the cognitive machinery needed for active agents is already present in base language models.

Judge as of 2023-08-08. This is a hedged interpretive claim ('seems'). True: published interpretability or capability research (e.g. induction heads, in-context learning, planning or tool-use evidence in base models) supports that base LLMs already contain much of the machinery agents use, such as in-context learning, pattern completion and reasoning, and this was a reasonable expert position then. Mostly true: there is supporting evidence but it is preliminary, and the claim slightly overstates it. Mixed: the evidence is genuinely split or shows only some relevant mechanisms. Mostly false: the cited work, e.g. induction heads, shows narrow mechanisms that do not reasonably support the broad claim about agent machinery, so a narrow finding is being stated as general. False: research then showed base models clearly lacked the core capabilities agents need.

Set before the research began.

1. E1 In-context Learning and Induction Heads Transformer Circuits Thread (Anthropic) · 8 March 2022 Own record

   “In particular, we present preliminary and indirect evidence for a tantalizing hypothesis: that induction heads might constitute the mechanism for the actual majority of all in-context learning in large transformer models.”

2. E2 In-context Learning and Induction Heads Transformer Circuits Thread (Anthropic) · 8 March 2022 Own record

   “Presently, fully reverse engineering them is beyond us, since they exist in large models with MLPs, which we don't have a strong framework for mechanistically understanding.”

3. E3 Blame for sources/arxiv-2209.11895.md · attention-wiki/knowledge-base at refs/pr/37 Hugging Face (attention-wiki knowledge-base dataset, archiver summary) Own record

   “the case for induction heads explaining in-context learning is stronger for small models than for large ones.”

4. E4 Benchmarking Agent Tool Use LangChain · 20 December 2023

   “Planning is still hard for LLMs - the likelihood of failure increases with the number of required steps, even for simple tasks.”

The only independent evidence (E4) is dated after the 2023-08-08 judging date and does not say whether base models were tested. The remaining items (E1–E3) are not independent of the subject and offer only preliminary support for in-context learning mechanisms, so they do not settle the claim under the criteria.