ClaimsDario Amodei

“我们和其他一些研究者发现,模型里有时会有某些特定的神经元和统计指标(这些通常不出现在它的外部反应中),这些指标能够指示模型是在撒谎还是在说真话。”

Dario Amodei, Anthropic CEO, interviewee 12 April 2024 Transcript: Ezra Klein Interviews Dario Amodei (translation) (baoyu.io) archived copy

Factual AI safety and alignment

我们和其他一些研究者发现,模型里有时会有某些特定的神经元和统计指标(这些通常不出现在它的外部反应中),这些指标能够指示模型是在撒谎还是在说真话。

Mixed Partly true and partly false in roughly equal measure.

Factual: Statements about the present or the past, checked against the record: did they hold up?

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

Judged as of 2024-04-12. True: published research by Anthropic and by other researchers before that date identified internal features, neurons or probe-detectable signals in language models that indicate whether the model's output is truthful or deceptive (for example Discovering Latent Knowledge, representation engineering, Geometry of Truth and Anthropic's probe or interpretability work on deception). Mostly true: other researchers' work clearly shows such internal truth or lie indicators, but Anthropic's own contribution addresses a closely related question (for example probes detecting deceptive 'sleeper agent' behaviour) rather than lying per se, or the signals are directions or probes rather than individual neurons. Mixed: the findings existed but were tentative and disputed in their reliability, or only Anthropic's or only others' work exists. Mostly false: the research showed only weak or inconsistent correlations, stated as a finding that the signals indicate lying. False: no such internal signal indicating lying or truthfulness had been demonstrated, or the work cited shows such probes failing.

Set before the research began.

Evidence (4)

  1. E1 The Geometry of Truth: Emergent Linear Structure in LLM Representations of True/False Datasets ar5iv (arXiv paper 2310.06824)

    “Overall, we present evidence that at sufficient scale, LLMs linearly represent the truth or falsehood of factual statements.”
  2. E2 The Geometry of Truth: Emergent Linear Structure in LLM Representations of True/False Datasets ar5iv (arXiv paper 2310.06824)

    “Recent work has developed techniques for inferring whether a LLM is telling the truth by training probes on the LLM’s internal activations. However, this line of work is controversial, with some authors pointing out failures of these probes to generalize in basic ways, among other conceptual issues.”
  3. E3 The Geometry of Truth: Emergent Linear Structure in LLM Representations of True/False Datasets ar5iv (arXiv paper 2310.06824)

    “This suggests these probes may be identifying not truth, but other features that correlate with truth on their training data.”
  4. E4 The Geometry of Truth: Emergent Linear Structure in LLM Representations of True/False Datasets ar5iv (arXiv paper 2310.06824)

    “In fact, the probes of Li et al. 2023b and Burns et al. 2023 are linear probes, suggesting the presence of a “truth direction” in model internals.”

Reasoning

E1 and E4 show that research by other groups (the Geometry of Truth paper, plus the probes of Li et al. and Burns et al.) found or suggested linear 'truth directions' in model activations that track whether a statement is true or false. The evidence describes these findings as linear probes and directions; it does not mention individual neurons. E2 and E3 also say this line of work was disputed, because the probes sometimes fail to generalise and may pick up features that only correlate with truth. None of the evidence covers Anthropic's own work, and E2 and E3 describe this line of work as disputed; the criteria's 'tentative and disputed' condition points to mixed.

Cite this claim

Dario Amodei, “我们和其他一些研究者发现,模型里有时会有某些特定的神经元和统计指标(这些通常不出现在它的外部反应中),这些指标能够指示模型是在撒谎还是在说真话。” (said 12 April 2024). Verdict: mixed. Reliability API, claim c_955999454309, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_955999454309

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Dario Amodei, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Dario Amodei

  1. Mostly true Factual 28 February 2025
    “So we did this experiment where we basically train the model to be all the good thing, helpful, honest, harmless, friendly.”

    Anthropic ran an experiment training a model to be helpful, honest, harmless and friendly.

  2. Resolved true Factual 28 February 2025
    “They're not yet at the stage where we think that there is a real and meaningful increase in the threat end to end,”

    Anthropic assessed that Claude 3.7 Sonnet is not yet at a stage giving a real and meaningful end-to-end increase in dangerous threat capability.

  3. Resolved true Factual 28 February 2025
    “10 to 25% referred to the chance of civilization getting substantially derailed, right?”

    Amodei's earlier 10 to 25% figure referred to the chance of civilization getting substantially derailed.

  4. Mostly true Factual 18 February 2025
    “this lack of understanding is essentially unprecedented in the history of technology.”

    The lack of understanding of how AI systems work is essentially unprecedented in the history of technology.

  5. Mostly true Factual 18 February 2025
    “We were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue.”

    Amodei said that "we" were able to provide a strong theoretical basis for having conviction that superposition was the right hypothesis to pursue; the surrounding text does not say who "we" refers to.

All of them

Verdicts are made automatically; how verdicts are made.