Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
76 70 82 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
60 42 78
All types combined: 74 (likely 68 to 80, high confidence), from 79 claims: 71 factual claims, 8 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Eliezer Yudkowsky's claims
46 claims
“The predict-as-if-honest circuitry is contextual, invoked to predict some streams of text but not others.”
Eliezer Yudkowsky, author on X 20 April 2026 Post by @ESYudkowsky on X, 2026-04-20 (X)
LLMs' predict-as-if-honest circuitry is contextual, invoked for predicting some text streams but not others.
Mostly true True in substance, wrong in a detail.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judged against empirical and interpretability research on large language models as of 20 April 2026. TRUE: published research shows that the internal mechanisms or representations LLMs use to produce honest or truthful continuations are activated depending on context. For example, interpretability work showing truthfulness directions or features that are engaged or suppressed depending on persona, prompt or genre, or that models represent the truth but output falsehoods in some contexts. MOSTLY TRUE: strong behavioural and representational evidence shows honesty-related processing is context-dependent, but no discrete 'circuit' has been identified, and the claim's word 'circuitry' is loose. MIXED: the evidence is split, with some studies finding context-dependent honesty mechanisms and others finding context-independent truth representations that are used consistently. MOSTLY FALSE: the claim states as established a mechanistic picture that the evidence does not yet support, e.g. only behavioural findings of context-dependent lying, with no mechanistic support, or interpretive speculation. FALSE: research shows that honesty-related mechanisms in LLMs are invoked uniformly regardless of context.
Evidence
- Paper page - The Truthfulness Spectrum Hypothesis Hugging Face (paper page; authors' arXiv-linked work) · 26 February 2026
“Linear probes generalize well across most domains but fail on sycophantic and expectation-inverted lying.”
- The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blind arXiv (ar5iv rendering) · 26 June 2025
“Analysis revealed that autolabeled SAE features for "deception" rarely activated during strategic dishonesty, and feature steering experiments across 100+ deception-related features failed to prevent lying.”
- How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs ACL Anthology (Adarsh, Maistro, Lioma; University of Copenhagen) · 14 September 2026
“We also find that context conflicting with parametric knowledge produces larger geometric changes than parametrically aligned context.”
Reasoning
Together these support context-dependent honesty-related processing, but no discrete 'circuit' switching honest prediction on or off has been identified, and E1 also finds broad cross-domain generalisation. E3 post-dates the judgment date and was not relied on.
Cite this claim
Eliezer Yudkowsky, “LLMs' predict-as-if-honest circuitry is contextual, invoked for predicting some text streams but not others.” (said 20 April 2026). Verdict: mostly true. Reliability API, claim c_84b75273b1ce, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_84b75273b1ce
Claims without a verdict
Is a verdict here wrong, or do you speak for Eliezer Yudkowsky? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.