Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
83 78 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
Insufficient record: 2 predictions resolved, 5 needed for a score.
-
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 83 (likely 78 to 88, high confidence), from 86 claims: 86 factual claims. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Daniela Amodei's claims
43 claims
“we released a writeup on Garcon, which is the infrastructure tool that our team used to conduct the research”
Daniela Amodei, Anthropic president, in podcast interview 2022 Podcast Transcript: Daniela and Dario Amodei on Anthropic (LessWrong)
Anthropic released a writeup on Garcon, the infrastructure tool its team used for interpretability research.
Resolved true The record shows it happened or was so.
The evidence is the person's or their organisation's own records.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judged as of 2022. True: Anthropic published, on or before the date the statement was made, a writeup describing Garcon as the infrastructure its team used for interpretability research (for example, on transformer-circuits.pub). Mostly true: the writeup exists, but a minor detail differs, for example it was published as part of another article rather than on its own, or it was released by members of the team rather than by Anthropic as such. Mostly false: Garcon is only mentioned in passing in other work, without a writeup, or the writeup appeared only after the statement. False: Anthropic has no published material on a tool called Garcon.
Evidence
- Garcon Transformer Circuits Thread (transformer-circuits.pub); the page does not name a publisher separately · 22 December 2021 Own record
“This document describes one of the core pieces of infrastructure we've built to enable interpretability research at Anthropic: a tool we call Garçon.”
- Garcon Transformer Circuits Thread (transformer-circuits.pub); the page does not name a publisher separately · 22 December 2021 Own record
“We've found Garçon very helpful for our research. We now use it for essentially all of our interpretability work, and other teams have also adopted it for ad-hoc exploration of models.”
- A Mathematical Framework for Transformer Circuits Transformer Circuits Thread (transformer-circuits.pub); the page does not name a publisher separately · 22 December 2021 Own record
“The crucial piece of software infrastructure which made our analysis of models possible is a library called Garcon (described in another article) created by Nelson Elhage with help from Tom Brown, Sam McCandlish, and Chris Olah.”
Reasoning
E1 is a standalone Garçon writeup on transformer-circuits.pub, dated 2021-12-22, which is before the 2022 statement. It describes Garçon as "one of the core pieces of infrastructure we've built to enable interpretability research at Anthropic." E2, from the same page, says the team uses it for essentially all of its interpretability work. E3 is a companion paper that names Garcon as the key infrastructure and points to the separate article. The evidence is the published writeup itself, which matches the criteria for true.
Cite this claim
Daniela Amodei, “Anthropic released a writeup on Garcon, the infrastructure tool its team used for interpretability research.” (said 2022). Verdict: resolved true. Reliability API, claim c_d4198b2fec5b, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_d4198b2fec5b
Claims without a verdict
Is a verdict here wrong, or do you speak for Daniela Amodei? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.