Anthropic is open about the fact that its AI models sometimes make things up or get confused.
Resolved true The record shows it happened or was so.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The evidence is the person's or their organisation's own records.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judged as of 2026-05-09. True: Anthropic's public materials from around that date (model and system cards, Claude interface disclaimers, help-centre pages, research posts) openly say that its models can hallucinate or make errors and become confused. Mostly true: the acknowledgments exist but appear mostly in technical documents, or cover making things up but not confusion. Mixed: acknowledgment is inconsistent across channels. Mostly false: acknowledgments are rare or buried, while Anthropic's main messaging presents the models as reliable. False: the evidence shows Anthropic denied or hid these limitations.
Set before the research began.
Evidence (4)
-
E1 Claude is providing incorrect or misleading responses. What's going on? | Claude Help Center Anthropic (Claude Help Center) · 16 March 2026 Own record
“In an attempt to be a helpful assistant, Claude can occasionally produce responses that are incorrect or misleading. This is known as "hallucinating" information, and it’s a byproduct of some of the current limitations of frontier Generative AI models, like Claude.”
-
E2 Claude is providing incorrect or misleading responses. What's going on? | Claude Help Center Anthropic (Claude Help Center) · 16 March 2026 Own record
“For example, in some subject areas, Claude might not have been trained on the most-up-to-date information and may get confused when prompted about current events.”
-
E3 Tracing the thoughts of a large language model Anthropic · 27 March 2025 Own record
“Often this extended thinking gives better answers, but sometimes this "chain of thought" ends up being misleading; Claude sometimes makes up plausible-sounding steps to get where it wants to go.”
-
E4 Consumer Terms of Service Anthropic · 8 October 2025 Own record
“Outputs may not always be accurate and may contain material inaccuracies even if they appear accurate because of their level of detail or specificity.”
Reasoning
That page covers both parts of the proposition in a consumer-facing channel, not only in technical documents. An Anthropic research post (E3) says Claude sometimes makes up plausible-sounding reasoning steps. The consumer terms (E4) warn that outputs may contain material inaccuracies. All of these are Anthropic's own published materials, which here are the record the claim is about, and they were in place before the judging date.
Cite this claim
Daniela Amodei, “Anthropic is open about the fact that its AI models sometimes make things up or get confused.” (said 9 May 2026). Verdict: resolved true. Reliability API, claim c_17e2f2a99095, method 0.1.0+28da144b, dataset release v1.1.2. https://epistacklabs.com/claims/c_17e2f2a99095
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Daniela Amodei
-
Mostly true
“In the lab, Claude designed new protein binders from scratch.”
In laboratory work, Anthropic's Claude designed new protein binders from scratch.
-
Mostly true
“including open-source code much of the internet runs on, before attackers could exploit them.”
-
Resolved true
“They also showed that more than 6,000 women had come in for prenatal care, but only 1,762 had delivered with a healthcare worker present, a discrepancy that the team is investigating and addressing.”
-
Resolved true
“Those dashboards flagged a spike of 25,000 pneumonia cases, so the team can anticipate and plan for future surges.”
-
Resolved true
“Sometimes Claude is wrong.”
Verdicts are made automatically; how verdicts are made.