Current AI models now hallucinate only a little bit.
Mixed Partly true and partly false in roughly equal measure.
Factual: Statements about the present or the past, checked against the record: did they hold up?
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
True: as of 2025-09-11, benchmark and independent evaluation data show leading AI models (e.g. GPT-5) hallucinating at low rates in typical use, e.g. low single-digit percentages on standard factuality tests. Mostly true: hallucination rates are low in common settings but noticeably higher in some notable ones. Mixed: rates are low on some widely used benchmarks but substantial on others, such as open-ended factual questions without retrieval. Mostly false: rates fell but remain substantial on many standard measures, so 'a little bit' understates them, or the claim generalises from one model or setting. False: evaluations show current models still hallucinate often across common tasks.
Set before the research began.
Evidence (4)
-
E1 ChatGPT-5 is more trustworthy than before according to new tests, but this big rival wins the crown for spreading AI nonsense Yahoo Tech (Future syndication) · 12 August 2025
“According to Vectara, GPT-5 has a grounded hallucination rate of 1.4%, compared to 1.8% for GPT-4, and 1.69% for GPT-4 turbo and 4o mini, with 1.49% for GPT-4o.”
-
E2 ChatGPT-5 is more trustworthy than before according to new tests, but this big rival wins the crown for spreading AI nonsense Yahoo Tech (Future syndication) · 12 August 2025
“While hallucination rates are dropping amongst almost all LLMs, it's still surprisingly common, and one of the main reasons that we can't trust AI to perform a task without human supervision.”
-
E3 Why language models hallucinate OpenAI · 5 September 2025 Own record
“Hallucinations remain a fundamental challenge for all large language models, but we are working hard to further reduce them.”
-
E4 SimpleQA AI Wiki (aiwiki.ai) · 30 October 2026
“the August 2025 GPT-5 system card reported gpt-5-thinking at 55% accuracy with a 40% hallucination rate on SimpleQA”
Reasoning
E1 reports that, according to Vectara, GPT-5's grounded hallucination rate was 1.4%, a low single-digit figure. E4, however, cites the August 2025 GPT-5 system card as giving gpt-5-thinking a 40% hallucination rate on SimpleQA. E2, an independent article, describes hallucination as still surprisingly common even though rates are falling. Rates are low on one widely used benchmark and substantial on open-ended factual QA, which matches the criteria's definition of mixed.
Cite this claim
Sam Altman, “Current AI models now hallucinate only a little bit.” (said 11 September 2025). Verdict: mixed. Reliability API, claim c_4e1607e0cf00, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_4e1607e0cf00
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Sam Altman
-
Mostly true
“not until 2020 did we train GPT-3, which was the first model that I think actually was over some threshold of usability.”
OpenAI trained GPT-3 in 2020, which Altman considers the first model above a threshold of usability.
-
Resolved true
“that it is so capable that it is discovering new knowledge, doing science, able to add huge economic value, create entire pieces of complex software.”
-
Mixed
“They don’t do anything unless you ask, right?”
AI models such as ChatGPT do not do anything unless a user asks them to.
-
Mostly true
“But in terms of figuring out what questions to ask or to work on something over a very long period of time, we are definitely not close to human performance.”
-
Mostly true
“It’s an integrated model, so you don’t have to pick in our model switcher and know if you should use GPT-4o or o3 or o4 mini or any of the complicated things.”
Verdicts are made automatically; how verdicts are made.