AI hallucinations persist but are becoming much less frequent.

The claim: as of March 2025, AI (LLM) hallucinations still occur but are becoming much less frequent. True: benchmark or evaluation data available by about March 2025 (e.g. Vectara hallucination leaderboard, SimpleQA, vendor system cards, independent studies) show hallucinations still happen in current models, and newer models have substantially lower rates than earlier ones (e.g. roughly halving or better over 1–2 years). Mostly true: hallucinations persist and are declining, but the improvement is moderate or uneven, not 'much' less frequent. Mixed: some benchmarks show clear declines while others show flat or higher rates (e.g. reasoning models hallucinating more). Mostly false: hallucination rates are broadly flat, and only a narrow benchmark shows a decline. False: hallucinations have stopped occurring, or evaluations show they are generally becoming more frequent.

Set before the research began.

1. E1 OpenAI o3 and o4-mini System Card - OpenAI Deployment Safety Hub OpenAI

   “Dataset Metric o3 o4-mini o1 SimpleQA accuracy (higher is better) 0.49 0.20 0.47 hallucination rate (lower is better) 0.51 0.79 0.44 PersonQA accuracy (higher is better) 0.59 0.36 0.47 hallucination rate (lower is better) 0.33 0.48 0.16”

2. E2 DeepSeek-R1 hallucinates more than DeepSeek-V3 Vectara · 30 January 2025

   “The results were surprising: Deepseek-R1’s hallucination rate was 14.3% - much higher than its non-reasoning predecessor: Deepseek-V3.”

3. E3 DeepSeek-R1 hallucinates more than DeepSeek-V3 Vectara · 30 January 2025

   “According to Table 3, while HHEM 2.1 shows that the reasoning LLM (GPT-o1) has a higher hallucination rate than the generic LLM (GPT-4o), FACTS shows otherwise.”

4. E4 Measuring short-form factuality in large language models OpenAI

   “Current frontier models sometimes produce false outputs or answers that are not substantiated by evidence, a problem known as “hallucinations.””

E4 confirms that frontier models still hallucinate, so the first half of the claim holds. The evidence does not show hallucinations becoming much less frequent. Vectara's January 2025 test found DeepSeek-R1 at a 14.3% hallucination rate, much higher than its predecessor DeepSeek-V3 (E2). Vectara also found that comparing o1 with GPT-4o gives opposite results depending on the measure: HHEM 2.1 shows o1 hallucinating more, while FACTS shows otherwise (E3). OpenAI's undated o3/o4-mini system card shows higher hallucination rates for o3 and o4-mini than for o1 on both PersonQA and SimpleQA (E1). With one measure showing a decline and others showing higher rates for newer reasoning models, this matches the criteria's description of a mixed result.