Hallucinations are inherent to how LLMs make predictions.

Judge against research as of March 2024. The claim is that hallucinations (fluent but false outputs) are inherent to how LLMs make predictions: they arise from the next-token prediction mechanism itself, not only from fixable flaws in data or training. TRUE: research or theoretical results available by then show, or broadly accept, that hallucination follows from how LLMs are built and cannot be fully eliminated within that design. Examples are formal arguments that hallucination is inevitable, or widely accepted analyses tracing it to probabilistic next-token generation. MOSTLY TRUE: the literature broadly supports a structural cause but treats full inevitability as not yet proven. MIXED: the research is genuinely split between structural causes and fixable training or data causes. MOSTLY FALSE: the claim is stated as established, but the research of the time was inconclusive or pointed mainly to contingent causes. FALSE: evidence shows that LLMs can be made essentially hallucination-free without departing from their prediction mechanism, or that hallucination is shown not to be tied to that mechanism.

Set before the research began.

1. E1 Hallucination is Inevitable: An Innate Limitation of Large Language Models arXiv (preprint; ar5iv HTML rendering)

   “In this paper, we formalize the problem and show that it is impossible to eliminate hallucination in LLMs.”

2. E2 Hallucination is Inevitable: An Innate Limitation of Large Language Models arXiv (preprint; ar5iv HTML rendering)

   “Under this definition, we present a fundamental result that hallucination is inevitable for any computable LLM, regardless of model architecture, learning algorithms, prompting techniques, or training data.”

3. E3 Hallucination is Inevitable: An Innate Limitation of Large Language Models arXiv (preprint; ar5iv HTML rendering)

   “It shows a significant fact that pretraining LLMs for predictive accuracy leads to hallucination even with perfect training data.”

4. E4 A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions arXiv (survey preprint) · 15 March 2023

   “Recognizing the multifaceted sources of LLM hallucinations, our survey identifies potential contributors into three main aspects: data, training, and inference stages.”

By early 2024, a preprint (arXiv 2401.11817) gave a formal argument that hallucination cannot be eliminated in any computable LLM, whatever the architecture, training or data (E1, E2). It also states that pretraining LLMs for predictive accuracy leads to hallucination even with perfect training data (E3). Both points tie hallucination to the prediction mechanism itself. However, a survey of the time grouped potential contributors to hallucination into data, training and inference stages (E4), and the inevitability result was a preprint. The research therefore broadly supports a structural cause, but full inevitability was not yet settled.