GPT-4 has not solved hallucinations, reliability or planning on complex tasks, as the paper's authors acknowledge for planning.

Judged as of 2023-03-24. The claim has three parts: GPT-4 had not solved (1) hallucinations, (2) reliability, and (3) planning on complex tasks. It also says the paper's authors acknowledged (3). True: evidence from that time shows GPT-4 still produced hallucinations and unreliable outputs and failed at complex planning, and the paper discussed (e.g. Bubeck et al.) acknowledges limits in planning. OpenAI's own GPT-4 report admitting hallucination and reliability limits counts toward (1) and (2). Mostly true: all three limits are confirmed but the attribution is slightly off, e.g. the authors acknowledged the planning limit only in qualified or different terms. Mixed: one of the three limits is shown to have been substantially solved at that time, or the attribution is plainly wrong while the substantive limits hold. Mostly false: evidence from that time shows two of the three were substantially solved. False: evidence shows GPT-4 had effectively eliminated hallucinations and unreliability and could plan complex tasks reliably.

Set before the research began.

1. E1 GPT-4 OpenAI · 14 March 2023

   “Most importantly, it still is not fully reliable (it “hallucinates” facts and makes reasoning errors).”

2. E2 GPT-4 Technical Report arXiv (OpenAI authors)

   “Most importantly, it still is not fully reliable (it “hallucinates” facts and makes reasoning errors).”

3. E3 Sparks of Artificial General Intelligence: Early experiments with GPT-4 arXiv (ar5iv rendering)

   “Throughout the paper we emphasize limitations whenever we found one, but we also dedicate Section to an in-depth analysis of the lack of planning, likely a direct consequence of the autoregressive nature of GPT-4’s architecture.”

4. E4 GPT-4 Technical Report arXiv (OpenAI authors)

   “GPT-4 significantly reduces hallucinations relative to previous GPT-3.5 models (which have themselves been improving with continued iteration).”

OpenAI's GPT-4 announcement and technical report (E1, E2) say GPT-4 "still is not fully reliable" and that it "hallucinates" facts and makes reasoning errors, which supports parts (1) and (2). The report also says GPT-4 significantly reduces hallucinations compared with GPT-3.5 (E4), but that is a reduction, not a solution. The Sparks of AGI paper by Bubeck et al. (E3) gives "an in-depth analysis of the lack of planning" in GPT-4, which supports part (3) and the claim that the authors acknowledged it.