Claims

Factual · AI capabilities

“GPT-5 is significantly better at healthcare related queries.”

Sam Altman, OpenAI CEO · 9 August 2025 · Transcript: Sam Altman Interview on Huge Conversations Podcast (Singju Post)

GPT-5 is significantly better than earlier models at healthcare-related queries.

Mixed Partly true and partly false in roughly equal measure.

The second judge said resolved true; the verdict shown is the first judge's.

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

True if benchmark results available around 2025-08-09 (e.g., HealthBench or HealthBench Hard, medical QA benchmarks, hallucination rates on health queries) from OpenAI's published evaluations or independent tests show that GPT-5 performed significantly better on healthcare-related queries than earlier OpenAI models such as GPT-4o and o3: a clear, material improvement, not a marginal one. Mostly true if GPT-5 improved clearly on most health measures but only modestly against the strongest earlier model (e.g., o3) on some. Mixed if the improvements were significant on some health benchmarks but negligible or reversed on others, or if OpenAI's figures show gains that independent evaluations do not confirm. Mostly false if the gains were only marginal or limited to a narrow sub-measure presented as general. False if the evidence shows GPT-5 performed no better or worse than earlier models on healthcare queries.

Evidence

1. Introducing GPT-5 OpenAI · 7 August 2025 Own record

   “The model scores significantly higher than any previous model on HealthBench”

2. GPT-5 System Card - OpenAI Deployment Safety Hub OpenAI · 7 August 2025 Own record

   “Errors in potentially urgent situations are reduced over 50x from GPT-4o and over 8x from OpenAI o3.”

3. Clinical Accuracy and Safety Concerns Following GPT-5 Public Demonstration in Cancer Care PubMed Central (authors at University Hospital Tübingen, per data statement) · 29 November 2025

   “Overall concordance with tumour board recommendations was 79/100 for GPT-4o-mini (95% CI 70.02–85.83%), and 85/100 for both GPT-4o and GPT-5”

4. Clinical Accuracy and Safety Concerns Following GPT-5 Public Demonstration in Cancer Care PubMed Central (authors at University Hospital Tübingen, per data statement) · 29 November 2025

   “found identical 85% concordance to GPT-4o, contradicting superiority claims.”

Reasoning

OpenAI's launch post says GPT-5 scores "significantly higher than any previous model on HealthBench" (E1). Its system card reports that errors in potentially urgent health situations fell more than 50x from GPT-4o and more than 8x from o3 (E2). Both are OpenAI's own evaluations. An independent study of 100 cancer-care cases found GPT-5 and GPT-4o equally in line with tumour-board recommendations, at 85/100 each, which runs against the claimed superiority (E3, E4). Under the criteria, gains in OpenAI's figures that an independent evaluation does not confirm count as mixed, though the independent study covers only one narrow clinical task.

Cite this claim

Sam Altman, “GPT-5 is significantly better than earlier models at healthcare-related queries.” (said 9 August 2025). Verdict: mixed. Reliability API, claim c_a0968b3dfb9d, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_a0968b3dfb9d