ClaimsGary Marcus

“Examples of concealed bias will be discovered within days or months.”

Gary Marcus, author 27 July 2025 What to Expect When You're Expecting … GPT-5 (Marcus on AI) archived copy

Prediction AI safety and alignment

Examples of concealed bias in GPT-5 will be discovered within days or months of release.

Unresolvable The public record could not settle it.

Prediction: Statements about events outside the person's control, checked once they fell due: did they come true?

The second judge would not resolve this claim; the verdict shown is the first judge's.

What would count as true or false

GPT-5 was released in August 2025. Read 'within days or months' as no later than about 12 months after release. True: within that window, researchers, journalists or users publicly document specific examples of bias in GPT-5's outputs (for example political, gender, racial or cultural bias) that OpenAI had not disclosed beforehand and that GPT-5 was supposedly free of. Mostly true: such examples are documented in the window but are few or weakly supported (anecdotal rather than systematic). Mostly false: bias examples are documented only after the window, or only for other models, or only for biases OpenAI had already disclosed. False: systematic, independent bias evaluations of GPT-5 within the window report no undisclosed bias.

Set before the research began.

Evidence (4)

  1. E1 New model, old risks: sociodemographic bias and adversarial hallucinations vulnerability in GPT-5 - npj Digital Medicine npj Digital Medicine (Nature Portfolio) · 4 April 2026

    “GPT-5 showed no measurable improvement over GPT-4o in sociodemographic-linked decision variation, with several LGBTQIA+ groups flagged for mental-health screening in 100% of cases.”
  2. E2 The Cheap Fix a Contract Pays For seldondance (Substack, independent author) · 29 October 2026

    “On the 500 most partisan items, with nothing above the question, the six models gave answers closer to Democrats than to Republicans on 70 to 77 percent of items.”
  3. E3 Like humans, language models demonstrate face-to-character biases PNAS Nexus (Oxford Academic) · 3 August 2026

    “To ensure these results were not an idiosyncratic feature of a particular LLM (GPT-4o), in experiments 8–13, we showed the same patterns, usually with an even greater degree of bias, in GPT-5, Gemini 3 Flash Preview, and Claude Sonnet 4.5.”
  4. E4 Measuring political bias in Claude Anthropic · 13 November 2025

    “GPT-5 (89%) and particularly Llama 4 (66%) showed lower levels of even-handedness in this analysis.”

Reasoning

Within the window, E1 (April 2026) and E4 (November 2025) document bias-related findings for GPT-5, and E3 is dated at the window's edge. None of the evidence shows whether OpenAI had disclosed these biases beforehand, and the criteria use that to separate true from mostly false.

Cite this claim

Gary Marcus, “Examples of concealed bias in GPT-5 will be discovered within days or months of release.” (said 27 July 2025). Verdict: unresolvable. Reliability API, claim c_13c73191bf93, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_13c73191bf93

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Gary Marcus, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More prediction claims by Gary Marcus

  1. Resolved true Prediction 27 July 2025
    “Guardrails (a la ChatGPT) may be in place, but the guardrails will teeter between being too weak (beaten by “jailbreaks”) and too strong (rejecting some perfectly reasonable requests).”

    If the model being discussed has guardrails like ChatGPT's, they will teeter between being too weak (beaten by jailbreaks) and too strong (rejecting some perfectly reasonable requests).

  2. Mostly true Prediction 25 December 2022
    ““Alignment” between what humans want and what machines do will continue to be a critical, unsolved problem.”

    AI alignment between human wants and machine behavior will remain a critical unsolved problem, with GPT-4 unable to reliably follow shared human values.

  3. Mostly true Prediction 27 July 2025
    “Some of its advice will be head-scratchingly bad.”

    Some of GPT-5's advice will be very bad.

  4. Mostly true Prediction 27 July 2025
    “GPT-5 will surely be better, a lot better than GPT-4.”

    GPT-5 will be considerably better than GPT-4.

  5. Resolved true Prediction 27 July 2025
    “GPT-5 really is about to drop, no foolin’.”

    OpenAI will release GPT-5 imminently.

All of them

Verdicts are made automatically; how verdicts are made.