Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
79 72 85 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
80 72 88 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Gary Marcus's claims
“The lack of reliability will continue to haunt generative AI.”
Gary Marcus, Substack author 1 January 2025 25 AI Predictions for 2025, from Marcus on AI (Marcus on AI)
Lack of reliability will continue to afflict generative AI in 2025.
Resolved true The record shows it happened or was so.
What would count as true or false
The claim is that unreliability would remain a significant, widely recognised problem for generative AI during 2025. Unreliability here means hallucinations, factual errors, and inconsistent or unpredictable outputs. True: through 2025, credible sources document ongoing reliability problems in leading generative AI systems. Examples include hallucination benchmarks showing non-trivial error rates, prominent publicised failures such as fabricated legal citations or erroneous outputs, and industry or academic reports naming reliability as a key barrier. Mostly true: such problems persist but are widely reported as substantially reduced. Mixed: the evidence is split, with major systems widely treated as reliable for most uses but notable failures still occurring. Mostly false: credible evidence shows reliability largely stopped being a significant concern for leading systems, with only isolated incidents. False: by 2025 leading generative AI is broadly documented as reliable, and reliability is no longer a recognised problem.
Evidence
- AI chatbots unable to accurately summarise news, BBC finds BBC News · 11 February 2025
“It found 51% of all AI answers to questions about the news were judged to have significant issues of some form.”
- AI chatbots fail at accurate news, major study reveals Deutsche Welle (DW) · 21 October 2025
“the study found that almost half of all answers had at least one significant issue, while 31% contained serious sourcing problems and 20% contained major factual errors.”
- How Strong Is AI When Hallucinations Haunt? G2 · 2 April 2025
“On average, reviews mentioning hallucinations dropped from 35% in March 2025 to 8.3% in October 2025, according to G2 review data.”
- AI Hallucinations May Soon Be History Inside Higher Ed · 1 October 2025
“While more advanced models may reduce the frequency of obvious factual mistakes, the issue persists in more subtle forms.”
Reasoning
Independent studies from 2025 document ongoing reliability problems in leading chatbots. The BBC found that 51% of AI answers about the news had significant issues (E1). A major study reported by DW in October 2025 found that almost half of answers had at least one significant issue, and 20% contained major factual errors (E2). G2 review data shows mentions of hallucinations falling over 2025 (E3), and Inside Higher Ed says the problem persists in subtler forms (E4). Taken together, unreliability remained a widely recognised problem through 2025, which meets the criteria for true.
Cite this claim
Gary Marcus, “Lack of reliability will continue to afflict generative AI in 2025.” (said 1 January 2025). Verdict: resolved true. Reliability API, claim c_4eeff44d9f40, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_4eeff44d9f40
Claims without a verdict
Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.