Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
79 72 85 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
80 72 88 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Gary Marcus's claims
42 claims
“AI “Agents” will be endlessly hyped throughout 2025 but far from reliable, except possibly in very narrow use cases.”
Gary Marcus, Substack author 1 January 2025 25 AI Predictions for 2025, from Marcus on AI (Marcus on AI)
AI agents will be heavily hyped throughout 2025 but remain far from reliable, except possibly in very narrow use cases.
Resolved true The record shows it happened or was so.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
This claim has two parts, judged over calendar year 2025: (a) AI agents were heavily hyped throughout 2025; (b) they remained far from reliable outside narrow use cases. True: both hold. Heavy hype means widespread industry marketing and media coverage calling 2025 'the year of agents' or similar, and major product launches. Far from reliable means independent benchmarks or evaluations of general-purpose agents in 2025 (for example multi-step computer-use or office-task benchmarks) showed low or inconsistent success rates, and reports of failures, with reliable deployment limited to narrow tasks such as coding help or customer support scripts. Mostly true: both broadly hold, but agents became reasonably reliable in one or two moderately broad areas beyond 'very narrow' ones. Mixed: only one of the two parts clearly holds. Mostly false: hype was modest, or agents were reliable across several broad domains, with only limited unreliability. False: agents were not heavily hyped, or credible evidence shows general-purpose agents performed reliably across broad tasks by the end of 2025.
Evidence
- The State of Agentic AI in 2025: A Year-End Reality Check — Arion Research LLC Arion Research LLC · 30 December 2025
“Predictions of agents working independently for hours or days on complex projects didn't materialize. Most successful agents still need frequent human check-ins or handle only bounded, well-defined tasks.”
- Was 2025 Really the Year of AI Agents? IEEE Spectrum · 29 January 2026
“Altman proclaimed that “in 2025, we may see the first AI agents ‘join the workforce’ and materially change the output of companies.” His remarks set the tenor for the AI industry through 2025.”
- Gartner Survey Finds 45% of Martech Leaders Say Existing Vendor-Offered AI Agents Fail to Meet Their Expectations of Promised Business Performance Gartner, Inc. · 29 October 2025
“Forty-five percent of martech leaders with AI agents in pilots or production say existing vendor-offered AI agent capabilities do not meet company expectations of promised business performance, according to a survey by Gartner, Inc., a business and technology insights company.”
- TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks arXiv (academic paper by its authors) · 22 October 2024
“We find in experiments that the best performing model, Gemini 2.5 Pro was able to autonomously perform 30.3% of the provided tests to completion, and achieve a score of 39.3% on our metric that provides extra credit for partially completed tasks.”
Reasoning
On hype, IEEE Spectrum (E2) says Sam Altman's prediction that agents would "join the workforce" in 2025 set the tone for the AI industry throughout the year. It also asks whether 2025 really was "the year of AI agents". On reliability, a year-end analyst review (E1) says agents did not work independently on complex projects as predicted, and that the successful ones still needed frequent human check-ins or handled only bounded tasks. A Gartner survey (E3) found that 45% of martech leaders said vendor AI agents fell short of the promised business performance, and on the TheAgentCompany benchmark (E4) the best model, Gemini 2.5 Pro, fully completed only 30.3% of realistic workplace tasks. Taken together, the sources support both parts of the prediction: heavy hype, and agents that stayed far from reliable outside narrow, bounded uses.
Cite this claim
Gary Marcus, “AI agents will be heavily hyped throughout 2025 but remain far from reliable, except possibly in very narrow use cases.” (said 1 January 2025). Verdict: resolved true. Reliability API, claim c_ac88e060b15d, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_ac88e060b15d
Claims without a verdict
Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.