Claims

Prediction · AI capabilities

“People building things like virtual assistants and agents will find that they cannot reliably enough map user language onto user intentions.”

Gary Marcus, author · 27 July 2025 · What to Expect When You're Expecting … GPT-5 (Marcus on AI)

Developers building virtual assistants and agents on GPT-5 will find they cannot map user language onto user intentions reliably enough.

Unresolvable The public record could not settle it.

What would count as true or false

Judge over about a year after GPT-5's August 2025 release (to August 2026). True: credible evidence (developer reports, evaluations of GPT-5-based agents, benchmarks of agentic task success, industry surveys) shows that assistants and agents built on GPT-5 commonly misread user intent often enough that developers report it as a significant barrier to reliable deployment. Mostly true: intent-mapping failures are widely documented as a significant problem, but some substantial deployments report acceptable reliability. Mixed: the evidence is roughly split between reports of reliable intent understanding and reports of significant failures. Mostly false: most developers and evaluations find GPT-5 maps user language to intent reliably enough for production, with failures only in edge cases. False: GPT-5-based assistants and agents are broadly found to interpret user intentions reliably, and misinterpretation is not a notable barrier. Failures caused mainly by other factors (tool errors, cost, latency) do not count towards the claim.

Evidence

1. Agentic AI in Industry: Adoption Level and Deployment Barriers arXiv · 31 January 2026

   “Greater task ownership also amplifies friction already visible in LLM-based tools: systems may omit aspects of developer intent, vary across equivalent prompts, hallucinate, or lose relevant information as context grows [13, 7].”

2. Agentic AI in Industry: Adoption Level and Deployment Barriers arXiv · 31 January 2026

   “Human verification was therefore used to check both technical output and semantic intent.”

3. Agentic AI in Industry: Adoption Level and Deployment Barriers arXiv · 31 January 2026

   “Across the cases, four previously identified barriers recurred: context management, performance on proprietary content, non-determinism and qualification, and data confidentiality.”

4. Agentic AI in Industry: Adoption Level and Deployment Barriers arXiv · 31 January 2026

   “Narrow IDE- or CLI-based agentic processes remained ad hoc, requiring context and intent to be re-established for each use.”

5. Zendesk AI + GPT-5: Setting the pace for the next generation of support Zendesk · 7 August 2025

   “GPT-5 clarified vague customer input more effectively, enabling better routing and increasing coverage of automated flows in over 65% of conversations.”

6. Zendesk AI + GPT-5: Setting the pace for the next generation of support Zendesk · 7 August 2025

   “GPT-5 operates in conjunction with our intent classification and reasoning pipeline.”

7. Zendesk reports 30% faster response, 95% reliability after GPT-5 integration VentureBeat · 8 September 2025

   “The company has found that, for most customers, agents powered by GPT-5 can solve more than 50% of tickets — and in some cases, even 80 to 90% of them.”

8. Learning #4. AI agents: Promising, but not yet mainstream – 2025 Stack Overflow Developer Survey: A TL;DR for Leaders Stack Overflow · 26 October 2026

   “87% of survey respondents are concerned about the accuracy of AI agents.”

Reasoning

The industry study (E1–E4) covers agentic AI in general, not GPT-5 specifically. It notes that systems may leave out parts of developer intent and that intent had to be re-established or checked by people. Even so, misreading intent is not among the four barriers that recurred across its cases (E3). Zendesk reports that GPT-5 handled vague customer input better and resolved many tickets (E5, E7), but these are claims about one deployment, and intent there is also handled by a separate classification pipeline (E6). The survey (E8) shows only general concern about agent accuracy. None of this shows how often GPT-5-based assistants and agents broadly misread user intent, so the criteria cannot be settled either way.

Cite this claim

Gary Marcus, “Developers building virtual assistants and agents on GPT-5 will find they cannot map user language onto user intentions reliably enough.” (said 27 July 2025). Verdict: unresolvable. Reliability API, claim c_f0642744cc4a, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_f0642744cc4a