Developers building virtual assistants and agents on GPT-5 will find they cannot map user language onto user intentions reliably enough.
Unresolvable The public record could not settle it.
Prediction: Statements about events outside the person's control, checked once they fell due: did they come true?
What would count as true or false
Judge over about a year after GPT-5's August 2025 release (to August 2026). True: credible evidence (developer reports, evaluations of GPT-5-based agents, benchmarks of agentic task success, industry surveys) shows that assistants and agents built on GPT-5 commonly misread user intent often enough that developers report it as a significant barrier to reliable deployment. Mostly true: intent-mapping failures are widely documented as a significant problem, but some substantial deployments report acceptable reliability. Mixed: the evidence is roughly split between reports of reliable intent understanding and reports of significant failures. Mostly false: most developers and evaluations find GPT-5 maps user language to intent reliably enough for production, with failures only in edge cases. False: GPT-5-based assistants and agents are broadly found to interpret user intentions reliably, and misinterpretation is not a notable barrier. Failures caused mainly by other factors (tool errors, cost, latency) do not count towards the claim.
Set before the research began.
Evidence (8)
-
E1 Agentic AI in Industry: Adoption Level and Deployment Barriers arXiv · 31 January 2026
“Greater task ownership also amplifies friction already visible in LLM-based tools: systems may omit aspects of developer intent, vary across equivalent prompts, hallucinate, or lose relevant information as context grows [13, 7].”
-
E2 Agentic AI in Industry: Adoption Level and Deployment Barriers arXiv · 31 January 2026
“Human verification was therefore used to check both technical output and semantic intent.”
-
E3 Agentic AI in Industry: Adoption Level and Deployment Barriers arXiv · 31 January 2026
“Across the cases, four previously identified barriers recurred: context management, performance on proprietary content, non-determinism and qualification, and data confidentiality.”
-
E4 Agentic AI in Industry: Adoption Level and Deployment Barriers arXiv · 31 January 2026
“Narrow IDE- or CLI-based agentic processes remained ad hoc, requiring context and intent to be re-established for each use.”
-
E5 Zendesk AI + GPT-5: Setting the pace for the next generation of support Zendesk · 7 August 2025
“GPT-5 clarified vague customer input more effectively, enabling better routing and increasing coverage of automated flows in over 65% of conversations.”
-
E6 Zendesk AI + GPT-5: Setting the pace for the next generation of support Zendesk · 7 August 2025
“GPT-5 operates in conjunction with our intent classification and reasoning pipeline.”
-
E7 Zendesk reports 30% faster response, 95% reliability after GPT-5 integration VentureBeat · 8 September 2025
“The company has found that, for most customers, agents powered by GPT-5 can solve more than 50% of tickets — and in some cases, even 80 to 90% of them.”
-
E8 Learning #4. AI agents: Promising, but not yet mainstream – 2025 Stack Overflow Developer Survey: A TL;DR for Leaders Stack Overflow · 26 October 2026
“87% of survey respondents are concerned about the accuracy of AI agents.”
Reasoning
The industry study (E1–E4) covers agentic AI in general, not GPT-5 specifically. It notes that systems may leave out parts of developer intent and that intent had to be re-established or checked by people. Even so, misreading intent is not among the four barriers that recurred across its cases (E3). Zendesk reports that GPT-5 handled vague customer input better and resolved many tickets (E5, E7), but these are claims about one deployment, and intent there is also handled by a separate classification pipeline (E6). The survey (E8) shows only general concern about agent accuracy. None of this shows how often GPT-5-based assistants and agents broadly misread user intent, so the criteria cannot be settled either way.
Cite this claim
Gary Marcus, “Developers building virtual assistants and agents on GPT-5 will find they cannot map user language onto user intentions reliably enough.” (said 27 July 2025). Verdict: unresolvable. Reliability API, claim c_f0642744cc4a, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_f0642744cc4a
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More prediction claims by Gary Marcus
-
Mostly true
“Some of its advice will be head-scratchingly bad.”
-
Mostly true
“GPT-5 will surely be better, a lot better than GPT-4.”
-
Resolved true
“GPT-5 really is about to drop, no foolin’.”
-
Resolved true
“It will still make a significant number of shake-your-head stupid errors, in ways that are hard to fully predict.”
GPT-5 will still make a significant number of obvious errors that are hard to predict.
-
Mostly true
“Neurosymbolic AI will become much more prominent.”
Verdicts are made automatically; how verdicts are made.