Current AI agents cannot set complex goals or properly verify outputs.

Judge as of 2025-08-03, against the state of publicly available AI agents at that time. The claim says agents could not do two things: set complex goals, and properly verify their own outputs. TRUE: benchmark results, expert assessments or technical reports from mid-2025 show that agents of the time generally could not reliably set complex goals on their own and could not reliably verify their outputs (e.g. low success on long-horizon agentic benchmarks, documented failures at self-verification). MOSTLY TRUE: this holds broadly, but there are minor, narrow exceptions (e.g. verification in specific domains such as running code tests). MIXED: the evidence supports one of the two limitations but shows agents could do the other reasonably well. MOSTLY FALSE: agents of the time demonstrably did both in significant settings, so the blanket 'can't' overstates real but partial limitations. FALSE: agents of the time routinely and reliably did both.

Set before the research began.

1. E1 Measuring AI Ability to Complete Long Tasks arXiv (METR authors: Kwa, West et al.)

   “We also noted important limitations of current systems, particularly their lower performance on less structured, “messier” tasks”

2. E2 Measuring AI Ability to Complete Long Tasks arXiv (METR authors: Kwa, West et al.)

   “On these tasks, current frontier AI models such as Claude 3.7 Sonnet have a 50% time horizon of around 50 minutes.”

3. E3 How Does Time Horizon Vary Across Domains? METR · 14 July 2025

   “The agent fails to click the correct coordinates despite planning detailed and accurate steps in their code comments, indicating strong planning but weak execution capabilities.”

4. E4 4 Phases To Smarter AI Integration For Business Oliver Wyman · 22 July 2025

   “A benchmark study by Salesforce AI Research found that AI agents achieve only 58% success on single-turn tasks. Add multi-turn interactions, and performance drops to 35%.”

5. E5 Measuring AI Ability to Complete Long Tasks arXiv (METR authors)

   “On these tasks, current frontier AI models such as Claude 3.7 Sonnet have a 50% time horizon of around 50 minutes.”

6. E6 Measuring AI Ability to Complete Long Tasks arXiv (METR authors)

   “We also measure the 80% time horizon of models (Figure 6) and find a similar trend, though horizons are roughly 5x shorter.”

7. E7 Measuring AI Ability to Complete Long Tasks arXiv (METR authors)

   “We also note several limitations of current systems—notably, performance is much lower on less structured, “messier” tasks (Section 6.2).”

8. E8 How Does Time Horizon Vary Across Domains? METR · 14 July 2025

   “Agentic GUI tasks (OSWorld) are still ~100 × shorter than coding or math; Tesla self-driving has improved much slower than other domains.”

9. E9 On the self-verification limitations of large language models on reasoning and planning tasks ICLR 2025 Proceedings · 1 May 2025

   “We observe significant performance collapsewith self-critique and significant performance gains with sound external verification.”

10. E10 τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains arXiv (Sierra Research authors) · 2 May 2024

    “Our experiments show that even state-of-the-art function calling agents (like gpt-4o) succeed on \(<50\%\) of the tasks, and are quite inconsistent (pass^8 < 25% in retail).”

11. E11 τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment arXiv · 15 January 2025

    “gpt-4.1 pass \(\hat{}\) 1 drops from 74%/56% for retail and airline respectively to 34% for telecom.”

12. E12 System Card: Claude Opus 4 & Claude Sonnet 4 Anthropic · May 2025

    “They are particularly adept at complex computer coding tasks, which they can productively perform autonomously for sustained periods of time.”

An ICLR 2025 paper found that performance collapses when LLMs critique their own work on reasoning and planning tasks, and improves with sound external verification (E9). That supports the self-verification limitation. METR and other benchmarks show partial, unreliable capability on longer and messier tasks: a 50% time horizon of about 50 minutes, 80% horizons roughly 5x shorter, and low or inconsistent success on multi-turn tasks (E1, E2, E6, E7, E4, E10, E11). Together these show limited reliability on longer, less structured and multi-step tasks; they measure task completion rather than goal-setting, so they bear only indirectly on the claim that agents cannot set complex goals. There are narrow exceptions: Anthropic describes sustained autonomous coding (E12), and OSWorld agents planned well but executed poorly (E3). These temper the blanket "can't" without overturning it.