People put instructions in LLM system prompts but cannot get the systems to follow them.

Judge as of 2025. The claim means that LLMs commonly fail to follow instructions placed in their system prompts reliably; it does not mean they never comply. True: documented evidence from 2025 (benchmarks of instruction or system-prompt adherence, vendor documentation, research papers, or well-documented incidents) shows that LLMs often fail to follow system-prompt instructions. Mostly true: the failures are well documented, but adherence was high in many settings, so the claim somewhat overstates how often failures happen. Mixed: the evidence is split, with good adherence on simple instructions and frequent failure on complex or conflicting ones. Mostly false: failures exist but are rare edge cases, with adherence generally reliable. False: the evidence shows LLMs reliably follow system prompts.

Set before the research began.

1. E1 A Closer Look at System Prompt Robustness arXiv (academic preprint; authors not Gary Marcus) · 25 November 2023

   “Even in the absence of conflicting instructions, current models still frequently fail to adhere to the initial instructions within the system prompt.”

2. E2 A Closer Look at System Prompt Robustness arXiv (academic preprint; authors not Gary Marcus) · 25 November 2023

   “the performance of recent LLMs uniformly approaches zero as the number of guardrails increases.”

3. E3 Control Illusion: The Failure of Instruction Hierarchies in Large Language Models ar5iv (arXiv mirror), arXiv ID 2502.15851 · 8 May 2024

   “Most models show dramatically lower performance (9.6–45.8% average R1) when handling conflicting constraints, compared to their baseline instruction-following capabilities.”

4. E4 A Closer Look at System Prompt Robustness arXiv (academic preprint; authors not Gary Marcus) · 25 November 2023

   “we see that o3 mini is substantially more robust in following system prompts.”

An independent 2025 paper on system-prompt robustness reports that current models "frequently fail" to follow system-prompt instructions even when nothing conflicts with them (E1). The same paper finds performance approaching zero as the number of guardrails grows (E2). A second independent paper finds sharply lower compliance, 9.6–45.8% on average, when constraints conflict (E3). The first paper also says o3-mini is substantially more robust in following system prompts (E4). So the failures are well documented, though one model, o3-mini, was found substantially more robust than others (E4).