The model sometimes gives a very short response because, when it knows it gains nothing from thinking further, it does not think for longer; a bound on how long it thinks is still described as valuable.

Judge as of 2025-02-28 for Anthropic's extended-thinking model (Claude 3.7 Sonnet). TRUE: Anthropic documentation, the system card or independent testing shows the model adapts how long it thinks, often using far less than the allowed thinking budget and giving short answers on easy tasks, and the thinking budget is presented as a useful upper bound or control. MOSTLY TRUE: the adaptive behaviour is documented, but the explanation (that the model 'knows' it gains nothing) is only loosely supported, or the budget's value is framed differently. MIXED: thinking length adapts only weakly or inconsistently. MOSTLY FALSE: the model generally uses close to its full budget regardless of difficulty, with only rare exceptions. FALSE: the evidence shows thinking length does not vary with task difficulty.

Set before the research began.

1. E1 Claude 3.7 Sonnet System Card Anthropic Own record

   “it allows Claude to spend more time on questions which require extensive reasoning to produce better outputs. Users can specify how many tokens Claude 3.7 Sonnet can spend on extended thinking.”

2. E2 Claude 3.7 Sonnet and Claude Code Anthropic · 24 February 2025 Own record

   “you can tell Claude to think for no more than N tokens, for any value of N up to its output limit of 128K tokens. This allows you to trade off speed (and cost) for quality of answer.”

3. E3 Extended thinking Amazon Web Services (AWS) Bedrock User Guide

   “Larger budgets can improve response quality by enabling more thorough analysis for complex problems, although Claude might not use the entire budget allocated, especially at ranges above 32K.”

4. E4 Extended thinking Amazon Web Services (AWS) Bedrock User Guide

   “The thinking budget is a target rather than a strict limit - actual token usage may vary based on the task.”

AWS Bedrock documentation says Claude might not use the entire thinking budget it is given, and that actual token usage varies with the task (E3, E4). This supports the claim that thinking length adapts. Anthropic's system card describes extended thinking as letting the model spend more time on questions that need extensive reasoning (E1). They present the budget as a user-set "no more than N tokens" control that trades speed and cost against answer quality (E2), which matches the claim that a bound is valuable. However, none of the evidence specifically shows short answers on easy tasks. It also does not support the explanation that the model "knows" it gains nothing from thinking further, and AWS calls the budget a target rather than a strict limit (E4). So the claim is mostly true rather than fully true.