Claims

Factual · AI compute, hardware and data

“Distillation of frontier models allows lagging companies to narrow the gap using a fraction of the cost it would take to develop their own AI independently.”

Dario Amodei, author, essay · 5 June 2025 · We Must Pace the Frontier (darioamodei.com)

Distillation of frontier models lets lagging companies narrow the gap at a fraction of the cost of developing their own AI independently.

Mostly true True in substance, wrong in a detail.

The second judge would not resolve this claim; the verdict shown is the first judge's.

What would count as true or false

Judge as of 2025-06-05. True: research and credible industry evidence show that distilling a frontier model, i.e. training on its outputs, lets developers who are behind reach performance substantially closer to frontier models at a cost far below independent frontier development (e.g. documented distilled models matching much of a teacher's capability for a small share of the compute or money). Mostly true: this is generally supported, but the size of the gap closed or the cost savings is overstated, or holds only for some capabilities. Mixed: evidence is split on whether distillation meaningfully narrows the frontier gap, as opposed to copying narrow behaviours. Mostly false: distillation gives only narrow or superficial gains, so the general claim overstates a limited finding. False: evidence shows distillation does not narrow capability gaps, or does not reduce cost compared with independent development.

Evidence

1. DeepSeek-R1/README.md at 0cf78561f1d51c84a21b2190626b21116d5c68bb · deepseek-ai/DeepSeek-R1 GitHub (deepseek-ai) · 20 January 2025

   “We demonstrate that the reasoning patterns of larger models can be distilled into smaller models, resulting in better performance compared to the reasoning patterns discovered through RL on small models.”

2. DeepSeek-R1/README.md at 0cf78561f1d51c84a21b2190626b21116d5c68bb · deepseek-ai/DeepSeek-R1 GitHub (deepseek-ai) · 20 January 2025

   “DeepSeek-R1-Distill-Qwen-32B outperforms OpenAI-o1-mini across various benchmarks, achieving new state-of-the-art results for dense models.”

3. DeepSeek-R1/README.md at 0cf78561f1d51c84a21b2190626b21116d5c68bb · deepseek-ai/DeepSeek-R1 GitHub (deepseek-ai) · 20 January 2025

   “DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek-R1-Distill-Qwen-7B, DeepSeek-R1-Distill-Qwen-14B and DeepSeek-R1-Distill-Qwen-32B are derived from Qwen-2.5 series, which are originally licensed under Apache 2.0 License, and now finetuned with 800k samples curated with DeepSeek-R1.”

4. DeepSeek-R1/README.md at 0cf78561f1d51c84a21b2190626b21116d5c68bb · deepseek-ai/DeepSeek-R1 GitHub (deepseek-ai)

   “We demonstrate that the reasoning patterns of larger models can be distilled into smaller models, resulting in better performance compared to the reasoning patterns discovered through RL on small models.”

5. DeepSeek-R1/README.md at 0cf78561f1d51c84a21b2190626b21116d5c68bb · deepseek-ai/DeepSeek-R1 GitHub (deepseek-ai)

   “DeepSeek-R1-Distill-Qwen-32B outperforms OpenAI-o1-mini across various benchmarks, achieving new state-of-the-art results for dense models.”

6. DeepSeek-R1/README.md at 0cf78561f1d51c84a21b2190626b21116d5c68bb · deepseek-ai/DeepSeek-R1 GitHub (deepseek-ai)

   “DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek-R1-Distill-Qwen-7B, DeepSeek-R1-Distill-Qwen-14B and DeepSeek-R1-Distill-Qwen-32B are derived from Qwen-2.5 series, which are originally licensed under Apache 2.0 License, and now finetuned with 800k samples curated with DeepSeek-R1.”

7. ai-system-design-guide-ch/03-training-and-adaptation/08-rlvr-and-reasoning-models.md at main · wangdd2008/ai-system-design-guide-ch GitHub (repository wangdd2008/ai-system-design-guide-ch, a fork)

   “Off-policy distillation (SFT on a teacher's traces). DeepSeek-R1 reports fine-tuning dense Qwen and Llama models on roughly 800K reasoning traces generated by R1, and that the distilled 32B model beats running large-scale RL directly on that same 32B base, which the paper notes takes far more compute and still does not match distillation.”

8. ai-system-design-guide-ch/03-training-and-adaptation/08-rlvr-and-reasoning-models.md at main · wangdd2008/ai-system-design-guide-ch GitHub (repository wangdd2008/ai-system-design-guide-ch, a fork)

   “The lesson: for a small model on a known domain, distilling from a strong reasoner is more effective and far cheaper than doing your own RL.”

9. ai-system-design-guide-ch/03-training-and-adaptation/08-rlvr-and-reasoning-models.md at main · wangdd2008/ai-system-design-guide-ch GitHub (repository wangdd2008/ai-system-design-guide-ch, a fork)

   “The reported result is roughly an order of magnitude more sample-efficient than RL for instilling reasoning, combining on-policy relevance with a dense, stable signal. Treat the exact multiple as reported, not established.”

10. ai-system-design-guide-ch/03-training-and-adaptation/08-rlvr-and-reasoning-models.md at main · wangdd2008/ai-system-design-guide-ch GitHub (repository wangdd2008/ai-system-design-guide-ch, a fork)

    “The mechanism is contested. An August 2026 paper (arXiv:2608.31046, posted August 31 and revised September 24) reports that the teacher's per-token signal in on-policy distillation is substantially noisy, more so with larger teachers, and that student performance is unchanged whether that noise is kept or removed.”

Reasoning

DeepSeek's R1 README reports that smaller Qwen-based models fine-tuned on about 800k samples curated with DeepSeek-R1 outperform small models trained with RL alone, and that the distilled 32B model outperforms OpenAI o1-mini across various benchmarks (E1, E2, E3). A third-party summary of the R1 paper says distillation beat large-scale RL on the same 32B base, which took far more compute, and concludes distilling is more effective and far cheaper (E7, E8). This broadly supports distillation narrowing capability gaps cheaply. However, no source gives a cost or compute figure, the evidence is limited to smaller models on reasoning tasks, and none compares directly with independent frontier development.

Cite this claim

Dario Amodei, “Distillation of frontier models lets lagging companies narrow the gap at a fraction of the cost of developing their own AI independently.” (said 5 June 2025). Verdict: mostly true. Reliability API, claim c_2e07cb043afb, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_2e07cb043afb