Scaling AI models will continue to produce gains, though less obviously than in the past five years.
Mostly true True in substance, wrong in a detail.
Prediction: Statements about events outside the person's control, checked once they fell due: did they come true?
The second judge said resolved true; the verdict shown is the first judge's.
What would count as true or false
The claim has two parts. (a) Scaling continued to produce gains: from February 2023 onward, larger models or more training compute kept giving measurable capability improvements, shown in benchmark results, technical reports or published scaling-law studies of frontier models released after the claim. (b) The gains were less obvious than in 2018-2023: credible sources (researcher statements, analyses, press coverage of diminishing returns, debate over a pre-training 'wall') show that the returns from scaling became more contested or harder to see than in the GPT-2 to GPT-4 era. True: both (a) and (b) hold. Mostly true: (a) clearly holds and (b) holds only weakly or is mildly disputed. Mixed: one part holds and the other clearly fails. For example, scaling gains were as obvious and undisputed as before, or the gains came mainly from methods other than scale while pure scaling stalled. Mostly false: scaling gave only marginal gains that most expert sources call negligible. False: scaling stopped producing gains or reversed. Judge on evidence from February 2023 up to the time of evaluation.
Set before the research began.
Evidence (4)
-
E1 Trends in AI Epoch AI · 5 February 2026
“Training compute for frontier language models has been growing at 5× per year since 2020.”
-
E2 Trends in AI Epoch AI · 5 February 2026
“Following the introduction of the first reasoning models (OpenAI’s o1-mini and o1-preview) in September 2024, the Epoch Capabilities Index (ECI) frontier has advanced linearly by 14 points per year, compared with 6 points per year for non-reasoning models.”
-
E3 AI Scaling Wall Wikipedia · 30 September 2026
“One 2025 analysis estimated that each tenfold rise in compute yields only one to two percentage points of MMLU-Pro accuracy.”
-
E4 The $19.6 billion pivot: How OpenAI's 2-year struggle to launch GPT-5 revealed that its core AI strategy has stopped working | Fortune Fortune · 25 February 2025
“These media reports indicated that Orion was better at some language tasks than GPT-4, but that its abilities at coding and mathematics was not significantly improved on the previous model.”
Reasoning
Epoch AI reports that frontier training compute kept growing about 5× per year (E1). It also reports that frontier capability, measured by its Capabilities Index, kept rising, by about 6 points per year for non-reasoning models and faster for reasoning models (E2). A 2025 analysis estimated that each tenfold rise in compute still adds one to two points of MMLU-Pro accuracy (E3), so scaling kept producing gains, but small ones. Fortune reports that OpenAI's Orion model was better than GPT-4 at some language tasks but not significantly improved in coding and mathematics (E4). This supports the 'less obvious' part, while E3 shows that added compute still produced measurable, if small, gains, and E2 shows the capability frontier advancing faster after reasoning models were introduced than for non-reasoning models.
Cite this claim
Scott Alexander, “Scaling AI models will continue to produce gains, though less obviously than in the past five years.” (said 20 February 2023). Verdict: mostly true. Reliability API, claim c_6690cfadae41, method 0.1.0+8bea457b, dataset release v1.1.2. https://epistacklabs.com/claims/c_6690cfadae41
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More prediction claims by Scott Alexander
-
Mostly true
“Our scenario focuses on coding in particular because we think coding is what starts the intelligence explosion.”
The authors of the scenario think coding is what will start the intelligence explosion.
-
Mostly true
“We expect that as the AI labs become more capable, they tell the government about this because they want government contracts, they want government support.”
-
Mostly true
“In our scenario, that starts with cyber warfare, the government sees that these AIs are now as capable as the best human hackers, but can be deployed at humongous scale.”
-
Resolved true
“So they become extremely interested and they discuss nationalizing the AI companies.”
-
Resolved true
“Training GPT-4 will cost $100M, which is a lot.”
Verdicts are made automatically; how verdicts are made.