Claims

Prediction · AI compute, hardware and data

“Training GPT-4 will cost $100M, which is a lot.”

Scott Alexander, blogger · 20 February 2023 · Grading My 2018 Predictions For 2023 (Astral Codex Ten)

Training GPT-4 will cost $100 million.

Resolved true The record shows it happened or was so.

The second judge would not resolve this claim; the verdict shown is the first judge's.

What would count as true or false

Treat '$100M' as a round, approximate figure for the compute and training cost of GPT-4. TRUE: OpenAI statements (such as Sam Altman's remark that it cost 'more than $100 million'), or credible analyst estimates, put the training cost roughly between $70M and $150M. MOSTLY TRUE: the best estimates fall roughly between $50M and $70M, or between $150M and $200M. MIXED: credible estimates disagree widely, with some near $100M and others far away. MOSTLY FALSE: the best estimates are off by a factor of 2 to 3 (about $33M–50M, or $200M–300M). FALSE: the cost is credibly shown to be more than 3 times higher or lower than $100M.

Evidence

1. Altman says the age of giant models is 'already over' and GPT-4 cost over $100M — AGI Watch AGI Watch · 17 April 2023

   “Speaking at MIT, Altman said returns from simply scaling parameters were diminishing and that progress would come from other directions, while confirming GPT-4's training cost exceeded $100 million.”

2. The rising costs of training frontier AI models arXiv (Epoch AI and Stanford University authors) · 15 January 2024

   “For example, we estimate that it cost $800M to acquire the hardware used to train GPT-4, compared to $40M for the amortized hardware CapEx + energy cost.”

3. The rising costs of training frontier AI models arXiv (Epoch AI and Stanford University authors) · 15 January 2024

   “while the amortized hardware cost over the whole model development increased from $4M to $90M.”

4. Report on Frontier Model Training — LessWrong LessWrong (post by YafahEdelman) · 30 August 2023

   “New state of the art (frontier) models will likely cost on the order of a billion dollars, with the most recent frontier model, GPT-4, having cost approximately half a billion dollars.”

Reasoning

E1 reports Sam Altman confirming that GPT-4's training cost was more than $100 million, and the resolution criteria name this statement as an example of a TRUE outcome. The Epoch AI study gives a figure of $90M for amortized hardware cost over a whole model's development (E3), though the quoted passage does not name the model. It also gives $40M for GPT-4's amortized hardware CapEx and energy cost, against $800M to acquire the hardware (E2). One estimate is much higher: about half a billion dollars, in an August 2023 LessWrong post (E4). Altman's reported statement that the cost exceeded $100 million (E1) matches the example the criteria give for a TRUE outcome, though other estimates range from $40M (E2) to about half a billion dollars (E4).

Cite this claim

Scott Alexander, “Training GPT-4 will cost $100 million.” (said 20 February 2023). Verdict: resolved true. Reliability API, claim c_dac7a6220f35, method 0.1.0+8bea457b. https://epistacklabs.com/claims/c_dac7a6220f35