As of late 2024, pre-training accounts for the majority of the cost of building AI models.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
Reasoning
Cost breakdowns are not published, and the claim covers all AI models, so no public record settles it.
Cite this claim
Dario Amodei, “As of late 2024, pre-training accounts for the majority of the cost of building AI models.” (said 11 November 2024). Verdict: unresolvable. Reliability API, claim c_5d1367599ca6, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_5d1367599ca6
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Dario Amodei
-
Mostly true
“the amount of computation that goes into the models doubles every four months.”
The amount of computation going into AI models doubles every four months.
-
Mostly true
“Distillation of frontier models allows lagging companies to narrow the gap using a fraction of the cost it would take to develop their own AI independently.”
-
Mostly false
“I thought it was a very plausible blocker one or two years ago.”
-
Mostly true
“Importantly, because this type of RL is new, we are still very early on the scaling curve”
-
Resolved true
“I'd guess the number today is maybe ~4x/year.”
The current rate of algorithmic curve shift is maybe about 4x per year.
Verdicts are made automatically; how verdicts are made.