Training failures where AIs are not trained to do what developers intended will get much worse as AIs become agents.
Unresolvable The public record could not settle it.
Prediction: Statements about events outside the person's control, checked once they fell due: did they come true?
Reasoning
Open-ended prediction with no horizon and vague 'much worse' measure.
Cite this claim
Scott Alexander, “Training failures where AIs are not trained to do what developers intended will get much worse as AIs become agents.” (said 3 April 2025). Verdict: unresolvable. Reliability API, claim c_e3d02585efaf, method 0.1.0+8bea457b, dataset release v1.1.2. https://epistacklabs.com/claims/c_e3d02585efaf
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More prediction claims by Scott Alexander
-
Mostly true
“Our scenario focuses on coding in particular because we think coding is what starts the intelligence explosion.”
The authors of the scenario think coding is what will start the intelligence explosion.
-
Mostly true
“We expect that as the AI labs become more capable, they tell the government about this because they want government contracts, they want government support.”
-
Mostly true
“In our scenario, that starts with cyber warfare, the government sees that these AIs are now as capable as the best human hackers, but can be deployed at humongous scale.”
-
Resolved true
“So they become extremely interested and they discuss nationalizing the AI companies.”
-
Mostly true
“I expect scaling to continue to produce gains, but this is less obvious than it’s been the past five years.”
Scaling AI models will continue to produce gains, though less obviously than in the past five years.
Verdicts are made automatically; how verdicts are made.