Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
83 77 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
69 49 87 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
69 49 87
All types combined: 81 (likely 76 to 86, high confidence), from 87 claims: 77 factual claims, 5 predictions, 5 promises. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Sam Altman's claims
40 claims
“today we are announcing reinforcement finetuning, which makes it really easy to create expert models in specific domains with very little training data.”
Sam Altman, OpenAI CEO, on X 6 December 2024 Post by @sama on X, 2024-12-06 (X)
OpenAI announced reinforcement finetuning on 2024-12-06, which makes it easy to create expert models in specific domains with little training data.
Mostly true True in substance, wrong in a detail.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judged as of 2024-12-06. True if OpenAI officially announced reinforcement fine-tuning (RFT) on 2024-12-06 (e.g. day 2 of '12 Days of OpenAI') and presented it as a way to customise models into domain-specific experts using small datasets (e.g. dozens to thousands of examples). Mostly true if it was announced within a day of that date, or a detail of the description differs slightly (e.g. at launch it was available only as a limited research preview). Mostly false if it was announced only as a much vaguer plan, or on a materially different date. False if OpenAI made no such announcement.
Evidence
- OpenAI’s 12 days of ‘ship-mas’: all the new announcements The Verge · 5 December 2024
“In a post after the livestream announcement, CEO Sam Altman said this will make it “really easy to create expert models in specific domains with very little training data.””
- 12 days of OpenAI: The Ars Technica recap Ars Technica · 12 December 2024
“OpenAI plans to release RFT to the public in early 2025 but currently offers limited access through its Reinforcement Fine-Tuning Research Program for researchers, universities, and companies.”
- That’s a wrap on day 2 TechCrunch · 6 December 2024
“Today’s stream featured demos for “Reinforcement Fine-Tuning,” which OpenAI describes as the ability to fine-tune its o1 model on users’ own data sets.”
- 12 Days of OpenAI OpenAI Own record
“This fine-tuning technique is designed to improve the performance of our reasoning models on verifiable domain-specific tasks.”
Reasoning
TechCrunch's 2024-12-06 day-2 wrap-up says that day's livestream showed demos of "Reinforcement Fine-Tuning", which OpenAI described as fine-tuning o1 on users' own data sets (E3). The Verge reports that after the livestream, Altman described it as making it easy to create expert models in specific domains with very little training data (E1). OpenAI's own page frames it as improving performance on domain-specific tasks (E4). Ars Technica reports that at launch, access was limited to a research program, with a public release planned for early 2025 (E2). The resolution criteria list a limited research preview as a slight difference that makes the claim mostly true rather than true.
Cite this claim
Sam Altman, “OpenAI announced reinforcement finetuning on 2024-12-06, which makes it easy to create expert models in specific domains with little training data.” (said 6 December 2024). Verdict: mostly true. Reliability API, claim c_0390dc446904, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_0390dc446904
Claims without a verdict
Is a verdict here wrong, or do you speak for Sam Altman? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.