On 2024-12-17 OpenAI released o1 in the API with structured outputs and function calling, plus realtime API updates and preference fine-tuning.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
The claim has several parts: o1 released in the API on 2024-12-17 with structured outputs and function calling; Realtime API updates; preference fine-tuning. True: OpenAI's announcements or changelog dated 2024-12-17 show all of these released or announced that day. Mostly true: all were released that day, but with a minor detail off (e.g., access was limited to certain usage tiers, or a feature was in beta). Mixed: about half of the named components shipped that day. Mostly false: only a minor part shipped that day, or the main item (o1 in the API with those features) came on a different date. False: none of these were released on or around that date.
Set before the research began.
Evidence (4)
-
E1 OpenAI o1 and new tools for developers OpenAI
“OpenAI o1, our reasoning model designed to handle complex multi-step tasks with advanced accuracy, is rolling out to developers on usage tier 5(opens in a new window) in the API.”
-
E2 OpenAI o1 and new tools for developers OpenAI
“Preference Fine-Tuning will roll out today for gpt-4o-2024-08-06 and will be available for gpt-4o-mini-2024-07-18 soon.”
-
E3 Realtime API updates — WebRTC, cheaper prices, 4o-mini, and more OpenAI Developer Community · 17 December 2024
“A bunch of big updates for the Realtime API today. We’re announcing support for WebRTC, meaning you can add speech-to-speech experiences with just a handful of lines of code.”
-
E4 Changelog | OpenAI API OpenAI
“Launched Preference Fine-tuning using Direct Preference Optimization (DPO).”
Reasoning
Only the Realtime API updates are dated 2024-12-17 [E3]. The OpenAI announcement [E1, E2] and the changelog [E4] are undated, and none of the evidence shows that o1 in the API came with structured outputs or function calling, so the date and features of the main item cannot be checked against the criteria.
Cite this claim
Sam Altman, “On 2024-12-17 OpenAI released o1 in the API with structured outputs and function calling, plus realtime API updates and preference fine-tuning.” (said 17 December 2024). Verdict: unresolvable. Reliability API, claim c_9e865d06131a, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_9e865d06131a
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Sam Altman
-
Mostly true
“not until 2020 did we train GPT-3, which was the first model that I think actually was over some threshold of usability.”
OpenAI trained GPT-3 in 2020, which Altman considers the first model above a threshold of usability.
-
Resolved true
“that it is so capable that it is discovering new knowledge, doing science, able to add huge economic value, create entire pieces of complex software.”
-
Mixed
“They now hallucinate a little bit.”
-
Mixed
“They don’t do anything unless you ask, right?”
AI models such as ChatGPT do not do anything unless a user asks them to.
-
Mostly true
“But in terms of figuring out what questions to ask or to work on something over a very long period of time, we are definitely not close to human performance.”
Verdicts are made automatically; how verdicts are made.