Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
83 77 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
69 49 87 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
69 49 87
All types combined: 81 (likely 76 to 86, high confidence), from 87 claims: 77 factual claims, 5 predictions, 5 promises. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Sam Altman's claims
41 claims
“The system combines reinforcement learning from human feedback and recursive task decomposition.”
Sam Altman, OpenAI CEO, on X 23 September 2021 Post by @sama on X, 2021-09-23 (X)
OpenAI's book-summarizing system combines reinforcement learning from human feedback and recursive task decomposition.
Resolved true The record shows it happened or was so.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judged as of 2021-09-23. True: OpenAI's book-summarization publication (the blog post 'Summarizing Books with Human Feedback' and the paper 'Recursively Summarizing Books with Human Feedback', Wu et al. 2021) describes the system as using both reinforcement learning from human feedback and recursive task decomposition. Mostly true: both methods are used, but one is described in slightly different terms (for example behavioral cloning plus RL from human feedback). Mixed: only one of the two methods is used. Mostly false: neither method is central, though related techniques appear. False: the system uses neither method, or no such OpenAI system exists.
Evidence
- Summarizing books with human feedback - OpenAI OpenAI · 23 September 2021 Own record
“Our approach: combining reinforcement learning from human feedback and recursive task decomposition”
- Recursively Summarizing Books with Human Feedback arXiv (ar5iv rendering) Own record
“Our method combines learning from human feedback with recursive task decomposition: we use models trained on smaller parts of the task to assist humans in giving feedback on the broader task.”
- Recursively Summarizing Books with Human Feedback arXiv (ar5iv rendering) Own record
“We train a single model to perform these tasks using standard cross-entropy behavioral cloning (BC) and reinforcement learning (RL) from human preferences (Christiano et al., 2017).”
- OpenAI unveils model that can summarize books of any length VentureBeat · 23 September 2021
“To create the model, OpenAI combined reinforcement learning with recursive task decomposition, which procedurally breaks up a difficult task (e.g., summarizing a long piece of text) into simpler, individual ones”
Reasoning
The criteria name OpenAI's own publication as the reference. The blog post, dated 2021-09-23, describes the approach as "combining reinforcement learning from human feedback and recursive task decomposition" (E1). The paper says the method combines learning from human feedback with recursive task decomposition, and that training uses behavioral cloning plus RL from human preferences (E2, E3). VentureBeat separately reported the same day that OpenAI combined reinforcement learning with recursive task decomposition (E4).
Cite this claim
Sam Altman, “OpenAI's book-summarizing system combines reinforcement learning from human feedback and recursive task decomposition.” (said 23 September 2021). Verdict: resolved true. Reliability API, claim c_61e7f892f4d8, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_61e7f892f4d8
Claims without a verdict
Is a verdict here wrong, or do you speak for Sam Altman? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.