Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
83 77 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
69 49 87 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
69 49 87
All types combined: 81 (likely 76 to 86, high confidence), from 87 claims: 77 factual claims, 5 predictions, 5 promises. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Sam Altman's claims
40 claims
“How we do it, we show the model examples of how we’d like it to respond in different cases, and from that it learns the sort of the overall personality.”
Sam Altman, OpenAI CEO, interviewee 9 August 2025 Transcript: Sam Altman Interview on Huge Conversations Podcast (Singju Post)
OpenAI shapes ChatGPT's personality by showing the model examples of how it should respond in different cases, from which it learns its overall personality.
Mostly true True in substance, wrong in a detail.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
True: OpenAI documentation or credible technical accounts as of 2025-08-09 show ChatGPT's personality is shaped mainly by training on examples of desired responses (e.g. supervised fine-tuning demonstrations, or RLHF built on example comparisons), from which the model generalises its overall behaviour. Mostly true: example-based training is a main method, but other methods described in the sources (e.g. system prompts, the Model Spec, reward models) also play a significant part that the claim simplifies. Mixed: example-based training is one of several roughly equal methods. Mostly false: personality is mainly set by other means (e.g. system prompts or rules) and examples play only a minor part. False: evidence shows OpenAI does not use example-based training to shape personality.
Evidence
- Expanding on what we missed with sycophancy OpenAI · 2 May 2025
“To post-train models, we take a pre-trained base model, do supervised fine-tuning on a broad set of ideal responses written by humans or existing models, and then run reinforcement learning with reward signals from a variety of sources.”
- Expanding on what we missed with sycophancy OpenAI · 2 May 2025
“The set of reward signals, and their relative weighting, shapes the behavior we get at the end of training.”
- Sycophancy in GPT-4o: What happened and what we’re doing about it OpenAI · 29 April 2025
“Refining core training techniques and system prompts to explicitly steer the model away from sycophancy.”
- Training language models to follow instructions with human feedback arXiv (OpenAI authors, InstructGPT paper)
“Starting with a set of labeler-written prompts and prompts submitted through the OpenAI API, we collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning.”
Reasoning
OpenAI's May 2025 post says its post-training includes supervised fine-tuning on a broad set of ideal responses (E1), and the InstructGPT paper describes fine-tuning on labeler demonstrations of the desired behaviour (E4). Both support the claim that the model learns from examples of how it should respond. OpenAI's May 2025 post also says that reward signals from reinforcement learning, and how they are weighted, shape the model's behaviour at the end of training (E2), and an April 2025 OpenAI post says system prompts are being refined to steer the model away from sycophancy (E3). Example-based training is therefore a main method, but the claim leaves out other methods that play a significant part.
Cite this claim
Sam Altman, “OpenAI shapes ChatGPT's personality by showing the model examples of how it should respond in different cases, from which it learns its overall personality.” (said 9 August 2025). Verdict: mostly true. Reliability API, claim c_93bc8e2d9a3f, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_93bc8e2d9a3f
Claims without a verdict
Is a verdict here wrong, or do you speak for Sam Altman? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.