ClaimsDario Amodei

“We kind of already do RL with RLHF.”

Dario Amodei, Anthropic CEO, interviewed 8 August 2023 Dario Amodei (Anthropic CEO) - $10 Billion Models, OpenAI, Scaling, & AGI in 2 years (Dwarkesh Podcast)

Factual AI capabilities

Amodei said Anthropic "kind of" already does reinforcement learning, through RLHF.

Resolved true The record shows it happened or was so.

Factual: Statements about the present or the past, checked against the record: did they hold up?

The evidence is the person's or their organisation's own records.

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

True: Anthropic's publications or official statements as of August 2023 show that it trained its language models with reinforcement learning from human feedback (RLHF) or its RL-based variants (e.g. RLAIF/Constitutional AI), which involve reinforcement learning. Mostly true: RL was used, but by a variant described slightly differently. Mostly false: RLHF-style methods were used only in research and not on its models. False: Anthropic did not use RL-based fine-tuning on its language models.

Set before the research began.

Evidence (4)

  1. E1 Model Card and Evaluations for Claude Models - Anthropic Anthropic Own record

    “They use a transformer architecture and are trained via unsupervised learning, RLHF, and Constitutional AI (including both”
  2. E2 Claude's constitution Anthropic · 9 May 2023 Own record

    “During the second phase, a model is trained via reinforcement learning, but rather than using human feedback, it uses AI-generated feedback based on the set of principles to choose the more harmless output.”
  3. E3 Training a helpful and harmless assistant with RLHF Anthropic · 12 April 2022 Own record

    “We apply preference modeling and reinforcement learning from human feedback (RLHF) to finetune language models to act as helpful and harmless assistants.”
  4. E4 Constitutional AI: Harmlessness from AI feedback Anthropic · 15 December 2022 Own record

    “In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences. We then train with RL using the preference model as the reward signal, i.e. we use 'RL from AI Feedback' (RLAIF).”

Reasoning

E2 is Anthropic's 2023-05-09 post on Claude's constitution, which predates the claim. It says that in the second training phase a model is trained with reinforcement learning using AI-generated feedback. E3 (April 2022) and E4 (December 2022) are Anthropic's own research publications describing RLHF and RL from AI feedback applied to its language models. E1, Anthropic's model card for its Claude models, says they are trained via RLHF and Constitutional AI, but it is undated. All of these are Anthropic's own official publications rather than independent sources, and they consistently show RL-based fine-tuning of its models before August 2023.

Cite this claim

Dario Amodei, “Amodei said Anthropic "kind of" already does reinforcement learning, through RLHF.” (said 8 August 2023). Verdict: resolved true. Reliability API, claim c_9def3a5c157d, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_9def3a5c157d

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Dario Amodei, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Dario Amodei

  1. Mixed Factual 23 September 2026
    “It’s easy to dismiss this as a one-off or curiosity, but we’ve repeatedly seen a pattern where AI performance in new intellectual domains goes from weak to superhuman in a matter of a few years.”

    AI performance in new intellectual domains has repeatedly gone from weak to superhuman within a few years.

  2. Mixed Factual 10 March 2025
    “we were measuring these trends back when models cost $1,000 or $10,000 to train”

    Amodei's group measured AI scaling trends when models cost $1,000 or $10,000 to train.

  3. Mixed Factual 28 February 2025
    “a lot of the other reasoning models in the market are trained primarily on math and competition coding,”

    Many other companies' reasoning models on the market are trained primarily on math and competition coding.

  4. Mostly true Factual 28 February 2025
    “I do think AI coding is moving the fastest of all the other areas.”

    Amodei said he thinks AI coding is moving the fastest of all the areas of AI.

  5. Mostly true Factual 28 February 2025
    “Because when it knows that it doesn't get any gain out of thinking further, it doesn't think for longer.”

    The model sometimes gives a very short response because, when it knows it gains nothing from thinking further, it does not think for longer; a bound on how long it thinks is still described as valuable.

All of them

Verdicts are made automatically; how verdicts are made.