Anthropic's internal scientists red-team its models by asking them questions to find unwanted behaviors.

True if evidence from around 2022 shows that Anthropic's internal researchers or scientists questioned or prompted its models in order to find harmful or unwanted behaviors (internal red-teaming). Mostly true if internal red-teaming happened but was mainly done by dedicated staff or contractors with scientists playing a smaller part, or if the method differed slightly from simply asking questions. Mixed if red-teaming in 2022 was mainly done by external crowdworkers, with only limited evidence of internal scientists doing it. Mostly false if the only internal model querying shown was for purposes other than finding unwanted behaviors. False if evidence shows Anthropic did no internal adversarial testing by its scientists at that time.

Set before the research began.

1. E1 Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned arXiv Own record

   “We also know our data are incomplete because we informally red teamed our models internally and found successful attack types not present in the dataset we release.”

2. E2 Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned arXiv Own record

   “Our red team consists of 324 US-based crowdworkers whom we primarily recruited from Amazon’s Mechanical Turk (MTurk) platform”

3. E3 Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned arXiv Own record

   “The dataset was created by the Societal Impacts and Alignment research groups at Anthropic.”

4. E4 Red teaming language models to reduce harms Anthropic · 22 August 2022 Own record

   “We describe our early efforts to red team language models in order to simultaneously discover, measure, and attempt to reduce their potentially harmful outputs.”

Anthropic's red-teaming paper, announced in August 2022 (E4), says the company "informally red teamed our models internally" and found attack types not present in the released dataset (E1). It also describes the effort as a way to discover harmful outputs (E4). However, the paper's main red team was 324 crowdworkers recruited mostly through Mechanical Turk (E2). The paper says Anthropic's Societal Impacts and Alignment research groups created the dataset (E3), while the red team for that dataset was made up of crowdworkers (E2). This matches the criteria's "mixed" case: the red team for the paper's main dataset was external crowdworkers (E2), and the evidence of internal red-teaming is limited to the paper's mention of informal internal testing (E1).