Citing Ernie Davis, Marcus said OpenAI had begun to incorporate user experiments into its models' training corpus.

Judged as of 2023-03-24. True: OpenAI documentation, policies or credible reporting show that by that date OpenAI was using user conversations or prompts from ChatGPT or its API (including users' test prompts) as training data for its models. Mostly true: OpenAI was using user data to train or fine-tune models (e.g. ChatGPT conversations under its data-usage policy), but there is no specific evidence that the 'experiments' testing generalisation were included. Mixed: OpenAI kept the right to train on user data and some use is documented, but how far it went before March 2023 is unclear. Mostly false: user data was used only for narrow purposes, such as safety review or feedback ratings, and not in the training corpus, with the claim overstating this. False: OpenAI's policies and practice show that user inputs were not used for training at that time.

Set before the research began.

1. E1 Addressing criticism, OpenAI will no longer use customer data to train its models by default | TechCrunch TechCrunch · 1 March 2023

   “Developers have long taken issue with OpenAI’s (now-deprecated) data processing policy, which they claim posed a privacy risk and allowed the company to profit off of their data.”

2. E2 ChatGPT is back in Italy – What changes have been made and what do users need to know? DSN Group · 5 May 2023

   “Until recently, all conversations taking place between private users and the bot were subject to being used for training the algorithm, which opened the possibility of user inputs being used by the machine for future public responses.”

3. E3 Italy lifts ban on ChatGPT after data privacy improvements Deutsche Welle (DW) · 29 April 2023

   “The country's data protection watchdog said its developers did not have a legal basis to justify the storage and collection of users' personal data in order to train the site's algorithms.”

4. E4 Sam Altman: OpenAI won't tap into customer APIs CNBC · 5 May 2023

   “We don't train on any API data at all, we haven't for a while,” Altman told CNBC.”

E2 reports that until spring 2023, all conversations between private users and ChatGPT could be used to train the model. E3 reports that Italy's regulator objected that OpenAI had no legal basis to collect users' personal data to train its algorithms. E1 notes that developers had criticised OpenAI's earlier data-processing policy, which they said let the company profit from their data. Together these indicate that, before spring 2023, OpenAI's policies allowed user conversations to be used for training, and that this drew criticism from developers [E1] and Italy's regulator [E3]. However, no item shows specifically that users' experiments testing generalisation went into the training corpus, and E4 has Altman saying API data had not been used for training "for a while".