ChatGPT might sound like the Star Trek computer, but for now it still cannot be trusted.

Judged as of 25 December 2022. The checkable part is that ChatGPT could not be trusted to give reliable, accurate output: true if contemporaneous evidence (documented errors, fabricated facts or citations, OpenAI's own stated limitations that it 'sometimes writes plausible-sounding but incorrect or nonsensical answers') shows ChatGPT often produced confident falsehoods at that date, and that it produced fluent conversational answers that people likened to a general-purpose assistant such as the Star Trek computer. Mostly true if the unreliability is well documented but the comparison to the Star Trek computer is an overstatement of its fluency. Mixed if the evidence shows errors were rare or limited to narrow areas, so 'can't trust it' holds only in part. Mostly false if contemporaneous evaluations found it generally reliable, with only isolated errors. False if evidence shows ChatGPT's factual output at that time was reliably accurate.

Set before the research began.

1. E1 Introducing ChatGPT OpenAI · 30 November 2022

   “ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth;”

2. E2 Introducing ChatGPT OpenAI · 30 November 2022

   “including substantial reductions in harmful and untruthful outputs achieved by the use of reinforcement learning from human feedback (RLHF).”

3. E3 ChatGPT: the bot that can engage in intelligent conversation | World Economic Forum World Economic Forum · 13 December 2022

   “an AI tool that cannot only hold a human-like conversation with you but will admit its mistakes, correct your false assumptions”

4. E4 ChatGPT: the bot that can engage in intelligent conversation | World Economic Forum World Economic Forum · 13 December 2022

   “Not that the trials have been problem free. Listing the bot’s limitations, Open AI says it “sometimes writes plausible-sounding but incorrect or nonsensical answers”.”

In its launch post, OpenAI lists as a limitation that ChatGPT "sometimes writes plausible-sounding but incorrect or nonsensical answers" and says fixing this is challenging (E1). The World Economic Forum repeated this limitation in December 2022 (E4). This is the kind of contemporaneous evidence of unreliability that the criteria name. The World Economic Forum also describes ChatGPT as able to "hold a human-like conversation" (E3), which supports its fluency, but no evidence compares it to the Star Trek computer or another general-purpose assistant, and the sources say "sometimes" rather than "often." So the unreliability is well documented, while the comparison is only partly supported.