Claims

Factual · AI capabilities

“The capabilities of LLMs trained on gargantuan datasets are more general than I (and almost everyone else) would have guessed at the time.”

Steven Pinker, author of open letter · 26 September 2026 · Steven Pinker Replies to Scott Alexander on AI (Quillette)

The capabilities of LLMs trained on huge datasets are more general than Pinker and almost everyone else would have guessed at the time he laid out his arguments, five years before LLM chatbots were released.

Mostly true True in substance, wrong in a detail.

The second judge would not resolve this claim; the verdict shown is the first judge's.

What would count as true or false

The claim has two parts: (a) Pinker's own expectations at the earlier time, and (b) the expectations of 'almost everyone else'. The proposition dates this to 2018. If the time Pinker refers to is a nearby year, treat that as a detail. TRUE: Pinker's statements from around that time express doubt about, or a narrower expectation of, what models trained on large text datasets could do in general, AND expert surveys, forecasts or commentary from then show most researchers likewise did not expect the broad general capabilities LLMs have shown by 2026. MOSTLY TRUE: both hold, but a noticeable minority of experts did predict such generality, or the reference year is slightly off. MIXED: one part holds and the other does not, e.g. Pinker was skeptical but many experts at the time expected broad generality. MOSTLY FALSE: contemporary evidence shows that expectations of broad generality were common, or Pinker himself had predicted it, while some people were still surprised. FALSE: Pinker and most experts at the time openly expected the capabilities that emerged.

Evidence

1. Why Counter-Enlightenment Convictions Are So Resilient Quillette · 20 April 2018 Own record

   ““General AI” does not exist at present, and is probably an incoherent concept: a sloppy extrapolation of individual differences among human beings.”

2. Book Review: Enlightenment Now, by Steven Pinker EA Forum (Effective Altruism) · 21 October 2018

   “I will add only that, given Pinker’s belief that humans have achieved incredible power and wealth through the use of reason and cooperation, it seems odd that he thinks AI will never be similarly capable.”

3. 2016 Expert Survey on Progress in AI [AI Impacts Wiki] AI Impacts · 14 December 2016

   “Here the median answer to the most similar question puts a 50% chance of HLMI in 2057”

4. 2016 Expert Survey on Progress in AI [AI Impacts Wiki] AI Impacts · 14 December 2016

   “Lots of ‘narrow’ AI milestones are forecast in the next decade as likely as not.”

Reasoning

In 2018 Pinker wrote that "General AI" does not exist and is "probably an incoherent concept" (E1). An independent review from the same year describes him as believing AI will never be similarly capable to humans (E2). Together these show he was sceptical at the time about broadly general AI. For other experts, the 2016 expert survey put the median 50% chance of human-level machine intelligence at 2057 (E3) and forecast many 'narrow' milestones as likely as not within the following decade (E4), which suggests most researchers did not expect broad generality by the mid-2020s. The verdict is mostly true rather than true because none of the evidence is specifically about models trained on large text datasets, and the survey quotes do not say how many experts expected faster progress.

Cite this claim

Steven Pinker, “The capabilities of LLMs trained on huge datasets are more general than Pinker and almost everyone else would have guessed at the time he laid out his arguments, five years before LLM chatbots were released.” (said 26 September 2026). Verdict: mostly true. Reliability API, claim c_dc850f38ffb2, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_dc850f38ffb2