The capabilities of LLMs trained on huge datasets are more general than Pinker and almost everyone else would have guessed at the time he laid out his arguments, five years before LLM chatbots were released.

The claim has two parts: (a) Pinker's own expectations at the earlier time, and (b) the expectations of 'almost everyone else'. The proposition dates this to 2018. If the time Pinker refers to is a nearby year, treat that as a detail. TRUE: Pinker's statements from around that time express doubt about, or a narrower expectation of, what models trained on large text datasets could do in general, AND expert surveys, forecasts or commentary from then show most researchers likewise did not expect the broad general capabilities LLMs have shown by 2026. MOSTLY TRUE: both hold, but a noticeable minority of experts did predict such generality, or the reference year is slightly off. MIXED: one part holds and the other does not, e.g. Pinker was skeptical but many experts at the time expected broad generality. MOSTLY FALSE: contemporary evidence shows that expectations of broad generality were common, or Pinker himself had predicted it, while some people were still surprised. FALSE: Pinker and most experts at the time openly expected the capabilities that emerged.

Set before the research began.

1. E1 Why Counter-Enlightenment Convictions Are So Resilient Quillette · 20 April 2018 Own record

   ““General AI” does not exist at present, and is probably an incoherent concept: a sloppy extrapolation of individual differences among human beings.”

2. E2 Book Review: Enlightenment Now, by Steven Pinker EA Forum (Effective Altruism) · 21 October 2018

   “I will add only that, given Pinker’s belief that humans have achieved incredible power and wealth through the use of reason and cooperation, it seems odd that he thinks AI will never be similarly capable.”

3. E3 2016 Expert Survey on Progress in AI [AI Impacts Wiki] AI Impacts · 14 December 2016

   “Here the median answer to the most similar question puts a 50% chance of HLMI in 2057”

4. E4 2016 Expert Survey on Progress in AI [AI Impacts Wiki] AI Impacts · 14 December 2016

   “Lots of ‘narrow’ AI milestones are forecast in the next decade as likely as not.”

In 2018 Pinker wrote that "General AI" does not exist and is "probably an incoherent concept" (E1). An independent review from the same year describes him as believing AI will never be similarly capable to humans (E2). Together these show he was sceptical at the time about broadly general AI. For other experts, the 2016 expert survey put the median 50% chance of human-level machine intelligence at 2057 (E3) and forecast many 'narrow' milestones as likely as not within the following decade (E4), which suggests most researchers did not expect broad generality by the mid-2020s. The verdict is mostly true rather than true because none of the evidence is specifically about models trained on large text datasets, and the survey quotes do not say how many experts expected faster progress.