GPT-4 will be a very large, powerful system (a 'monster').

This vague prediction means GPT-4 would be a very large, very capable system. Judge it at GPT-4's release (March 2023). An undisclosed parameter count does not by itself settle it. TRUE: on release, GPT-4 was widely recognized as a major jump in capability over GPT-3.5 and prior systems, shown by large benchmark and professional-exam gains in OpenAI's technical report and confirmed by independent evaluations and broad coverage. MOSTLY TRUE: GPT-4 was clearly a significant step up, but some assessments called the gains moderate in certain areas. MIXED: assessments were split between seeing it as a major leap and seeing it as an incremental upgrade. MOSTLY FALSE: GPT-4 was generally seen as an incremental improvement with modest gains. FALSE: GPT-4 was widely regarded as underwhelming or no meaningful advance over GPT-3.5.

Set before the research began.

1. E1 GPT-4 OpenAI · 14 March 2023

   “For example, it passes a simulated bar exam with a score around the top 10% of test takers; in contrast, GPT‑3.5’s score was around the bottom 10%.”

2. E2 OpenAI Says GPT-4 Is Better in Nearly Every Way. What Matters More Is Millions Will Use It SingularityHub · 19 March 2023

   “It also outperformed the upgraded GPT-3 (called GPT 3.5) on a slew of standardized tests, placing among the top 10 percent of human test-takers on the bar licensing exam for lawyers and scoring either a 4 or a 5 on 13 out of 15 college-level advanced placement (AP) exams for high school students.”

3. E3 Why GPT-4 is notable but not groundbreaking University of Sydney (originally published in The Conversation) · 16 March 2023

   “One could argue GPT-4 represents only an incremental improvement over its predecessors in many practical scenarios. Results showed human judges preferred GPT-4 outputs over the most advanced variant of GPT-3.5 only about 61% of the time.”

4. E4 OpenAI Says GPT-4 Is Better in Nearly Every Way. What Matters More Is Millions Will Use It SingularityHub · 19 March 2023

   “While GPT-4 may not to be the step change some predicted, the scale of its deployment almost certainly will be.”

OpenAI's launch report (E1) shows large exam gains over GPT-3.5, such as a simulated bar exam score near the top 10% of test takers, compared with near the bottom 10% for GPT-3.5. Launch-week coverage (E2) confirmed it outperformed GPT-3.5 on many standardized tests. Some independent assessments, however, called the gains moderate: E3 described GPT-4 as an incremental improvement in many practical scenarios, citing a 61% human preference rate over GPT-3.5, and E4 said it may not be the step change some predicted. This fits the 'mostly true' criterion: a clear, significant step up, with some assessments calling the gains moderate.