Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
79 72 85 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
80 72 88 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Gary Marcus's claims
“I guarantee that minds will be blown.”
Gary Marcus, author 25 December 2022 What to Expect When You're Expecting … GPT-4 (Marcus on AI)
GPT-4 will blow people's minds when released.
Mostly true True in substance, wrong in a detail.
The second judge said resolved true; the verdict shown is the first judge's.
What would count as true or false
Judge on GPT-4's reception in the weeks after its release (March 2023). True: widespread reports from media, experts and users describe GPT-4 as strikingly impressive or a major leap, for example coverage of its exam results, multimodal demos and viral reactions. Mostly true: the reaction was strongly positive and impressed but somewhat less sweeping, or confined mainly to specific capabilities. Mixed: reactions were roughly evenly split between amazement and indifference or disappointment. Mostly false: the reception was mostly muted, with only scattered enthusiasm. False: GPT-4 was broadly seen as underwhelming, or was not released.
Evidence
- GPT-4 Is Acing Almost Every Higher-Learning Exam InsideHook · 16 March 2023
“OpenAI’s latest language model, GPT-4, released earlier this week, can now easily ace bar exams, LSATs and other higher-learning tests.”
- GPT-4 Has Been Out For 1 Day. These New Projects Show Just How Much More Powerful It Is Yahoo (news page; article carries Getty Images credit, consistent with a Time-style news report) · 15 March 2023
“The tool is so powerful that it can take a photo of a hand-drawn mockup for a simple website and turn it into an actual website using HTML and Javascript code.”
- First impressions of OpenAI’s new GPT-4 AI model Lux Capital · 15 March 2023
“Overall, I do think the model is meaningfully better in performance quality, the big launch around multimodal.”
- GPT-4’s Successes, and GPT-4’s Failures Communications of the ACM (blog, republished from The Road to AI We Can Trust) · 15 March 2023 Own record
“Massive scaling has not thus far lead to revolution. Although GPT-4 is clearly better than GPT-3 and 3.5, it is not so far as I can tell, qualitatively better, only quantitatively better;”
Reasoning
GPT-4 came out in March 2023, and independent coverage within days described it as a striking leap. E1 reported that it could "easily ace" bar exams, LSATs and other tests, and E2 said it was "so powerful" that it could turn a hand-drawn mockup into a working website. A first-impressions piece from an investor (E3) was more measured, calling the model "meaningfully better" and noting its multimodal launch. The subject's own essay (E4) was skeptical, but E4 is not independent, so I gave it little weight. The reception was clearly impressed, but these few sources do not fully show the widespread reaction that the "true" rating requires, so the verdict is mostly true.
Cite this claim
Gary Marcus, “GPT-4 will blow people's minds when released.” (said 25 December 2022). Verdict: mostly true. Reliability API, claim c_b63d9454d4c3, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_b63d9454d4c3
Claims without a verdict
Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.