Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
68 61 75 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
42 25 60 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 65 (likely 58 to 71, high confidence), from 78 claims: 69 factual claims, 9 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Ray Kurzweil's claims
9 claims
“But I got to 10 billion. It started to do things.”
Ray Kurzweil, speaking at MWC 25 March 2025 Ray Kurzweil: 2022-2025 Updates (transcripts) (LifeArchitect.ai)
Kurzweil said that at about 10 billion connections neural networks started to do things.
Mixed Partly true and partly false in roughly equal measure.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judged as of March 2025. True: the record of AI development shows that neural networks with on the order of 10 billion parameters or connections (roughly 5 to 20 billion) marked roughly when language models began to show useful or notably new abilities, for example models of about 10 billion parameters around 2019–2020 such as T5-11B or Megatron, and the emergence of abilities around that scale. Mostly true: useful abilities did emerge near that scale, but the threshold figure is off by a modest factor (2–3x). Mixed: 10 billion is one of several plausible thresholds, and useful abilities clearly appeared at much smaller scales (for example BERT at about 340 million, GPT-2 at 1.5 billion). Mostly false: useful neural networks existed well below 10 billion, so the claim distorts a real scaling trend. False: the evidence shows no link between that scale and the onset of useful abilities.
Evidence
- Emergent Abilities of Large Language Models OpenReview (anonymous TMLR review version; Wei et al. paper)
“GPT-3 and LaMDA (Thoppilan et al., 2022) have close-to-zero performance for several orders of magnitude of training compute, before performance jumps to sharply above random at 2 · 1022 training FLOPs (13B parameters) for GPT-3, and 1023 training FLOPs (68B parameters) for LaMDA.”
- Emergent Abilities of Large Language Models OpenReview (anonymous TMLR review version; Wei et al. paper)
“For GPT-3, Gopher, and Chinchilla, models of ∼1022 training FLOPs (∼10B parameters) or smaller do not perform better than guessing on average over all the topics, scaling up to 3–5 ·1023 training FLOPs (70B–280B parameters) enables performance to substantially surpass random.”
- Emergent Abilities of Large Language Models OpenReview (anonymous TMLR review version; Wei et al. paper)
“Although we may observe an emergent ability to occur at a certain scale, it is possible that the ability could be later achieved at a smaller scale”
- Are Emergent Abilities of Large Language Models a Mirage? NeurIPS 2023 (Schaeffer, Miranda, Koyejo, Stanford University) · 2023
“for a particular task and model family, when analyzing fixed model outputs, emergent abilities appear due to the researcher’s choice of metric rather than due to fundamental changes in models with scale.”
Reasoning
E1 (Wei et al.) supports the claim to a degree: it reports that GPT-3's performance jumped from near zero to above random at about 13B parameters, which is within the claim's range. E2 reports that models of about 10B parameters or smaller performed no better than guessing on average over all topics, and that substantially above-random performance took roughly 70B–280B parameters. That is far more than the claimed threshold. E3 notes that such thresholds can move to smaller scales, and E4 argues that apparent emergence may come from the choice of metric. So 10 billion is one plausible threshold among several, not a clear onset point.
Cite this claim
Ray Kurzweil, “Kurzweil said that at about 10 billion connections neural networks started to do things.” (said March 2025). Verdict: mixed. Reliability API, claim c_5186a1cc596d, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_5186a1cc596d
Claims without a verdict
Is a verdict here wrong, or do you speak for Ray Kurzweil? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.