Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
68 61 75 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
42 25 60 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 65 (likely 58 to 71, high confidence), from 78 claims: 69 factual claims, 9 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Ray Kurzweil's claims
18 claims
“Computers will be able to read on their own, understanding and modeling what they have read, by the second decade of the twenty-first century.”
Ray Kurzweil, essay author January 2001 The Law of Accelerating Returns (Kurzweil Library)
Computers will be able to read on their own, understanding and modeling what they read, by the second decade of the 21st century.
Mostly false False in substance, right in a detail.
What would count as true or false
Judge at the end of 2020. True: by the end of 2020, documented computer systems could read unrestricted natural-language text on their own (without human hand-coding of its content) and show understanding by building usable models of it. That understanding would show in answering questions, summarizing or drawing inferences about general text at roughly human level on broad evaluations. Mostly true: such systems existed by 2020, with only minor caveats, such as known failure modes that do not undercut general reading ability. Mostly false: by 2020 machines matched humans only on narrow reading-comprehension benchmarks or restricted domains, without general autonomous reading and understanding. False: by the end of 2020 no system came close to autonomous reading with understanding of general text.
Evidence
- Machines Beat Humans on a Reading Test. But Do They Understand? | Quanta Magazine Quanta Magazine · 17 October 2019
“But it's definitely not understanding English in a comprehensive and robust way.”
- Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data ACL Anthology (Bender and Koller, ACL 2020) · 2020
“we argue that a system trained only on form has a priori no way to learn meaning.”
- Measuring Massive Multitask Language Understanding arXiv (Hendrycks et al.) · September 2020
“However, on every one of the 57 tasks, the best models still need substantial improvements before they can reach human-level accuracy.”
Reasoning
E1 (Quanta, 2019) reports machines beating humans on a reading test but quotes an expert saying such a system is 'definitely not understanding English in a comprehensive and robust way.' E3 (Hendrycks et al., September 2020) finds that on a broad 57-task language-understanding evaluation the best models still needed substantial improvement to reach human-level accuracy on every task. E2 (ACL 2020) argues that systems trained only on form cannot learn meaning. Together these show human-level performance only on narrow reading benchmarks by 2020, without general autonomous reading with understanding, which matches the mostly-false criterion.
Cite this claim
Ray Kurzweil, “Computers will be able to read on their own, understanding and modeling what they read, by the second decade of the 21st century.” (said January 2001). Verdict: mostly false. Reliability API, claim c_3685c9e4bfb5, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_3685c9e4bfb5
Claims without a verdict
Is a verdict here wrong, or do you speak for Ray Kurzweil? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.