As of 2017, neural nets provided near human-level and in some cases superhuman performance on tasks such as translating languages, driving cars and playing go.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
What would count as true or false
This is a claim of degree. Judge it as of March 2017 for each of the three named tasks: translation, driving and go. True: neural-net systems had demonstrated near-human performance in translation (e.g., Google Neural Machine Translation evaluations) and in driving, and superhuman performance in at least one task (e.g., AlphaGo beating top professionals at go). Mostly true: two of the three tasks clearly meet the near-human standard, and the third is meaningfully advanced but somewhat short of it. Mixed: only about one of the three named tasks meets the standard. Mostly false: neural nets were prominent in these areas but fell clearly short of near-human level in most of them. False: no named task reached near-human level using neural nets.
Set before the research began.
Evidence (11)
-
E1 Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation ar5iv (arXiv 1609.08144) · 1 May 2016
“Additional experiments suggest the quality of the resulting translation system gets closer to that of average human translators.”
-
E2 Hyperbolic? Experts Weigh In on Google Neural Translate Slator · 4 October 2016
“Rico Sennrich, on the other hand, says the study performed human evaluation on “relatively simplistic and isolated” sentences, and the claim that Google’s systems “bridge the gap between human and machine translation” simply “raises wrong expectations.””
-
E3 AlphaGo versus Lee Sedol - Wikipedia Wikipedia · 9 March 2016
“AlphaGo won all but the fourth game;”
-
E4 A Neural Network for Machine Translation, at Production Scale Google Research blog · 27 September 2016
“Machine translation is by no means solved. GNMT can still make significant errors that a human translator would never make, like dropping words and mistranslating proper names or rare terms, and translating sentences in isolation rather than considering the context of the paragraph or page.”
-
E5 Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation arXiv (ar5iv) · 1 May 2016
“Using human-rated side-by-side comparison as a metric, we show that our GNMT system approaches the accuracy achieved by average bilingual human translators on some of our test sets.”
-
E6 (none) Slator
“Sennrich quotes Google that machine translation is still far from solved, and adds that the quality of machine translation still “does not reach human translation quality on most text types.””
-
E7 (none) Slator
“This latest report describes incremental improvements, which is a fine accomplishment using some novel approaches; but it’s not groundbreaking.”
-
E8 AlphaGo versus Lee Sedol - Wikipedia Wikipedia · 9 March 2016
“AlphaGo won all but the fourth game;[1] all games were won by resignation.[2]”
-
E9 End-to-End Deep Learning for Self-Driving Cars | NVIDIA Technical Blog NVIDIA · 18 August 2016
“For a typical drive in Monmouth County NJ from our office in Holmdel to Atlantic Highlands, we are autonomous approximately 98% of the time.”
-
E10 The 2,578 Problems With Self-Driving Cars IEEE Spectrum · 2 February 2017
“That represents a sharp reduction in disengagements from the previous year, from 0.8 disengagements for every 1,000 miles of autonomous driving down to just 0.2.”
-
E11 The 2,578 Problems With Self-Driving Cars IEEE Spectrum · 2 February 2017
“But that doesn't mean Waymo's cars are 8,000 times safer than Bosch's, as every company has its own way of counting disengagements.”
Reasoning
The evidence supports strong go performance [E3, E8], but it does not show near-human driving [E9, E10, E11]. Near-human translation is contested: some sources say it approached human quality on some test sets [E1, E5] and others dispute this [E4, E6], so the evidence does not establish that two of the three tasks clearly met the near-human standard.
Cite this claim
Ray Kurzweil, “As of 2017, neural nets provided near human-level and in some cases superhuman performance on tasks such as translating languages, driving cars and playing go.” (said March 2017). Verdict: unresolvable. Reliability API, claim c_37febc8d24f6, method 0.1.0+28da144b, dataset release v1.1.2. https://epistacklabs.com/claims/c_37febc8d24f6
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Ray Kurzweil
-
Mostly false
“That was not true a year ago. But that’s true today.”
-
Mixed
“AI hallucinations are still there, but are becoming much less frequent.”
AI hallucinations persist but are becoming much less frequent.
-
Mostly true
“And it’s going very, very quickly — at an exponential rate”
AI progress is advancing very quickly, at an exponential rate.
-
Mostly true
“It was not quite as good, but you could recognize it as Mozart or as Chopin.”
-
Mostly true
“Yes, it was very much for AI.”
The prizes received by Geoffrey Hinton and by Demis Hassabis and John Jumper were very much for AI.
Verdicts are made automatically; how verdicts are made.