Statistical AI systems have pretty decisively won out over rule-based systems.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
The claim is hedged ('pretty decisively'). It says that by July 2017, statistical or machine-learning approaches had clearly overtaken rule-based or symbolic approaches in AI. TRUE: as of mid-2017, state-of-the-art results and deployed systems in major AI areas (vision, speech, machine translation, game playing) were dominated by statistical or neural methods, and the expert consensus recognised this. MOSTLY TRUE: dominance was clear in most major areas, with some notable domains (for example certain planning, verification or expert-system applications) still relying on rule-based methods. MIXED: statistical methods led in some major areas, while rule-based methods remained competitive or dominant in others of comparable importance. MOSTLY FALSE: statistical methods were rising but not yet dominant in most areas. FALSE: rule-based approaches still dominated in 2017.
Set before the research began.
Evidence (3)
-
E1 The JHU Machine Translation Systems for WMT 2017 Johns Hopkins University (Ding, Khayrallah, Koehn, Post, Kumar, Duh); Proceedings of the Conference on Machine Translation (WMT), Volume 2 · September 2017
“Ourphrase-basedsystemswereclearlyoutperformedbyNMTsystemsforalllanguagepairs,byadifferenceof3.2to8.3BLEUpoints.”
-
E2 The University of Edinburgh's Neural MT Systems for WMT17 arXiv (ar5iv rendering), University of Edinburgh authors
“Among the constrained submissions to the news task, our submissions are ranked tied first for 11 out of the 12 translation directions in which we participated.”
-
E3 Learning Transferable Architectures for Scalable Image Recognition arXiv (Zoph, Vasudevan, Shlens, Le; Google Brain) · July 2017
“achieves stateof-the-art accuracy of 82.3% top-1 and 96.0% top-5 on ImageNet, which is 0.8%”
Reasoning
The evidence shows neural systems beating or leading phrase-based systems in two teams' 2017 machine translation results [E1][E2], plus one state-of-the-art image-recognition result [E3]. It says nothing about speech, game playing, areas that still used rule-based methods, or expert consensus, so it cannot show whether statistical methods dominated most major AI areas as the criteria require.
Cite this claim
Dario Amodei, “Statistical AI systems have pretty decisively won out over rule-based systems.” (said 21 July 2017). Verdict: unresolvable. Reliability API, claim c_e7819e995657, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_e7819e995657
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Dario Amodei
-
Mixed
“It’s easy to dismiss this as a one-off or curiosity, but we’ve repeatedly seen a pattern where AI performance in new intellectual domains goes from weak to superhuman in a matter of a few years.”
-
Mixed
“we were measuring these trends back when models cost $1,000 or $10,000 to train”
Amodei's group measured AI scaling trends when models cost $1,000 or $10,000 to train.
-
Mixed
“a lot of the other reasoning models in the market are trained primarily on math and competition coding,”
-
Mostly true
“I do think AI coding is moving the fastest of all the other areas.”
Amodei said he thinks AI coding is moving the fastest of all the areas of AI.
-
Mostly true
“Because when it knows that it doesn't get any gain out of thinking further, it doesn't think for longer.”
Verdicts are made automatically; how verdicts are made.