AlphaZero surpassed human Go knowledge in less than a day starting from scratch, without human games, theories or Go-specific code beyond what chess used.
Mixed Partly true and partly false in roughly equal measure.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge said mostly false; the verdict shown is the first judge's.
What would count as true or false
True: DeepMind's published AlphaZero results (2017 preprint / 2018 Science paper) show that AlphaZero, trained from random play with no human game data and with no Go-specific code beyond what its chess version used (apart from rules), reached or exceeded top-human / prior human-knowledge-based Go performance (e.g. surpassing AlphaGo Lee or equivalent) within less than 24 hours of training. Mostly true: everything holds but a secondary detail is slightly off (e.g. some small domain-specific adjustment such as symmetry augmentation existed, or the benchmark surpassed was a human-knowledge-based program rather than humans directly), while the time was still under a day. Mixed: AlphaZero did start from scratch and surpass human-level Go, but the reported time to do so was over a day or unclear, or the 'less than a day' figure applies only to chess/shogi. Mostly false: the under-a-day figure is real only for a different game or measure, and in Go AlphaZero took clearly longer or did not clearly surpass human-level play within the timeframe. False: AlphaZero used human games or substantial Go-specific knowledge, or did not surpass human-level Go play.
Set before the research began.
Evidence (4)
-
E1 Science Journals (reprint of Silver et al., 'A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play', Science 362, 1140–1144) ailab-ua.github.io course resources (copy of Science paper) · 7 December 2018
“and in Go, AlphaZero first outperformed AlphaGo Lee (9) after 30 hours (74,000 steps).”
-
E2 Science Journals (reprint of Silver et al., 'A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play', Science 362, 1140–1144) ailab-ua.github.io course resources (copy of Science paper) · 7 December 2018
“Training lasted for approximately 9 hours in chess, 12 hours in shogi, and 13 days in Go (see table S3) (20).”
-
E3 Science Journals (reprint of Silver et al., 'A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play', Science 362, 1140–1144) ailab-ua.github.io course resources (copy of Science paper) · 7 December 2018
“Starting from random play and given no domain knowledge except the game rules, AlphaZero convincingly defeated a world champion program in the games of chess and shogi (Japanese chess), as well as Go.”
-
E4 AlphaGo Zero - Wikipedia Wikipedia
“In December 2017, AlphaZero beat the 3-day version of AlphaGo Zero by winning 60 games to 40, and with 8 hours of training it outperformed AlphaGo Lee on an Elo scale.”
Reasoning
The Science paper says AlphaZero started from random play with no domain knowledge beyond the game rules and beat a world-champion program in Go as well as in chess and shogi [E3]. This supports the "starting from scratch" part of the claim. On timing, Wikipedia reports that the December 2017 preprint had AlphaZero outperforming AlphaGo Lee after 8 hours [E4]. The formal Science publication of these results instead puts the first time it outperformed AlphaGo Lee in Go at 30 hours, with Go training lasting 13 days [E1, E2]. The sources give different times for when AlphaZero first outperformed AlphaGo Lee in Go (8 hours in [E4], 30 hours in [E1]), so the Go time is unclear, which matches the "mixed" criterion.
Cite this claim
Eliezer Yudkowsky, “AlphaZero surpassed human Go knowledge in less than a day starting from scratch, without human games, theories or Go-specific code beyond what chess used.” (said 28 February 2018). Verdict: mixed. Reliability API, claim c_c0ac4f6024e8, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_c0ac4f6024e8
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Eliezer Yudkowsky
-
Resolved true
“We learned that that's false, but we did not learn that you can have a thing that can redesign a tree and build an AI much smarter than itself, that builds an AI much smarter than itself, and that…”
-
Mostly true
“I was a passionate noticer of the human brain not being homogenous, and was therefore unpleasantly surprised by transformers scaling as well as they did.”
-
Mostly true
“First of all, you realize in principle that the task of minimizing losses on predicting human text does not stop when you’re as smart as a human, right?”
-
Mostly true
“Transformers are the main thing like that.”
-
Mostly true
“Because the basic paradigm of machine learning that we are presently operating under is that you can have the loss function, but only for things you can evaluate.”
Verdicts are made automatically; how verdicts are made.