AlphaZero surpassed all accumulated human knowledge about Go after about a day of self-play, without human playbooks or sample games.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judge as of 2022-06-10 against DeepMind's published AlphaZero/AlphaGo Zero papers and reports. The claim has three parts: (a) AlphaZero passed the level of all accumulated human Go knowledge, meaning it beat top human-level play or a system that had already beaten world-champion humans, such as AlphaGo Lee; (b) it did so after 'a day or so' of self-play training, read loosely as roughly 1–2 days of wall-clock training; (c) it used no human game records or playbooks, only the rules. True: the sources show all three for AlphaZero. Mostly true: all three hold but with a minor slip, such as the superhuman milestone coming at somewhat under a day or up to about 3 days, or the figure belonging to the closely related AlphaGo Zero rather than AlphaZero. Mixed: one of the three parts clearly fails and the others hold, for example the training took well beyond a few days. Mostly false: the system did pass human level without human data, but the timing is badly wrong (weeks or more), or the 'all human knowledge' framing is unsupported by any benchmark. False: AlphaZero relied on human game data, or never reached superhuman Go play.
Set before the research began.
Evidence (4)
-
E1 AlphaZero: Shedding new light on chess, shogi, and Go DeepMind · 6 December 2018
“The amount of training the network needs depends on the style and complexity of the game, taking approximately 9 hours for chess, 12 hours for shogi, and 13 days for Go.”
-
E2 Mastering the game of Go without human knowledge - Nature Nature · 19 October 2017
“Here we introduce an algorithm based solely on reinforcement learning, without human data, guidance or domain knowledge beyond game rules.”
-
E3 Mastering the game of Go without human knowledge - Nature Nature · 19 October 2017
“Extended Data Figure 1 Tournament games between AlphaGo Zero (20 blocks, 3 days) versus AlphaGo Lee using 2 h time controls.”
-
E4 AlphaZero: Shedding new light on chess, shogi, and Go DeepMind · 6 December 2018
“In Go, AlphaZero defeated AlphaGo Zero, winning 61% of games.”
Reasoning
The evidence shows that AlphaGo Zero was trained without human data [E2] and that AlphaZero took 13 days to train for Go [E1]. It does not show the result of the 3-day AlphaGo Zero's games against AlphaGo Lee [E3], and it does not link any of these systems to a benchmark of top human play. That leaves the human-level milestone and its timing unsettled.
Cite this claim
Eliezer Yudkowsky, “AlphaZero surpassed all accumulated human knowledge about Go after about a day of self-play, without human playbooks or sample games.” (said 10 June 2022). Verdict: unresolvable. Reliability API, claim c_9483d6271cac, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_9483d6271cac
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Eliezer Yudkowsky
-
Resolved true
“We learned that that's false, but we did not learn that you can have a thing that can redesign a tree and build an AI much smarter than itself, that builds an AI much smarter than itself, and that…”
-
Mostly true
“I was a passionate noticer of the human brain not being homogenous, and was therefore unpleasantly surprised by transformers scaling as well as they did.”
-
Mostly true
“First of all, you realize in principle that the task of minimizing losses on predicting human text does not stop when you’re as smart as a human, right?”
-
Mostly true
“Transformers are the main thing like that.”
-
Mostly true
“Because the basic paradigm of machine learning that we are presently operating under is that you can have the loss function, but only for things you can evaluate.”
Verdicts are made automatically; how verdicts are made.