Sixty years later, AI has made progress on a number of the earlier problems discussed, such as vision, though AI systems have not yet improved themselves.
Mostly true True in substance, wrong in a detail.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judge as of 2023-03-30. '60 years later' is a rough figure for the time since the early AI agenda (e.g., the 1956 Dartmouth proposal), so roughly 55–70 years counts. The claim has two parts: (a) by 2023, AI had made real progress on several problems from that early agenda, such as vision and language; (b) no AI system had yet improved itself in the sense of autonomous, recursive self-improvement. True: the evidence supports both parts. Mostly true: both parts hold with a minor slip, such as the year count being somewhat off. Mixed: one part is clearly supported and the other is contested, for example there were documented cases widely regarded as AI systems substantially improving their own design. Mostly false: the progress was real but much narrower than stated, or self-improvement had clearly occurred in a meaningful sense. False: there had been no meaningful progress on these problems, or AI systems had clearly and recursively improved themselves.
Set before the research began.
Evidence (4)
-
E1 A PROPOSAL FOR THE DARTMOUTH SUMMER RESEARCH PROJECT ON ARTIFICIAL INTELLIGENCE J. McCarthy, M. L. Minsky, N. Rochester, C.E. Shannon (proposal authors) · 31 August 1955
“An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.”
-
E2 Microsoft Researchers' Algorithm Sets ImageNet Challenge Milestone - Microsoft Research Microsoft Research · 10 February 2015
“the researchers say their system achieved a 4.94 percent error rate on the 1000-class ImageNet 2012 classification dataset”
-
E3 Technical Performance Stanford Institute for Human-Centered AI (HAI), AI Index · 1 January 2023
“Recently released models challenge that trend; BEiT-3, PaLI, and Gato, among others, are single AI systems increasingly capable of navigating multiple tasks (for example, vision, language).”
-
E4 Technical Performance Stanford Institute for Human-Centered AI (HAI), AI Index · 1 January 2023
“Nvidia used an AI reinforcement learning agent to improve the design of the chips that power AI systems. Similarly, Google recently used one of its language models, PaLM, to suggest ways to improve the very same model.”
Reasoning
E1 shows that the 1955 Dartmouth proposal set out goals including language, abstraction and machines that "improve themselves". That is about 67 years before 2023, which falls within the accepted range. E2 and E3 show clear progress on vision and language by 2023, including a 4.94 percent error rate on ImageNet classification [E2] and multi-task models that handle both [E3]. E4 reports that AI was used to help improve chip design and the PaLM model. Those are human-directed, AI-assisted improvements, not autonomous recursive self-improvement, so they mostly fit the claim that AI had not yet improved itself, but they add some nuance and do not directly confirm that no such case existed.
Cite this claim
Eliezer Yudkowsky, “Sixty years later, AI has made progress on a number of the earlier problems discussed, such as vision, though AI systems have not yet improved themselves.” (said 30 March 2023). Verdict: mostly true. Reliability API, claim c_d20bbbdfc55e, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_d20bbbdfc55e
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Eliezer Yudkowsky
-
Resolved true
“We learned that that's false, but we did not learn that you can have a thing that can redesign a tree and build an AI much smarter than itself, that builds an AI much smarter than itself, and that…”
-
Mostly true
“I was a passionate noticer of the human brain not being homogenous, and was therefore unpleasantly surprised by transformers scaling as well as they did.”
-
Mostly true
“First of all, you realize in principle that the task of minimizing losses on predicting human text does not stop when you’re as smart as a human, right?”
-
Mostly true
“Transformers are the main thing like that.”
-
Mostly true
“Because the basic paradigm of machine learning that we are presently operating under is that you can have the loss function, but only for things you can evaluate.”
Verdicts are made automatically; how verdicts are made.