Nobody currently knows how to build a safe, friendly ("nice") AI.
Resolved true The record shows it happened or was so.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judge as of 2018-02-28. True: the AI safety/alignment literature and expert statements of that time show there was no known, demonstrated method for building a reliably safe, friendly (aligned) advanced or general AI. Mostly true: this was the clear consensus view, with only a few dissenters claiming a solution that was not widely accepted. Mixed: a substantial share of relevant experts held that a known method was sufficient. Mostly false: the claim rests on real open problems but overstates them, for example methods were widely regarded as adequate for the systems in question. False: a demonstrated, broadly accepted method for building safe, friendly advanced AI existed at the time.
Set before the research began.
Evidence (4)
-
E1 Concrete Problems in AI Safety arXiv (ar5iv rendering) · 24 June 2015
“Hard-coding against every possible failure is unlikely to be feasible in these cases, so a more principled approach to preventing harmful exploration seems essential.”
-
E2 Concrete Problems in AI Safety arXiv (ar5iv rendering) · 24 June 2015
“with the understanding that these avenues are preliminary ideas that have not been fully fleshed out.”
-
E3 2016 Expert Survey on Progress in AI AI Impacts · 14 December 2016
“Nearly half of researchers want to see more safety research than we currently have (compared to only 11% who think we are already prioritizing safety too much).”
-
E4 Introductory Resources on AI Safety Research - Future of Life Institute Future of Life Institute · 29 February 2016
“Research agenda focusing on accident risks that apply to current ML systems as well as more advanced future AI systems.”
Reasoning
The AI safety research agenda "Concrete Problems in AI Safety" treats safe behaviour as an open research problem. It says hard-coding against failures is unlikely to be feasible and that a more principled approach is needed (E1). It also calls its own proposed approaches preliminary ideas that have not been fully developed (E2). The 2016 expert survey found that nearly half of researchers wanted more safety research (E3), and FLI's resources present safety as an ongoing research agenda for current and future systems (E4). None of the evidence points to a demonstrated, accepted method for building reliably safe, friendly advanced AI as of early 2018.
Cite this claim
Eliezer Yudkowsky, “Nobody currently knows how to build a safe, friendly ("nice") AI.” (said 28 February 2018). Verdict: resolved true. Reliability API, claim c_489f26cfbd00, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_489f26cfbd00
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Eliezer Yudkowsky
-
Mixed
“I was observing that obvious-seeming hypothesis myself, sometime around the time of GPT 3.5.”
Yudkowsky himself observed the persona-selection-like hypothesis around the time of GPT 3.5.
-
Mostly true
“The test-passing preference stands on its own once formed.”
-
Mostly true
“LLMs surely have neural circuits they learned to model text streams from fictional and nonfictional personas that are not lying, deceiving, cheating.”
-
Mostly true
“The predict-as-if-honest circuitry is contextual, invoked to predict some streams of text but not others.”
-
Mostly true
“Presumably, somewhere along the way LLMs learn a dispreference for tests that don't pass, independently of how that gets prevented or whether the new plan does what the user really wanted.”
Verdicts are made automatically; how verdicts are made.