Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
76 70 82 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
60 42 78
All types combined: 74 (likely 68 to 80, high confidence), from 79 claims: 71 factual claims, 8 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Eliezer Yudkowsky's claims
46 claims
“on the current optimization paradigm there is no general idea of how to get particular inner properties into a system, or verify that they’re there”
Eliezer Yudkowsky, author 10 June 2022 AGI Ruin: A List of Lethalities (MIRI)
Under the current optimization paradigm there is no general method known to get particular inner properties into an AI system or verify they are present.
Mostly true True in substance, wrong in a detail.
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judge as of 2022-06-10. The question is whether, under the gradient-descent and deep-learning paradigm, a general method was then known both to instil chosen internal properties (goals or cognition) in a trained system and to verify that they are present. True: reviews, surveys or expert statements from around mid-2022 say that no such general method existed, and that interpretability and alignment techniques were narrow, partial or unproven. Mostly true: that is broadly the case, but some methods did give limited verification or instilling of specific properties in restricted settings, which the word 'general' in the claim allows for. Mixed: the experts of the time were substantially divided on whether existing methods amounted to a general approach. Mostly false: credible evidence shows that methods existed by then which did general work in one of the two directions (instilling or verifying), and the claim overstates their absence. False: a demonstrated, broadly applicable method for both instilling and verifying such properties was known by June 2022.
Evidence
- Unsolved Problems in ML Safety arXiv (Hendrycks et al.)
“In contrast to typical software, ML control flows are specified by inscrutable weights learned by gradient optimizers rather than programmed with explicit instructions and general rules from humans.”
- Interpretable machine learning: Fundamental principles and 10 grand challenges Statistics Surveys (hosted by NSF Public Access Repository) · 2022
“In that sense, explainability methods are often used as an excuse to use a black box model–whether or not one is actually needed.”
- The Alignment Problem from a Deep Learning Perspective arXiv (ar5iv rendering)
“First, since present neural networks are effectively black boxes (Buhrmester et al. 2021), we cannot formally verify that they will reliably behave as intended, and need to rely more on informal analysis.”
- EIS III: Broad Critiques of Interpretability Research — LessWrong LessWrong (cross-posted to AI Alignment Forum) · 14 February 2023 Own record
“The interpretability field probably should -- but does not yet -- have clear and consistent evaluation methods.”
Reasoning
Independent reviews from around 2021–2022 describe deep-learning systems as governed by inscrutable weights learned by gradient optimizers [E1]. A review also says that because networks are effectively black boxes, their intended behaviour cannot be formally verified and more reliance on informal analysis is needed [E3]. A 2023 critique says interpretability still lacked clear and consistent evaluation methods [E4], which fits the claim that verification techniques were partial. The evidence supports the verification half of the claim, although partly through sources dated after the judging date, and speaks to the instilling half only indirectly. So the claim is broadly supported, and not every part of it is directly confirmed.
Cite this claim
Eliezer Yudkowsky, “Under the current optimization paradigm there is no general method known to get particular inner properties into an AI system or verify they are present.” (said 10 June 2022). Verdict: mostly true. Reliability API, claim c_b1990b620542, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_b1990b620542
Claims without a verdict
Is a verdict here wrong, or do you speak for Eliezer Yudkowsky? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.