Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
83 78 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
Insufficient record: 2 predictions resolved, 5 needed for a score.
-
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 83 (likely 78 to 88, high confidence), from 86 claims: 86 factual claims. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Daniela Amodei's claims
38 claims
“And the goal of that is really developing more monitoring tools around AI risk and AI risk management.”
Daniela Amodei, Anthropic president, interview 4 March 2022 Daniela and Dario Amodei on Anthropic (Future of Life Institute)
The goal of NIST's AI Risk Management Framework is to develop more monitoring tools for AI risk and its management.
Mostly true True in substance, wrong in a detail.
What would count as true or false
Judge as of 2022-03-04 against NIST's stated purpose for the AI RMF (RFI, concept paper, initial draft). True: NIST describes the framework's aim as helping organizations identify, assess, measure, monitor or manage AI risks, consistent with developing tools for monitoring and managing AI risk. Mostly true: the stated aim is managing AI risk broadly and monitoring is only one part of it, so the claim is a modest paraphrase. Mixed: monitoring tools are a minor or ambiguous aspect and the framework is chiefly voluntary guidance. Mostly false: the framework is a voluntary guidance document and NIST does not describe developing monitoring tools as a goal, so the claim distorts its purpose. False: NIST's stated goals are unrelated to AI risk management.
Evidence
- AI Risk Management Framework Concept Paper NIST · 13 December 2021
“The NIST AI RMF is intended to serve as a blueprint for mapping, measuring, and managing risks related to AI systems across a wide spectrum of types, applications, and maturity.”
- AI Risk Management Framework Concept Paper NIST · 13 December 2021
“Functions organize AI risk management activities at their highest level to establish the context and enumerate, assess, treat, monitor, review, and report risk.”
- AI Risk Management Framework: Initial Draft - March 17, 2022 NIST · 17 March 2022
“It is intended to improve understanding of and to help organizations manage both enterprise and societal risks related to the development, deployment, and use of AI systems.”
- Brief Summary NIST · 15 October 2021
“Metrics could support not only measurement of performance regarding risk management strategy, but also enable monitoring to detect data governance risks (e.g., conscious and unconscious bias within processes).”
Reasoning
NIST's concept paper describes the AI RMF as a blueprint for mapping, measuring and managing AI risks (E1), and lists monitoring as one of several risk management activities within its core functions (E2). The initial draft frames the aim as helping organizations manage enterprise and societal AI risks (E3). Managing AI risk broadly is the stated aim, with monitoring only one part of it, so the claim is a modest paraphrase.
Cite this claim
Daniela Amodei, “The goal of NIST's AI Risk Management Framework is to develop more monitoring tools for AI risk and its management.” (said 4 March 2022). Verdict: mostly true. Reliability API, claim c_127a744d9fad, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_127a744d9fad
Claims without a verdict
Is a verdict here wrong, or do you speak for Daniela Amodei? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.