Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
83 77 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
Insufficient record: 4 predictions resolved, 5 needed for a score.
-
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
77 65 88
All types combined: 82 (likely 77 to 86, high confidence), from 94 claims: 77 factual claims, 17 promises. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Satya Nadella's claims
43 claims
“we built all the responsible AI and guardrails right at the very beginning so that when people are deploying the Copilot, they know that they have the best safety around the Copilot built in.”
Satya Nadella, Microsoft CEO 15 November 2023 CNBC Transcript: Nadella on The Exchange (CNBC)
Microsoft built responsible AI measures and guardrails into Copilot from the very beginning.
Mostly true True in substance, wrong in a detail.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judged as of 2023-11-15. True: documentation from Microsoft or third parties shows that Copilot products had responsible-AI measures and guardrails at their initial launch, such as content filtering, safety systems, or responsible-AI reviews applied before or at launch (e.g. Bing Chat/Copilot in Feb 2023, Microsoft 365 Copilot). Mostly true: guardrails were present at launch but significant ones were added or strengthened soon after in response to problems. Mixed: some Copilot products had launch guardrails and others clearly did not. Mostly false: dated reports show major guardrails were added mainly after launch in response to incidents (e.g. the Bing chat behaviour incidents of Feb 2023), so 'from the very beginning' overstates what happened. False: evidence shows Copilot launched without meaningful responsible-AI measures or guardrails.
Evidence
- Our approach to Responsible AI - The new Bing Microsoft · April 2023 Own record
“As described above, we have implemented safeguards that mitigate potentially harmful content and we provide parental controls to enable appropriate use by younger users of the new Bing.”
- Microsoft AI Red Team building future of safer AI | Microsoft Security Blog Microsoft Security Blog · 7 August 2023 Own record
“For instance, before the new Bing chat experience was released, a team of dozens of security and responsible AI experts across the company spent hundreds of hours probing for novel security and responsible AI risks.”
- OpenAI reportedly warned Microsoft about Bing's bizarre AI responses The Verge · 13 June 2023
“Microsoft was quick to limit Bing Chat responses to stop the AI from getting real weird, and it has taken months of work to get the Bing chatbot back to a point where you can have a long back-and-forth conversation without an unexpected outburst.”
Reasoning
Microsoft's own security blog [E2] says dozens of security and responsible-AI experts red-teamed the new Bing chat for responsible-AI risks before it was released. Microsoft's April 2023 Responsible AI document [E1] describes safeguards against harmful content. The Verge [E3], an independent source, reports that Microsoft was quick to limit Bing Chat responses to stop unusual behaviour, and that it took months of further work before long conversations were possible without unexpected outbursts. Together this fits the criteria's "mostly true": there were responsible-AI measures at launch, but significant guardrails were added soon after in response to problems.
Cite this claim
Satya Nadella, “Microsoft built responsible AI measures and guardrails into Copilot from the very beginning.” (said 15 November 2023). Verdict: mostly true. Reliability API, claim c_786e794df0fa, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_786e794df0fa
Claims without a verdict
Is a verdict here wrong, or do you speak for Satya Nadella? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.