A joint DeepMind, Google Health and Cancer Research UK study showed AI can spot breast cancer in screening mammograms more accurately than experts.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
True: a peer-reviewed study (e.g. McKinney et al., Nature, January 2020) co-authored by DeepMind, Google Health and Cancer Research UK-affiliated researchers reports that an AI system had fewer false positives and/or false negatives than human expert readers on screening mammograms. Mostly true: the study exists and shows this, but there is a small inaccuracy in a detail, such as which organisations took part, or the AI beat experts on some metrics or datasets while roughly matching them on others. Mixed: the study exists, but the AI outperformed experts in only some comparisons and fell short in others. Mostly false: the study exists but found only performance similar to experts, or the superiority claim was formally corrected or retracted. False: no such joint study exists.
Set before the research began.
Evidence (4)
-
E1 AI outperforms radiologists in mammographic screening - Nature Reviews Clinical Oncology Nature Reviews Clinical Oncology (Nature Portfolio) · 21 January 2020
“The AI system resulted in absolute reductions of 1.2% and 2.7% in the rates of false-positive and false-negative detection of biopsy-confirmed breast cancers, respectively, in the UK test set and 5.7% and 9.4% in the US dataset, relative to the judgement of the first or sole radiologists.”
-
E2 Researchers criticize opacity of Google's mammogram-reading AI study - Becker's Oncology Becker's Oncology · 11 March 2020
“In the initial study, Google researchers found the DeepMind AI produced fewer false positives and false negatives than an expert panel of six radiologists in making breast cancer diagnoses based on mammograms.”
-
E3 Researchers criticize opacity of Google's mammogram-reading AI study - Becker's Oncology Becker's Oncology · 11 March 2020
“In the response, 19 researchers and the Massive Analysis Quality Control Society board of directors criticized Google’s failure to explain in the study how the AI was developed and to release the DeepMind algorithm’s code.”
-
E4 International evaluation of an AI system for breast cancer screening DeepMind · 1 January 2020 Own record
“we developed an AI system capable of surpassing clinical specialists from the UK and US in predicting breast cancer from mammograms, as confirmed by biopsy.”
Reasoning
Independent evidence shows a study in which the AI had fewer false positives and false negatives than radiologists on UK and US data [E1][E2], with E2 attributing it to Google researchers and the DeepMind AI. No evidence item confirms that Cancer Research UK-affiliated researchers co-authored the study, which the criteria require for "true" as distinct from "mostly true".
Cite this claim
Demis Hassabis, “A joint DeepMind, Google Health and Cancer Research UK study showed AI can spot breast cancer in screening mammograms more accurately than experts.” (said 1 January 2020). Verdict: unresolvable. Reliability API, claim c_5bce07945a8d, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_5bce07945a8d
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Demis Hassabis
-
Mostly true
“what’s important about them is their 3D structure.”
A protein's 3D structure is what determines, or partially determines, its function.
-
Resolved true
“So over 3 million scientists are now using AlphaFold.”
-
Resolved true
“Only about 10% of drugs actually get through all the clinical stages.”
Only about 10% of drugs get through all the clinical stages.
-
Mostly true
“The AlphaFold Server is empowering non-commercial researchers globally to harness this technology, accelerating their ability to formulate and test new hypotheses.”
-
Mostly true
“Our AlphaProteo model can design novel, high-strength protein binders that target diverse molecules – including those associated with cancer and diabetes.”
Verdicts are made automatically; how verdicts are made.