Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
81 76 87 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
Insufficient record: 2 predictions resolved, 5 needed for a score.
-
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
65 45 83
All types combined: 80 (likely 75 to 85, high confidence), from 84 claims: 78 factual claims, 6 promises. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Demis Hassabis's claims
37 claims
“AlphaFold2 achieved atomic accuracy at CASP14 (2020)”
Demis Hassabis, Nobel lecture slides December 2024 Accelerating scientific discovery with AI (Nobel lecture slides) (NobelPrize.org)
AlphaFold2 achieved atomic accuracy at CASP14 in 2020.
Resolved true The record shows it happened or was so.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judged as of 2024-12. True if CASP14 assessors' reports, the CASP14 results or peer-reviewed papers (e.g. Jumper et al. 2021, Nature) describe AlphaFold2's CASP14 (2020) predictions as reaching atomic or near-atomic accuracy. A benchmark is a median backbone RMSD of about 1 Å or a median GDT around 90 or more. Mostly true if sources describe the accuracy as 'competitive with experimental' or 'near-atomic' for most targets but not all. Mostly false if it was only the top performer, without accuracy approaching the atomic level. False if AlphaFold2 did not compete at CASP14 or performed poorly.
Evidence
- Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XIV Proteins (author manuscript, PMC); Kryshtafovych, Schwede, Topf, Fidelis, Moult (CASP assessors and Prediction Center)
“One research group, AlphaFold2 from the company DeepMind, submitted models competitive with experimental accuracy for at least 2/3 of the targets”
- Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XIV Proteins (author manuscript, PMC); Kryshtafovych, Schwede, Topf, Fidelis, Moult (CASP assessors and Prediction Center)
“In CASP14, about 2/3 of the 96 targets reached GDT_TS values greater than that, and so are considered competitive with experiment in backbone accuracy.”
- Applying and improving AlphaFold at CASP14 - PMC Proteins (PMC); Jumper, Evans, Pritzel et al. (DeepMind) · 24 November 2021 Own record
“The predictions made by AlphaFold had a median domain GDT_TS of 92.4”
- Highly accurate protein structure prediction with AlphaFold - Nature Nature · 15 July 2021 Own record
“AlphaFold structures had a median backbone accuracy of 0.96 Å r.m.s.d.95”
Reasoning
The independent CASP14 assessors' overview says AlphaFold2 submitted models competitive with experimental accuracy for at least two-thirds of the targets (E1, E2). The peer-reviewed Jumper et al. 2021 Nature paper, which the criteria name directly, reports a median backbone accuracy of 0.96 Å r.m.s.d.95 (E4). That meets the benchmark of about 1 Å. The DeepMind Proteins paper reports a median domain GDT_TS of 92.4 (E3), which is above the benchmark of about 90. E3 and E4 come from the subject's own group. The independent assessors report that AlphaFold2 was competitive with experimental accuracy for at least two-thirds of the targets [E1][E2], which is consistent with them, and the median domain GDT_TS of 92.4 for AlphaFold's CASP14 predictions in the peer-reviewed E3 meets the benchmark for true.
Cite this claim
Demis Hassabis, “AlphaFold2 achieved atomic accuracy at CASP14 in 2020.” (said December 2024). Verdict: resolved true. Reliability API, claim c_f86e662be682, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_f86e662be682
Claims without a verdict
Is a verdict here wrong, or do you speak for Demis Hassabis? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.