Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
76 70 82 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
60 42 78
All types combined: 74 (likely 68 to 80, high confidence), from 79 claims: 71 factual claims, 8 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Eliezer Yudkowsky's claims
21 claims
“AlphaFold2 was apparently also a trendline-breaking sudden jump in capability.”
Eliezer Yudkowsky, posting on X 12 December 2020 Post by @ESYudkowsky on X, 2020-12-12 (X)
AlphaFold2 was apparently a trendline-breaking sudden jump in capability on protein structure prediction.
Resolved true The record shows it happened or was so.
What would count as true or false
Judge as of 2020-12-12. True: CASP14 results show AlphaFold2 well above the historical trend of CASP performance (e.g. median GDT around 92 versus earlier progress), widely described as a discontinuous jump. Mostly true: a large jump that some analysts see as an acceleration partly anticipated by AlphaFold1, but still far above trend. Mixed: experts are seriously split on whether the gain broke the trend. Mostly false: the gain was large but roughly in line with the extrapolated trend after AlphaFold1. False: the CASP data show no unusual jump.
Evidence
- Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XIV Proteins (author manuscript, PMC), by Kryshtafovych, Schwede, Topf, Fidelis, Moult · 27 March 2025
“Astonishingly, the trend curve for CASP14 (the black straight-line) starts at a GDT_TS of about 95, and finishes at about 85 for difficult targets.”
- Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XIV Proteins (author manuscript, PMC), by Kryshtafovych, Schwede, Topf, Fidelis, Moult · 27 March 2025
“In CASP13 (2018)18, with the introduction of effective deep learning methods, the trend line rose to above 60 on the GDT_TS scale, even for the most difficult targets, a major advance from the previous CASP.”
- Before and after AlphaFold2: An overview of protein structure prediction PMC (mini-review journal article) · 28 February 2023
“Presented at CASP14 between May and July 2020, AlphaFold2 predicted protein structures with more accuracy than other competing methods, demonstrating a root-mean-square deviation (RMSD) among prediction and experimental backbone structures of 0.8Å versus the 2.8Å from the next best performing method.”
- Home Protein Structure Prediction Center (CASP, UC Davis, NIH/NIGMS-sponsored)
“After the success of AlphaFold2 in CASP14 (2020), it was expected that deep learning methodology that brought monomeric modeling to qualitatively new level will be extended to multimeric modeling.”
Reasoning
The CASP14 assessment calls the result "astonishing": the CASP14 trend curve runs from a GDT_TS of about 95 down to about 85 for difficult targets (E1). At CASP13 the trend line had risen only to just above 60 for the hardest targets, and that was already called a major advance (E2). An independent review reports a backbone RMSD of 0.8Å for AlphaFold2, against 2.8Å for the next-best method (E3). The CASP organisers say AlphaFold2 brought monomeric modelling to a "qualitatively new level" (E4). Together these show a jump far above the earlier trend, and none of the evidence presents it as an expected extension of AlphaFold1.
Cite this claim
Eliezer Yudkowsky, “AlphaFold2 was apparently a trendline-breaking sudden jump in capability on protein structure prediction.” (said 12 December 2020). Verdict: resolved true. Reliability API, claim c_b6bfe09c187d, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_b6bfe09c187d
Claims without a verdict
Is a verdict here wrong, or do you speak for Eliezer Yudkowsky? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.