AlphaFold2 was apparently a trendline-breaking sudden jump in capability on protein structure prediction.
Resolved true The record shows it happened or was so.
Factual: Statements about the present or the past, checked against the record: did they hold up?
What would count as true or false
Judge as of 2020-12-12. True: CASP14 results show AlphaFold2 well above the historical trend of CASP performance (e.g. median GDT around 92 versus earlier progress), widely described as a discontinuous jump. Mostly true: a large jump that some analysts see as an acceleration partly anticipated by AlphaFold1, but still far above trend. Mixed: experts are seriously split on whether the gain broke the trend. Mostly false: the gain was large but roughly in line with the extrapolated trend after AlphaFold1. False: the CASP data show no unusual jump.
Set before the research began.
Evidence (4)
-
E1 Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XIV Proteins (author manuscript, PMC), by Kryshtafovych, Schwede, Topf, Fidelis, Moult · 27 March 2025
“Astonishingly, the trend curve for CASP14 (the black straight-line) starts at a GDT_TS of about 95, and finishes at about 85 for difficult targets.”
-
E2 Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XIV Proteins (author manuscript, PMC), by Kryshtafovych, Schwede, Topf, Fidelis, Moult · 27 March 2025
“In CASP13 (2018)18, with the introduction of effective deep learning methods, the trend line rose to above 60 on the GDT_TS scale, even for the most difficult targets, a major advance from the previous CASP.”
-
E3 Before and after AlphaFold2: An overview of protein structure prediction PMC (mini-review journal article) · 28 February 2023
“Presented at CASP14 between May and July 2020, AlphaFold2 predicted protein structures with more accuracy than other competing methods, demonstrating a root-mean-square deviation (RMSD) among prediction and experimental backbone structures of 0.8Å versus the 2.8Å from the next best performing method.”
-
E4 Home Protein Structure Prediction Center (CASP, UC Davis, NIH/NIGMS-sponsored)
“After the success of AlphaFold2 in CASP14 (2020), it was expected that deep learning methodology that brought monomeric modeling to qualitatively new level will be extended to multimeric modeling.”
Reasoning
The CASP14 assessment calls the result "astonishing": the CASP14 trend curve runs from a GDT_TS of about 95 down to about 85 for difficult targets (E1). At CASP13 the trend line had risen only to just above 60 for the hardest targets, and that was already called a major advance (E2). An independent review reports a backbone RMSD of 0.8Å for AlphaFold2, against 2.8Å for the next-best method (E3). The CASP organisers say AlphaFold2 brought monomeric modelling to a "qualitatively new level" (E4). Together these show a jump far above the earlier trend, and none of the evidence presents it as an expected extension of AlphaFold1.
Cite this claim
Eliezer Yudkowsky, “AlphaFold2 was apparently a trendline-breaking sudden jump in capability on protein structure prediction.” (said 12 December 2020). Verdict: resolved true. Reliability API, claim c_b6bfe09c187d, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_b6bfe09c187d
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Eliezer Yudkowsky
-
Resolved true
“We learned that that's false, but we did not learn that you can have a thing that can redesign a tree and build an AI much smarter than itself, that builds an AI much smarter than itself, and that…”
-
Mostly true
“I was a passionate noticer of the human brain not being homogenous, and was therefore unpleasantly surprised by transformers scaling as well as they did.”
-
Mostly true
“First of all, you realize in principle that the task of minimizing losses on predicting human text does not stop when you’re as smart as a human, right?”
-
Mostly true
“Transformers are the main thing like that.”
-
Mostly true
“Because the basic paradigm of machine learning that we are presently operating under is that you can have the loss function, but only for things you can evaluate.”
Verdicts are made automatically; how verdicts are made.