ClaimsEliezer Yudkowsky

“AlphaFold2 was apparently also a trendline-breaking sudden jump in capability.”

Eliezer Yudkowsky, posting on X 12 December 2020 Post by @ESYudkowsky on X, 2020-12-12 (X)

Factual AI capabilities

AlphaFold2 was apparently a trendline-breaking sudden jump in capability on protein structure prediction.

Resolved true The record shows it happened or was so.

Factual: Statements about the present or the past, checked against the record: did they hold up?

What would count as true or false

Judge as of 2020-12-12. True: CASP14 results show AlphaFold2 well above the historical trend of CASP performance (e.g. median GDT around 92 versus earlier progress), widely described as a discontinuous jump. Mostly true: a large jump that some analysts see as an acceleration partly anticipated by AlphaFold1, but still far above trend. Mixed: experts are seriously split on whether the gain broke the trend. Mostly false: the gain was large but roughly in line with the extrapolated trend after AlphaFold1. False: the CASP data show no unusual jump.

Set before the research began.

Evidence (4)

  1. E1 Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XIV Proteins (author manuscript, PMC), by Kryshtafovych, Schwede, Topf, Fidelis, Moult · 27 March 2025

    “Astonishingly, the trend curve for CASP14 (the black straight-line) starts at a GDT_TS of about 95, and finishes at about 85 for difficult targets.”
  2. E2 Critical Assessment of Methods of Protein Structure Prediction (CASP) – Round XIV Proteins (author manuscript, PMC), by Kryshtafovych, Schwede, Topf, Fidelis, Moult · 27 March 2025

    “In CASP13 (2018)18, with the introduction of effective deep learning methods, the trend line rose to above 60 on the GDT_TS scale, even for the most difficult targets, a major advance from the previous CASP.”
  3. E3 Before and after AlphaFold2: An overview of protein structure prediction PMC (mini-review journal article) · 28 February 2023

    “Presented at CASP14 between May and July 2020, AlphaFold2 predicted protein structures with more accuracy than other competing methods, demonstrating a root-mean-square deviation (RMSD) among prediction and experimental backbone structures of 0.8Å versus the 2.8Å from the next best performing method.”
  4. E4 Home Protein Structure Prediction Center (CASP, UC Davis, NIH/NIGMS-sponsored)

    “After the success of AlphaFold2 in CASP14 (2020), it was expected that deep learning methodology that brought monomeric modeling to qualitatively new level will be extended to multimeric modeling.”

Reasoning

The CASP14 assessment calls the result "astonishing": the CASP14 trend curve runs from a GDT_TS of about 95 down to about 85 for difficult targets (E1). At CASP13 the trend line had risen only to just above 60 for the hardest targets, and that was already called a major advance (E2). An independent review reports a backbone RMSD of 0.8Å for AlphaFold2, against 2.8Å for the next-best method (E3). The CASP organisers say AlphaFold2 brought monomeric modelling to a "qualitatively new level" (E4). Together these show a jump far above the earlier trend, and none of the evidence presents it as an expected extension of AlphaFold1.

Cite this claim

Eliezer Yudkowsky, “AlphaFold2 was apparently a trendline-breaking sudden jump in capability on protein structure prediction.” (said 12 December 2020). Verdict: resolved true. Reliability API, claim c_b6bfe09c187d, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_b6bfe09c187d

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Eliezer Yudkowsky, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Eliezer Yudkowsky

  1. Resolved true Factual September 2025
    “We learned that that's false, but we did not learn that you can have a thing that can redesign a tree and build an AI much smarter than itself, that builds an AI much smarter than itself, and that…”

    No AI has yet been shown to redesign a tree and build a much smarter AI that builds a still smarter AI; that scenario has not occurred.

  2. Mostly true Factual 25 March 2024
    “I was a passionate noticer of the human brain not being homogenous, and was therefore unpleasantly surprised by transformers scaling as well as they did.”

    Yudkowsky was unpleasantly surprised by how well transformers scaled, having believed the human brain is not homogeneous.

  3. Mostly true Factual 6 April 2023
    “First of all, you realize in principle that the task of minimizing losses on predicting human text does not stop when you’re as smart as a human, right?”

    In principle, the task of minimizing loss on predicting human text does not stop being improvable once a system is as smart as a human.

  4. Mostly true Factual 30 March 2023
    “Transformers are the main thing like that.”

    Transformers were the main architectural change that gave a non-linear jump in performance, compared with earlier approaches such as RNNs.

  5. Mostly true Factual 30 March 2023
    “Because the basic paradigm of machine learning that we are presently operating under is that you can have the loss function, but only for things you can evaluate.”

    The basic paradigm of machine learning currently in use is that a loss function can be defined only for things that can be evaluated.

All of them

Verdicts are made automatically; how verdicts are made.