Little progress was made in CASP protein structure prediction accuracy for a decade before AlphaFold.
Mostly true True in substance, wrong in a detail.
Factual: Statements about the present or the past, checked against the record: did they hold up?
What would count as true or false
Judge against published CASP results (predictioncenter.org and CASP assessment papers) for roughly CASP7–CASP12 (about 2006–2016), before AlphaFold's entry in CASP13 (2018). True: top GDT_TS scores on free-modelling/hard targets were largely flat over about a decade. Mostly true: progress was small and slow, with modest gains (e.g. some improvement around CASP11–12 from contact prediction) that the claim's rough wording understates. Mixed: the sources disagree substantially, or real progress came in a substantial part of that decade. Mostly false: CASP accuracy rose substantially over that decade before AlphaFold. False: the records show large, steady gains throughout.
Set before the research began.
Evidence (4)
-
E1 Critical Assessment of Methods of Protein Structure Prediction (CASP) – Progress and New directions in Round XI Proteins (CASP11 assessment review; NIHPA Author Manuscript) · 1 June 2016
“free modeling – FM, now often effective for small proteins, but largely stuck for the last decade until new contact prediction methods partially came to the rescue this round”
-
E2 AlphaFold @ CASP13: “What just happened?” moalquraishi.wordpress.com (Mohammed AlQuraishi, Harvard Medical School) · 9 December 2018
“Historically progress in CASP has ebbed and flowed, with a ten year period of almost absolute stagnation, finally broken by the advances seen at CASP11 and 12, which were substantial.”
-
E3 Critical Assessment of Methods of Protein Structure Prediction (CASP) – Progress and New directions in Round XI Proteins (CASP11 assessment review; NIHPA Author Manuscript) · 1 June 2016
“Although overall model accuracy has not improved much when the increased size of sequence and structure databases is taken into account (Figures 5 and 6), examination of improvement in accuracy without correcting for database changes shows impressive improvement (Figure 7).”
-
E4 Critical Assessment of Methods of Protein Structure Prediction (CASP) – Progress and New directions in Round XI Proteins (CASP11 assessment review; NIHPA Author Manuscript) · 1 June 2016
“Results here also suggest no substantial progress in overall model accuracy.”
Reasoning
The CASP11 assessment review describes free-modelling accuracy as "largely stuck for the last decade" until new contact prediction methods partly helped in CASP11 (E1). It also reports no substantial progress in overall model accuracy (E4), and little improvement once database growth is accounted for, though without that correction it reports impressive improvement (E3). An independent academic account likewise describes about ten years of near-stagnation, but says it was broken by substantial advances at CASP11 and CASP12 before AlphaFold entered (E2). The claim therefore broadly holds, but its wording leaves out the real gains made around CASP11–12, which matches the "mostly true" criterion.
Cite this claim
Demis Hassabis, “Little progress was made in CASP protein structure prediction accuracy for a decade before AlphaFold.” (said December 2024). Verdict: mostly true. Reliability API, claim c_1192e90f1487, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_1192e90f1487
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Demis Hassabis
-
Mostly true
“what’s important about them is their 3D structure.”
A protein's 3D structure is what determines, or partially determines, its function.
-
Resolved true
“So over 3 million scientists are now using AlphaFold.”
-
Resolved true
“Only about 10% of drugs actually get through all the clinical stages.”
Only about 10% of drugs get through all the clinical stages.
-
Mostly true
“The AlphaFold Server is empowering non-commercial researchers globally to harness this technology, accelerating their ability to formulate and test new hypotheses.”
-
Mostly true
“Our AlphaProteo model can design novel, high-strength protein binders that target diverse molecules – including those associated with cancer and diabetes.”
Verdicts are made automatically; how verdicts are made.