Claims

Factual · Science and research methods

“Re-architected AlphaFold2 for CASP14 to reach atomic accuracy - leading the organisers to declare the problem solved!”

Demis Hassabis, Nobel lecture slides · December 2024 · Accelerating scientific discovery with AI (Nobel lecture slides) (NobelPrize.org)

AlphaFold2 was re-architected for CASP14 to reach atomic accuracy, leading the CASP organisers to declare the problem solved.

Mostly true True in substance, wrong in a detail.

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

True if the evidence shows both of these: (a) DeepMind entered a substantially new or re-architected AlphaFold system (AlphaFold2, different in architecture from the CASP13 AlphaFold) in CASP14 (2020), and it reached atomic or near-experimental accuracy (for example a median GDT around 90 or backbone accuracy at about atomic scale); and (b) the CASP organisers (for example John Moult or the CASP organising committee) stated publicly that the protein structure prediction problem had been solved, or largely solved. Mostly true if both parts hold but a detail is slightly off, for example the organisers said the problem was 'essentially' or 'largely' solved for single-chain proteins, or the accuracy was described as 'competitive with experiment' rather than 'atomic'. Mixed if only one of the two parts holds. Mostly false if AlphaFold2 performed strongly at CASP14 but the organisers made no statement that the problem was solved and the accuracy claim is overstated. False if AlphaFold2 was not a re-architected system entered at CASP14, or if it did not reach high accuracy there.

Evidence

1. CASP14 press release CASP (Critical Assessment of Techniques for Protein Structure Prediction) organisers · 30 November 2020

   “Now that the problem has been largely solved for single proteins, the way is open for development of new methods for determining the shape of protein complexes”

2. AI untangles one of biology's great challenges France 24 (AFP) · 1 December 2020

   “"In some sense the problem is solved," said John Moult, a computational biologist at the University of Maryland who co-founded the Critical Assessment of Techniques for Protein Structure Prediction (CASP) competition in 1994, quoted by the journal Nature.”

3. Highly accurate protein structure prediction with AlphaFold - Nature Nature (paper authored by DeepMind researchers) · 15 July 2021 Own record

   “The neural network AlphaFold that we developed was entered into the CASP14 assessment (May–July 2020; entered under the team name ‘AlphaFold2’ and a completely different model from our CASP13 AlphaFold system10).”

4. AlphaFold: a solution to a 50-year-old grand challenge in biology Google DeepMind · 30 November 2020 Own record

   “In the results from the 14th CASP assessment, released today, our latest AlphaFold system achieves a median score of 92.4 GDT overall across all targets.”

Reasoning

The peer-reviewed Nature paper by DeepMind (E3) states that the CASP14 entry, AlphaFold2, was a completely different model from the CASP13 AlphaFold system. DeepMind reports a median score of 92.4 GDT across all CASP14 targets (E4). On part (b), the CASP14 press release (E1) says the problem is "largely solved for single proteins", and CASP co-founder John Moult said that "in some sense the problem is solved" (E2). Both parts hold, but the organisers' statements were qualified, which fits the criteria's "mostly true" case.

Cite this claim

Demis Hassabis, “AlphaFold2 was re-architected for CASP14 to reach atomic accuracy, leading the CASP organisers to declare the problem solved.” (said December 2024). Verdict: mostly true. Reliability API, claim c_c78c8688cfd0, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_c78c8688cfd0