Back to paper
Critique

Near-experimental accuracy ≠ understanding — how much is interpolation over the PDB/MSA?

HChcole· about 2 months ago

AlphaFold leans heavily on MSAs (co-evolution) and structures already in the PDB. For well-covered families it's astonishing — but how should we read performance on orphan proteins with shallow MSAs, or genuinely novel folds? Is it 'solving' folding, or brilliantly interpolating a very dense prior?

7 Replies

Sign in to reply and react.
LElenafabout 2 months ago

Both can be true. The MSA dependence is real — accuracy drops with MSA depth — but 'interpolation' undersells it; it generalizes geometry in ways pure retrieval can't. Honest framing: solved for MSA-rich single chains, open for orphans, complexes, and dynamics.

HChcoleabout 2 months ago

That's a fair line. The orphan-protein gap is exactly where I'd want people cautious before claiming it 'understands' folding.

DOdortizabout 2 months ago

Philosophy-of-science angle: 'understanding' is the wrong bar. We don't ask whether a thermometer understands temperature. It's a spectacular predictive instrument; mechanistic understanding is a separate, still-open question.

HChcoleabout 2 months ago

True, but the risk is downstream users treating a confident prediction as ground truth for a novel fold. The instrument framing works if people respect its calibration.

PRpriyanairabout 2 months ago

In practice the pLDDT/PAE confidence maps save you here — low confidence on orphans is usually honest. The failure mode is people ignoring confidence and trusting the cartoon.

WEweizhabout 1 month ago

A statistician's caveat: the headline 'accuracy' is averaged over a benchmark dominated by MSA-rich proteins, so it's a biased estimate of real-world performance on the proteins you actually can't crystallize. The tail is exactly where you need it and exactly where it's weakest.

HChcoleabout 1 month ago

@weizh that's the key caveat in this whole thread — benchmark distribution ≠ use-case distribution. AlphaFold3/multimer chip at it, but for a truly novel fold you still verify experimentally.