A Structural Antibody Benchmark of AlphaFold3 reveals Hallucinated Epitopes and a Bias for Orderness 1 They benchmarked AlphaFold3 (AF3) for antibody–antigen binding on a large, structure-grounded dataset: 3401 experimentally validated complexes from SAbDab plus 23798 negative controls (random human proteins and six fixed “many-to-one” decoys) to quantify both true hits and false positives. 2 Using AF3 confidence at the interface (PAE and IPTM) as a binary “bind / not bind” classifier (PAE < 2.5 or IPTM > 0.7), AF3 achieved recall ~0.46 with precision ~0.64 (F1 ~0.53) on 10-seed inference; an intrinsic false positive rate of ~3% appeared consistently across negative-control batches. 3 A key qualitative failure mode: AF3 can hallucinate plausible antibody–target interfaces on decoy proteins, distributing predicted epitopes across much of a folded target’s surface. These false-positive “hotspots” suggest AF3 can generate convincing but non-biological interfaces even when no binding should exist. 4 Another notable bias: AF3 performed better on complexes whose experimental structures came from X-ray crystallography than from electron microscopy. In their split, X-ray-derived positives had higher recall (~0.52) than EM-derived positives (~0.30), consistent with training-set composition and/or higher flexibility/disorder typical of EM-captured systems. 5 Target size matters: positive prediction rates decreased as antigen length, surface area, and volume increased (i.e., larger search space → fewer detected binders). This trend was visible across both positive and negative controls, and was strongest when using surface area/volume rather than sequence length alone. 6 Disorder interacts with epitope discovery in a specific way: while overall target disorder fraction did not strongly separate predicted positives vs negatives, AF3 tended to avoid disordered regions when hallucinating interfaces (e.g., on PD-L1/CD274 decoys, false-positive antibodies clustered on ordered domains rather than the disordered terminus). This supports the paper’s framing of a bias toward “orderness.” 7 They tested whether performance was inflated by training data leakage (PDB overlap with AF3 training set). Although many benchmark antibodies were “leaked,” leaked vs non-leaked antibodies showed no meaningful difference in AF3 confidence distributions, suggesting leakage was not the dominant driver of outcomes in this benchmark. 8 Antibody-sequence features were surprisingly not major confounders: amino-acid composition across CDRs and CDR lengths showed little to no correlation with AF3 interface confidence (PAE), arguing AF3 is not simply favoring certain residues or longer/more flexible loops when it predicts binding. 9 Epitope-centric analysis added nuance beyond RMSD: using DockQ plus two custom measures (epitope shift and antibody displacement), they found many “false negatives” still landed near the correct epitope. About 34% of false negatives had epitope shift < 10 Å despite poor DockQ, implying that downstream refinement (e.g., MD, docking, more recycling/relaxation) might rescue additional true interactions. 10 Increasing inference seeds improves recall but is costly: rerunning 1915 false negatives at 100 seeds recovered 344 additional true positives, raising recall to ~53%. They also note AF3’s internal model selection by ranking score may not always pick the lowest-PAE interface, suggesting alternative selection heuristics could help in screening workflows. 📜Paper: biorxiv.org/content/10.64898… #AlphaFold3 #Antibodies #ProteinStructure #ComputationalBiology #DrugDiscovery #Benchmarking #ProteinProteinInteractions #Bioinformatics #StructuralBiology #MachineLearning