🌀 Tertiary structure: 215 diverse 3D structures from recent PDB entries. NuFold leads monomers (0.393 TM-score), AlphaFold3 dominates complexes (0.381 TM-score). Non-Watson-Crick interactions remain a major challenge for all methods 6/9

🔗 Secondary structure: 901k chemical mapping profiles using DMS & 2A3 reactivity. EternaFold achieves top performance (0.656 F1-score), closely followed by CONTRAfold & Vienna. Traditional thermodynamic methods are still competitive with newer deep learning approaches 5/9

🔬 Fitness prediction: 70 assays across tRNA, ribozymes, aptamers & mRNAs (1M+ mutations). Evo 2 performs best overall, but performance varies dramatically by RNA type: RNA-FM excels at tRNA/aptamers while Evo 2 leads mRNA tasks. Lots of room for improvement across the board! 4/9

🚨 New paper 🚨 RNA modeling just got its own Gym! 🏋️ Introducing RNAGym, large-scale benchmarks for RNA fitness and structure prediction. 🧵 1/9

👑New model topping the ProteinGym leaderboard👑 Congrats to the SaProt team! @duguyuan @LTEnjoy Exciting to see the progress happening at the intersection of protein sequence & structure modeling.

Work with an amazing team @_nathanrollins @yaringal (@OATML_Oxford) @sandercbio @deboramarks (@DeboraMarksLab)

Several metrics geared towards fitness prediction or protein design. Different segmentation variables to analyze the performance of models in different settings: function type, MSA depths, taxa, etc.

We now also have a clinical benchmark for both substitutions and indels in humans with over 66k mutants across 3k genes based on human annotation from experts from ClinVar & gnomAD

🚨The new version of ProteinGym is out!🚨 biorxiv.org/content/10.1101/… #NeurIPS2023 #GenBio #Proteins #Benchmark #MAVE

Lastly, attention maps recapitulate certain known dependencies between labels and residues -- here, the highest attention score in red corresponds to a substrate binding site for the DFHR protein -- and may help identify new residue dependencies for functions of interest