Pinned Tweet
We release our protein chatGPT, Evola! 🌟 chat-protein.com/
Evola comes in two versions: 10B & 80B. The 80B model has a 1.3B Saprot encoder & a 70B LLaMA3 decoder.
Trained on 546 protein question-text pairs with an 150 billion word tokens! 💡🔬
biorxiv.org/content/10.1101/…
This Post is from an account that no longer exists. Learn more
fajie yuan retweeted
I also remember Pinal doing this almost a year ago: github.com/westlake-repl/Den…
Designing Protein Structures From Natural Language
1 Prompteina is introduced as the first text-conditioned all-atom protein structure generation model that directly generates both sequence and all-atom 3D structure from natural-language prompts, aiming to replace property-by-property engineered conditioning inputs with a single language interface.
2 A key enabling contribution is Prompteina-Data: ~1.8M protein-chain structure entries (from PDB, AlphaFold DB, and synthetic dimers) paired with 52.4M LLM-written captions describing global and residue-specific properties (average ~29 captions per structure), spanning up to 59 annotation types per entry.
3 The captions are not just high-level function blurbs: they include fine-grained, residue-linked constraints (e.g., disulfide bonds, catalytic sites, protein–protein contacts). The dataset also includes phrase-to-residue correspondences when possible, enabling supervised alignment between text spans and residue subsets.
4 Methodologically, Prompteina adapts a pretrained all-atom generator (Complexa) with two text-conditioning pathways: pooled text embeddings broadcast for global properties, and token-level cross-attention for residue-specific control. Both are injected into the denoiser to steer generation throughout sampling (rather than only at inference-time guidance).
5 A supervised text–residue alignment objective is added during fine-tuning: alignment annotations define a sparse residue-by-token matrix, and the model learns which residues correspond to which text spans, improving control over features like disulfide and salt-bridge counts while preserving global fold control.
6 On text-guided structural control across 18 tasks in five property groups (fold/topology, shape, secondary, tertiary, and combined constraints), Prompteina leads baselines on 15/18 tasks and supports the broadest range of properties through one interface; it also shows robustness to paraphrasing and maintains designability/novelty metrics relative to baselines.
7 For functional prompts across 122 protein families, evaluation with ProTrek (sequence/structure/text similarity) shows Prompteina achieves the highest mean similarity, including when weighting by predicted fold confidence (pLDDT), compared with sequence-only text-to-protein methods that must be folded post hoc.
8 In binder design, Prompteina can directly steer interface size via natural language: across 20 targets, requested vs realized interface residue counts track nearly linearly (Spearman ρ ~0.994–1.000), enabling explicit tuning of binder–target interface extent.
9 In agentic binder design (same LLM agent writing specs tailored to each model’s interface), Prompteina yields the highest interface-confidence metric (expected top-k ipSAEmin) across 20 targets and multiple sequence conditions (native, full ProteinMPNN redesign, and fixed-interface redesign). Ablations suggest the largest gains come from being able to specify interface properties (size/geometry/chemistry) in the prompt.
10 Prospective wet-lab validation targets GM2A (a disease-relevant protein with no previously reported de novo binders): from 89 expressed and SPR-assayed designs, nine were classified as binders, including a strongest apparent KD of 107 nM (with sub-micromolar binders spanning diverse architectures).
💻Code: github.com/uw-ssec/promptein…
📜Paper: arxiv.org/abs/2610.13917
#ProteinDesign #GenerativeModels #DiffusionModels #ProteinStructure #TextToStructure #ComputationalBiology #Bioinformatics #MachineLearning
fajie yuan retweeted
After the *best* 3-year review/revise saga, the final version of EvoDiff is finally out at @eLife!
I'm still as excited about protein sequence models as ever, and extremely proud of the team that came together for this.
EvoDiff combines evolutionary-scale data with diffusion models for controllable protein sequence generation.
In addition to generating plausible proteins, we can scaffold structural motifs in sequence space!
Preprint: biorxiv.org/cgi/content/shor…
Code: github.com/microsoft/evodiff
fajie yuan retweeted
Antibodies remain a particularly difficult challenge even for protein language models — a point that may diverge from the more optimistic views held by many working on PLMs. In light of this (and consistent with the limitations highlighted here), I believe the work of aligning specialized protein models with LLMs is especially meaningful: it can genuinely serve as an interface between humans and sequences. Current PLMs focus heavily on representation and understanding on the protein side, yet they are still quite cumbersome to use in practice. Several groups have already done excellent work aligning protein representations with natural language (e.g., ProTrek and Pinal). Going further, natively incorporating carefully curated and corrected protein sequence–structure–function–name pairs alongside natural language during the training of GPT-5-level models would be truly interesting.
EpiBench: Can LLMs Understand Epitopes for Antibody Drug Discovery?
1. The paper introduces EpiBench, a closed-book, sequence-only benchmark designed to test whether general-purpose LLMs can infer epitope information directly from antigen and antibody sequences, in a way that mirrors real antibody drug discovery decisions rather than isolated prediction tasks.
2. EpiBench is built around a connected 5-stage workflow: (T1) targetable region discovery, (T2) antibody-conditioned epitope identification, (T3) epitope binning, (T4) functional epitope assessment, and (T5) antibody escape assessment—covering discovery, characterization, and resistance risk.
3. The dataset contains 1,609 curated, automatically scorable samples grounded in experimental evidence from multiple sources: structural antibody–antigen contacts (AsEP, SAbDab), curated functional B-cell assays (IEDB), and deep mutational scanning escape measurements (DMS).
4. A key design choice is de-identification for “closed-book” evaluation: prompts remove PDB IDs, antibody/antigen names, accessions, and paper metadata, keeping only sequence-level evidence (e.g., VH/VL sequences and CDRs, numbered antigen residues, candidate epitope residue sets, and mutation descriptions).
5. The benchmark also includes shortcut-control strategies to reduce artifacts: balanced multiple-choice answer letters (Tasks 2/4), balanced binary labels (Tasks 3/5), candidate order randomization, antigen/antibody clustering to reduce redundancy, similarity-matched positive/negative pairs for binning, and mutation-level matching for escape tasks.
6. Nine general-purpose LLMs are evaluated zero-shot under a unified protocol (with explicit intermediate reasoning allowed). Results suggest models capture partial epitope-related signals but struggle with antibody-specific grounding, residue-level localization in long contexts, and biologically grounded functional/escape reasoning.
7. Task-level patterns: models show some ability to prioritize broad epitope-like regions in T1 (RegionRecall@50 improves over random), but AUROC stays near random, indicating weak residue-wise discrimination. T2 and T5 remain challenging because they require conditioning on a specific antibody to localize binding or predict mutation-specific escape.
8. Antigen length sensitivity is a major bottleneck in region discovery: average RegR@50 drops sharply as antigens get longer (reported from ~81 for <200 aa to ~13 for >800 aa), highlighting difficulties with long-context residue indexing and sparse-signal retrieval from large sequences.
9. Explicit reasoning (CoT) provides inconsistent gains: it helps some models notably on T1 (organizing open-ended residue selection) but does not reliably improve antibody-conditioned, functional, or escape decisions (T2–T5). Error-trace analysis shows frequent reliance on indirect proxies (motif fixation, generic physicochemical heuristics, similarity shortcuts, and “safe” guessing), plus concrete failures like coordinate misalignment in mutation mapping.
📜Paper: arxiv.org/abs/2608.06022
#AntibodyDiscovery #Epitopes #LLM #BioNLP #ComputationalBiology #ProteinScience #Benchmark #DrugDiscovery #AIforScience
fajie yuan retweeted
Raygun 1, our model for redesigning proteins the way evolution does, is out today in @Nature. For an arbitrary protein, it can:
- miniaturize (eGFP, shorter than any natural FP)
- magnify (EGF, into a tighter EGFR binder)
- modify (dozens of edits at once-- indels and subs)
fajie yuan retweeted
ProtBLIP2-SST: Protein Function Prediction via BLIP2 with Sequence, Structure, and Text
1 ProtBLIP2-SST reframes protein function prediction as open-ended Swiss-Prot-style caption generation (FUNCTION + SUBCELLULAR LOCATION + SIMILARITY) instead of rigid GO multi-label classification, enabling a single free-text output that can jointly express mechanism, localization, and homology context.
2 The key design choice is early, unified sequence–structure fusion using SaProt: residues are tokenized together with Foldseek-derived 3Di structural tokens in a structure-aware vocabulary, so sequence and 3D information interact from the first representation layer (not via separate encoders or late fusion).
3 The framework adapts BLIP-2 to proteins with a two-stage pipeline: Stage 1 aligns protein features to text space using a lightweight Q-Former (8 learnable query tokens) while keeping large unimodal backbones frozen; Stage 2 conditions a scientific LLM to generate captions from the aligned protein embeddings.
4 Stage 1 uses a frozen SaProt protein encoder and a frozen BiomedBERT text encoder, training only the Q-Former via three objectives: protein–text contrastive learning (InfoNCE), protein–text matching with hard negatives, and protein captioning with a causal mask that forces generation to flow through the query bottleneck.
5 Stage 2 connects Q-Former outputs to Galactica-1.3B by linearly projecting the 8 query embeddings into the LLM embedding space and prepending them to the prompt; only parameter-efficient adapters are trained (LoRA on attention and FFN layers, r=8), while base PLM/LLM weights remain frozen.
6 Data: 441k Swiss-Prot protein–text pairs were filtered to require AlphaFold DB structures, yielding 410,925 train / 9,236 val / 9,239 test triplets; sequences are truncated to 1024 residues and captions to 128 tokens.
7 Retrieval (Stage 1): ProtBLIP2-SST improves over sequence-only ProtT3 at matched scales on Swiss-Prot test retrieval after PTM re-ranking over 128 candidates; the 650M variant reports the best overall average (90.83) with test Acc 69.28 (P→T) and 68.70 (T→P), highlighting gains from explicit structural integration.
8 Captioning (Stage 2): ProtBLIP2-SST achieves consistent improvements over ProtT3 on BLEU/ROUGE/METEOR; at 650M it reaches EM 33.07 with B-4 58.53, R-L 68.23, METEOR 69.54, while the 35M model also improves over ProtT3(35M) (e.g., B-4 55.37 vs 54.74).
9 Analysis and takeaways: field-stratified comparisons show structure helps most for FUNCTION text (fold/active-site constraints), while SIMILARITY labels benefit less (often motif-driven). The authors also note a decoder bias: Galactica tends to start outputs with FUNCTION even when the reference begins with other fields, suggesting future work on field-balanced decoding and improved alignment objectives.
📜Paper: biorxiv.org/content/10.64898…
#ProteinFunctionPrediction #ProteinLanguageModels #MultimodalLearning #ProteinStructure #AlphaFold #SwissProt #BLIP2 #LLM #Bioinformatics #ComputationalBiology
fajie yuan retweeted
Why bigger is not better in protein language models
Scaling laws are close to an article of faith in deep learning, so protein language models are an awkward case. ESM2-650M and 3B beat the 15B model at predicting mutation effects. xTrimoPGLM-3B beats the 10B and 100B. ProGen3-1B beats the 46B. Chao Hou and coauthors explain why, and the mechanism is more interesting than a data-scarcity story.
Start with what these models actually output. A protein language model reads sequences the way a text model reads sentences, and it can score how plausible a given protein looks under everything it has seen. That score, p(sequence), is its estimate of how likely that exact chain of amino acids is. Mutation effects then come almost for free: swap one residue, rescore, and the drop in likelihood stands in for how damaging the mutation is.
The trouble is that this likelihood absorbs factors unrelated to fitness. The authors show that homolog counts in the training set, model size, and even the random seed all shift it. And because probabilities at each site must sum to one, a bigger model that pushes the wild-type residue toward certainty necessarily drives every mutation score toward extreme negative values. A tiny model does the mirror image and collapses everything toward zero. Both extremes are uninformative, and neither reproduces the bimodal spread seen in experiments.
Across 154 ProteinGYM assays, performance versus wild-type likelihood traces a clean bell curve, peaking near a likelihood of about 0.3, and it holds for sequence-only, structure-hybrid, autoregressive and inverse-folding models alike. Inside that optimal band, model size stops mattering. Family-specific models such as EVE show no bell curve at all. General models track evolutionary conservation best at intermediate likelihoods, and it is that alignment, not parameter count, that predicts performance.
For teams in variant interpretation, antibody engineering or enzyme optimization, this reframes model selection. The question is not which model is largest but whether it has learned your protein family at all, and the paper offers a cheap check: correlate its per-residue likelihoods against a simple alignment-frequency model, with about 0.5 as a working cutoff, before committing a wet-lab campaign.
Paper: Hou et al., Nature Computational Science (2026), journal license | doi.org/10.1038/s43588-026-0…
fajie yuan retweeted
Very excited to share ProtGPT3! After ProtGPT2, we release open Hugging Face models up to 10B, with MSA prompting, DPO alignment, and experimentally expressed, soluble defluorinase designs. Congratulations to @infinitycrab2, @LasseMiddendorf, @Filippo_Stocco_, and the rest of the team. Read the details below :)
🧬 Introducing ProtGPT3, a new open-source family of protein language models, from 112M to 10B parameters, now live on Hugging Face! @Filippo_Stocco_ @LasseMiddendorf @ferruz_noelia
Thread 🧵👇
fajie yuan retweeted
A bit of news: After nearly 9 years, I have decided to leave Google DeepMind and join Anthropic (after taking some time to recharge). I am incredibly grateful for my time at GDM. @demishassabis took a real chance letting me lead the AlphaFold team just six months after finishing my PhD, and the entire GDM team taught me so much about how to do great science. GDM is a special place, and I’ll still be excited to hear about what amazing things they discover next.
fajie yuan retweeted
Amazing work, based on the data I vibe coded a little game that let's you check your own stability prediction intuition:
Can you beat SaProtΔG/ESM3ΔG or at least the NPSA baseline?
moritzertelt.github.io/Stabl…
🚀 Excited to share our new work: Absolute Stability Predictor!
📊: forms.gle/4ZnXZSnTBvaykkAi9
Built the MGnify Stability Dataset (1.8M+ measurements) and developed stability prediction models, together with @grocklin, @KotaroTsuboyama, @sokrypton, and teams.
fajie yuan retweeted
New binder scoring metric just dropped
Absolute stability prediction with seq-based (ESM3dG) and structure based (SaProtdG) PLMs
Adding stability prediction boosts ipSAE performance in discriminating binders from non-binders
Congrats @ChoYehlin !
🚀 Excited to share our new work: Absolute Stability Predictor!
📊: forms.gle/4ZnXZSnTBvaykkAi9
Built the MGnify Stability Dataset (1.8M+ measurements) and developed stability prediction models, together with @grocklin, @KotaroTsuboyama, @sokrypton, and teams.
fajie yuan retweeted
Learn which structural domains are compatible with each other, then generate protein sequences conditioned on the desired structural domains.
@duguyuan
Congrats to the amazing team! 🎉🧬
Want to fine-tune ESMC? Try the Colab version here:
👉 colab.research.google.com/gi…
More ColabPLMs (ESM1b, ESM2, ESM3, SaProt, ProTrek, ProtBert & more) available at OPMC:
🔗 theopmc.github.io/
Today we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology.
The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics.
We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity.
We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures.
ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences.
A world model of protein biology emerges through language modeling.
We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins.
The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science.
This understanding emerges without prior knowledge, just from language modeling of protein sequences.
Language models are becoming a powerful substrate to understand and program biology.
The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders.
I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.
fajie yuan retweeted
I'm so excited to show the world what we've been working on the for the past months!! I'm going to highlight some of the fun results from this paper that I find particularly exciting.
Today we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology.
The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics.
We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity.
We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures.
ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences.
A world model of protein biology emerges through language modeling.
We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins.
The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science.
This understanding emerges without prior knowledge, just from language modeling of protein sequences.
Language models are becoming a powerful substrate to understand and program biology.
The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders.
I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.
fajie yuan retweeted
We fine-tuned sequence- and structure-pretrained models on our large-scale stability data to predict absolute folding stability. The resulting SaProtΔG and ESM3ΔG achieved Spearman r = 0.88 and 0.87, with RMSE = 0.80 kcal mol⁻¹.
fajie yuan retweeted
🚀 Excited to share our new work: Absolute Stability Predictor!
📊: forms.gle/4ZnXZSnTBvaykkAi9
Built the MGnify Stability Dataset (1.8M+ measurements) and developed stability prediction models, together with @grocklin, @KotaroTsuboyama, @sokrypton, and teams.
fajie yuan retweeted
A growing protein cage has no architect.
Each piece only responds to its local environment.
Yet somehow, thousands of pieces can come together to form a large, precise sphere.
Today in @Nature, two back-to-back papers from our team and @UWproteindesign show how de novo protein design can make this kind of self-assembly programmable.
The trick: we didn't design perfect symmetry.
We designed the rules for breaking it. [1/17]
rdcu.be/fjNgF
nature.com/articles/s41586-0…
fajie yuan retweeted
proteins aren’t static structures. they constantly sample rare higher-energy states that shape function, aggregation, and disease. this paper maps those hidden conformational fluctuations across thousands of protein domains at scale, opening the door to predicting protein energy landscapes instead of just native structures.
nature.com/articles/s41586-0…
fajie yuan retweeted
Does AlphaFold’s latent space encode only the native state or something like a distribution over conformations? We begin to answer this question with ConforNets, a mechanism for producing diverse states, or very specific ones, via inference-time adaption of OF3p’s latent space👇
fajie yuan retweeted
I find this funny-sad. On July 25 GigaRef was introduced as the largest protein seq DB to date with 3.34B seqs. But on Sep 1, the LOGAN DB was released with >100B seqs. This means the title of the largest seq DB only lasted ~1month. Meanwhile the DeepClust DB was never released☠️
An update on this figure. I'm really excited about the recent results from LOGAN, the largest database of protein sequences 🔥
Logan: Planetary-Scale Genome Assembly Surveys Life's Diversity
biorxiv.org/content/10.1101/…
fajie yuan retweeted
Excited to co-host #AIxBio 2026 (8–11 June) at @wellcomegenome / @sangerinstitute in Cambridge, UK!
Join an amazing speaker lineup alongside co-hosts , @ferruz_noelia, Jun Cheng, Jussi Taipale @BenLehner, @deboramarks,
The conference will explore five themes:
🔬 Solving the gene regulatory code
🧪 Solving proteins — data, structure and design
💊 Solving chemistry and therapeutics
🫀 Solving cells, tissues and organs
⚙️ AI methods development
Abstract submissions are open — deadline 27 April 2026.
Selected abstracts will be featured in the main programme alongside invited talks.
Register and submit your abstract here:
shorturl.at/ZlvzP
#AIxBio #AI #Biology #MachineLearning #Genomics







































