Pinned Tweet
✨Introducing Prompteina: all-atom protein design directly from natural language✨
- Control diverse protein properties through a single text prompt
- Higher predicted binding confidence than baselines
- First experimentally validated de novo binders reported for GM2A
🧵
Joseph Lee retweeted
Today, RFdiffusion3 is out in @ScienceMagazine!
science.org/doi/10.1126/scie…
We have many exciting updates to share! We're also releasing Chaperone, an agent harness for general bio-molecular design. 1/n
Joseph Lee retweeted
One n-gram! Two n-grams! 797 gigabytes of n-grams! Ah ah ah! 🧛⚡
We’ve been testing if n-gram counts in read-only CPU RAM help LMs. At small scale, others just broke nanoGPT speedrun records with a similar trick. With the COUNT we scale to 1B and match 57% more compute.
Joseph Lee retweeted
We can scale up our text-to-protein work thanks to generous support from Google.org! blog.google/company-news/out…
With @UWproteindesign, we're working on sustainability. Hopefully one day we'll be able to simply tell the model, "design an enzyme to degrade plastic"🪄
Super excited to share our year-long work on designing proteins directly from natural language, led by @jleechung in collaboration with @UWproteindesign @allen_ai!
Current protein design models are constrained in what they can condition on. As an analogy, think of an image generation model that only conditions on discrete class labels like "dog" or "cat", or "outdoors" or "indoors"; in contrast, a modern text-to-image model can be much more expressively controlled.
We do the same for protein design here, and the resulting model natively turns natural instructions into corresponding protein structures. This makes it much more effective at designing proteins with particular form/function, and it works especially well for agent-driven protein design!
Joseph Lee retweeted
We're thrilled to share that we've been selected as a recipient Google.org's Impact Challenge: AI for Science! We’ll be partnering with Dr. Pang Wei Koh to design proteins with natural language to advance sustainability applications, such as plastic degradation.
Joseph Lee retweeted
Super excited to share our year-long work on designing proteins directly from natural language, led by @jleechung in collaboration with @UWproteindesign @allen_ai!
Current protein design models are constrained in what they can condition on. As an analogy, think of an image generation model that only conditions on discrete class labels like "dog" or "cat", or "outdoors" or "indoors"; in contrast, a modern text-to-image model can be much more expressively controlled.
We do the same for protein design here, and the resulting model natively turns natural instructions into corresponding protein structures. This makes it much more effective at designing proteins with particular form/function, and it works especially well for agent-driven protein design!
Prompteina's expressive conditioning lets an agent iteratively improve designs by refining many properties, including topology, interface geometry, and size.
On the seven hardest targets, its top-8 ipSAEmin gain over iid generation is 57% larger than the next-best baseline's.
Check out the following links for more details and resources:
🌐 Website: jxlee.me/prompteina/
📄 Paper: arxiv.org/abs/2610.13917
💻 Code: github.com/jleechung/prompte…
🤗 Weights: huggingface.co/jleechung/pro…
🗂️ Data: will be released shortly!
Thanks to collaborators Yanjing Li, @boqiang1222, and David Baker at @UWProteinDesign; @greglau at @NUSingapore; and Rory Donovan-Maiye, @AllysonEttinger, and @nlpnoah at @allen_ai. And to my advisors, @HannaHajishirzi and @PangWeiKoh, for their generosity and support!
To train Prompteina, we build Prompteina-Data: 1.8M structures paired with 52.4M captions describing global and residue-specific properties.
We adapt @nvidia’s Proteina-Complexa via text-conditioned pretraining and fine-tuning, aligning residues with their corresponding text.
Beyond fold and topology, Prompteina's control extends to binder-target interfaces: across 20 targets, it reliably steers the number of residues at the interface.
Current protein design models rely on hand-designed, property-specific conditioning, limiting which objectives they can express directly.
Recent work uses agents to orchestrate these models (e.g., @AnthropicAI), but remains limited by the controls of the underlying models.
Joseph Lee retweeted
I'll talk at 2:50pm on "Context, Creativity, and Control" -- feat. @RulinShao’s Context Language Models; @scottgeng00’s Gacha Decoding; and designing proteins from natural language by @jleechung. Come hear about our brilliant students' latest work! 🧵
I’ll be at COLM this Friday for our LM4Sci workshop: lm4sci.github.io/
Come hang out and listen to great talks by:
- @nc_frey (Anthropic)
- Patrick Beukema (Ai2 / Stealth)
- @MengdiWang10 (Princeton)
- @hengjinlp (UIUC)
- Jeremy Avigad (CMU)
- @PangWeiKoh (UW / Microsoft)
Joseph Lee retweeted
Introducing Gacha Decoding 🎲: we show how to diversify LM responses, beating prior work by >11x in sample efficiency.
Reroll your LM for varied ideas, data, RL envs, {your use here}!
We get diversity from instruction following + external RNG, not token entropy. Here's how 🧵
Joseph Lee retweeted
‼️The Bitter Lesson for context management: Giving LMs unrestricted control over their context beats human-designed SOTA!
Introducing 🩵Context Language Models (CLMs)🩵
- Natively manage their own context
- Treat context as a file
- Learn policies in CLM weights, no harness
Joseph Lee retweeted
Introducing ⚓ 𝗔𝗻𝗰𝗵𝗼𝗿𝗲𝗱 𝗗𝗲𝗰𝗼𝗱𝗶𝗻𝗴: a copyright mitigation strategy for any language model! With @uwnlp
LMs today reproduce copyrighted text—raising concerns for creator consent and potential legal (and 💸 💸) liabilities for AI developers. 🫠
𝗔𝗻𝗰𝗵𝗼𝗿𝗲𝗱 𝗗𝗲𝗰𝗼𝗱𝗶𝗻𝗴 relies on two off-the-shelf LMs:
🧼A 𝘀𝗮𝗳𝗲 𝗟𝗠 trained only on permissively licensed text,
⚠️A higher-utility 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 trained on any data.
The 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 drives generation, but the 𝘀𝗮𝗳𝗲 𝗟𝗠 acts as an anchor. If the 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 drifts into memorization, the 𝘀𝗮𝗳𝗲 𝗟𝗠 pulls it back ↩️.
🤝We provide a formal guarantee: outputs stays within a user-set budget of the 𝘀𝗮𝗳𝗲 𝗟𝗠.
Details below! 👇
[1/⚓]
ALT Example generation with Anchored Decoding