Building AI for drug discovery @LeashBio
Salt Lake City, UT
Joined June 2012
- Tweets114
- Following554
- Followers137
- Likes1K
Max Kleinsasser retweeted
Agentic models can act, observe consequences, and decide what to do next.
This is great for coding, but if you want them to do science, lab experiments take forever.
To fix this, we made a lab-in-the-loop with zero latency!
1/10
Max Kleinsasser retweeted
Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.
This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:
- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.
This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
github.com/karpathy/nanochat…
All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.
And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.
Max Kleinsasser retweeted
Current protein models seem to only memorize their wild-type training sets and lack physics understanding.
One nice way forward is physically engineering mutations and then taking more ground-truth measurements.
It works. We've already shown this! (1/n)
Adversarial Sequence Mutations in AlphaFold and ESMFold Reveal Nonphysical Structural Invariance, Confidence Failures, and Concerns for Protein Design
1. A new adversarial study systematically evaluates AlphaFold 3's robustness by introducing point mutations (up to 70%) and deletions (up to 10%) across 200 proteins, revealing striking structural invariance that raises fundamental questions about the model's biophysical reasoning capabilities.
2. The most concerning finding: AlphaFold 3 maintains virtually identical predicted structures even when 40% of residues are mutated with deliberately destabilizing substitutions, or when 10% of residues are deleted—perturbations that would catastrophically destabilize real proteins.
3. This structural invariance persists even for experimentally validated fold-switching proteins, where specific mutations are known to induce alternative conformations. AlphaFold 3 fails to capture these biologically critical transitions, suggesting limited sequence-structure coupling.
4. Confidence metrics prove unreliable: AlphaFold 3's ranking score selects the most accurate structure only ~25% of the time, and these scores correlate more strongly with template availability in the training set than with actual prediction quality.
5. Comparative analysis with ESMFold reveals that the protein language model-based approach shows significantly greater sensitivity to mutations, with structures diverging more rapidly as sequence perturbations increase—suggesting superior learned sequence-structure relationships despite lower absolute accuracy.
6. The study's template analysis provides quantitative evidence that AlphaFold 3's confidence reflects structural similarity to training-set exemplars (Pearson r=0.39) rather than genuine biophysical assessment, indicating heavy reliance on memorized patterns over learned principles.
7. These findings have profound implications for the entire AlphaFold ecosystem: protein design tools like RFdiffusion, binder design methods like BoltzGen and BindCraft, and drug discovery pipelines may inherit these fundamental limitations, potentially generating non-physical sequences or missing viable candidates.
8. The work identifies critical gaps in current structure prediction—models trained primarily on stable, wild-type proteins lack exposure to destabilized mutants and misfolded states, limiting their ability to generalize beyond the training distribution.
📜Paper: biorxiv.org/content/10.64898…
#AlphaFold #AlphaFold3 #ProteinStructurePrediction #StructuralBiology #ProteinDesign #MachineLearning #Bioinformatics #ComputationalBiology #AIforScience #ProteinEngineering #DeepLearning #Biophysics
Max Kleinsasser retweeted
Hermes: Large DEL Datasets Train Generalizable Protein-Ligand Binding Prediction Models
1. A new study introduces Hermes, a lightweight transformer trained exclusively on DNA-encoded chemical library (DEL) screening data from hundreds of protein targets—representing one of the largest and most protein-diverse DEL training sets for protein-ligand interaction modeling to date.
2. The key innovation lies in demonstrating that DEL data alone, despite being binary enrichment scores rather than traditional affinity measurements, can produce transferable representations that generalize to held-out protein targets, novel chemical scaffolds, and external benchmarks derived from public binding data.
3. Hermes achieves remarkable inference speeds of 28.2 samples/second/GPU, representing a 500-700× speedup over state-of-the-art structure-based models like Boltz-2, making it practical for large-scale virtual screening campaigns.
4. The model employs a minimal architecture combining pre-trained ESM2 protein embeddings with ChemBERTa ligand representations, connected through joint cross-attention modules—proving that sophisticated inductive biases are not always necessary for effective PLI prediction.
5. Evaluation across four benchmarks reveals strong performance on kinase targets, consistent with the kinase-enriched training data, while also showing modest generalization to non-kinase proteins and entirely different assay formats including high-throughput screening data.
6. The work highlights critical challenges in PLI model evaluation, including data leakage risks with structure-based models and the importance of rigorous benchmark construction, while establishing DEL-derived data as a viable path toward more consistent, scalable training resources for drug discovery.
📜Paper: arxiv.org/abs/2602.13503
#DrugDiscovery #MachineLearning #ProteinLigandBinding #DEL #DNAEncodedLibraries #VirtualScreening #ComputationalChemistry #Bioinformatics #AlphaFold #ChemBERTa
Max Kleinsasser retweeted
Yes, and especially in BIO, the right data is not just an advantage, it is everything.
It has been challenging to do FM model partnerships with pharma, because they have no use for models trained on your bespoke experiment.
However, everyone values FMs trained on human tumors from the clinic to improve their oncology clinical pipeline.
If I were a16z, yc, or sequoia, I’d be aggressively investing in startups that are building novel ways to collect and annotate real-world data.
> Billions of hours of driving data
> Factory workers interacting with appliances and heavy machinery
> Audio segmentation with deep dialectical and cultural understanding
> Wet-lab experimental data
> Continuous collection and annotation of agent traces at compute scale
When we built LLMs, most of the data already existed on the internet. We just had to scrape, clean, and scale. But as we move toward world foundation models, the bottleneck is high-quality, real-world, well-annotated data.
And annotation quality matters. There’s a massive difference between:
“Apple on a tree”
and
“Ripe apples on a tree. The wind is blowing at 2 miles per hour. The temperature is around 18°C.”
The question is simple. How much of the world can you actually capture?
Today, LLMs know that apples fall because of gravity, not because they understand causality, but because they understand language correlations extremely well. Understanding the causal structure comes next.
If I were building towards that future, I’d anchor data collection in India and other South and Southeast Asian regions. I’d deploy hardware, collect thousands of hours of human activity data, health signals, and vitals, and run annotation pipelines continuously. Day and night.
If I were a16z, I’d fund founders to do this.
I might just have the urge to do it myself.
Max Kleinsasser retweeted
if you think about it, this is actually a very bad thing.
now each of us is a moderator of our own private StackOverflow. Claude asks the questions. we provide answers. accessible only to Anthropic, who may or may not choose to include in the next training run
bad bad bad
Max Kleinsasser retweeted
“Show me the mechanism of action.”
“Uh. from the p-p-perturb seq?
right here. Knocks down Gene X, and the cells shift into Cluster 7."
“You used fkn UMAP again. FUCK. Zoom in.”
“Sorry?”
“Zoom. The fuck. In.”
“…Okay.”
“Do you see it?”
“…See what?”
“He doesn’t see it. The cell biology postdoc from Stanford does not know how to see a cell. He stares at a two-million-cell embedding and doesn’t see the conflations he just planted in my S3 bucket.
WHERE is the temporal resolution?”
“The… what?”
“Where is the time? You hit the cell with a perturbation and you measured it once. Once. That’s not a mechanism. That’s a fucking POSTCARD!”
“We sequenced at 72 hours. That’s standard.”
“Standard is not the same as correct. You are aliasing causality. You compressed a dynamic process into a static endpoint and you’re think you will cure metastatic cancer.”
“But the differential expression is significant.”
“WOW! DIFFEWENTIAL EXPRESSION IS SIGNIFICANT! WOW! WHEN THE FUCK IS IT NOT.
Yes, because statistics is Mario Kart.
A fantasy land where causality is optional and variance disappears if you collect enough cells. Real biology has inertia.
Feedback.
Competing pathways.
Do you understand the difference between correlation and mechanism? Or did you flunk out of STAT 101?”
“…I mean, we saw Gene X regulate Pathway Y.”
“No. You saw Pathway Y exist in the same cell after you kicked it down the stairs and waited three days. That’s not even a crime scene, you dimwit. That is a post-mortem autopsy.”
“The model inferred a trajectory--”
“STOP the buffoonery. Do not blame the model.
The model is a mirror.
In this particular case, you can see it mirrors the clusterfuck you just created in my biosafety cabinet.
If the reflection is warped, it’s because your measurement is warped.”
“Pull up the raw counts.”
“…Okay.”
“Scroll. Cell 14,982. Read it to me. What does it say?”
“Uh… mitochondrial genes up, ribosomal genes down-”
“And?”
“…And stress response markers?”
“Yes. Because you poisoned the cell and waited long enough for it to panic.
Where is the early signaling?
Where is the metabolic inflection?
Where is the first irreversible decision?”
“We don’t capture that.”
“Exactly. You built a platform that cannot see the mechanism. YOUR PLATFORM IS FUCKING BLIND.”
“The virtual cell...showed the perturbation effect cleanly.”
“Yes. because your VIRTUAL CELL IS VIRTUAL. it assumes the cell is a bag of transcripts. Do you think metabolites show up? OR AN ISOFORM? AN ISOFORM, THAT WILL DEGRADE IN THE CELL BEFORE YOU EVER REACH YOUR SENSOR? THAT ONE? YOU THINK THAT'S HOW CELLS WORK?”
“So what do you want me to do?”
“Delete the atlas.”
“What?”
“Delete it. The whole thing.”
“But that’s the core result.”
“It’s three weeks of garbage narrative built on a blind instrument. Delete it.”
“…Now what?”
“Now you rebuild the measurement. You observe the cell while the perturbation propagates. You track chemistry, not just transcripts. You capture the first divergence, not the final corpse.”
“That’s… not perturb-seq anymore.”
“Exactly.”
door slams
Max Kleinsasser retweeted
AI for small molecules is really hard, @leashbio is an example of how to do it right. A really innovative and daring data generation story. Must read and congrats to @allmeasures and @Andrewdblevins for building such a cool platform.
An ML drug discovery startup trying really, really hard to not cheat
owlposting.com/p/an-ml-drug-…
on the 12-person, Utah-based startup @leashbio, their culture of rigor, and the many ways small molecule models accidentally learn the wrong thing
Max Kleinsasser retweeted
An ML drug discovery startup trying really, really hard to not cheat
owlposting.com/p/an-ml-drug-…
on the 12-person, Utah-based startup @leashbio, their culture of rigor, and the many ways small molecule models accidentally learn the wrong thing
Max Kleinsasser retweeted
This is one of the craziest ideas I've ever seen. He converted a drawing of a bird into a spectrogram (PNG -> Soundwave) then played it to a Starling who sung it back reproducing the PNG.
Using the birds brain as a hard drive with 2mbps read write speed.
youtube.com/watch?si=HMtVdHB…
Max Kleinsasser retweeted
Today OpenAI announced o3, its next-gen reasoning model. We've worked with OpenAI to test it on ARC-AGI, and we believe it represents a significant breakthrough in getting AI to adapt to novel tasks.
It scores 75.7% on the semi-private eval in low-compute mode (for $20 per task in compute ) and 87.5% in high-compute mode (thousands of $ per task). It's very expensive, but it's not just brute -- these capabilities are new territory and they demand serious scientific attention.
Max Kleinsasser retweeted
LLMs are useful, but they are an off ramp on the road to human-level AI.
If you are a PhD student, don't work on LLMs.
Try to discover methods that would lift the limitations of LLMs.
Max Kleinsasser retweeted
Academia cares whether an idea is new. It doesn't really have to work
Industry only cares if an idea works. Doesn't matter if it's new
This creates a gap. Actually a few gaps:
Max Kleinsasser retweeted
Graph of decision quality among professional Go players. A sudden increase after AlphaGo. It is not only because they are learning from the AI. Players are suddenly inventing new moves at a faster rate too!
Max Kleinsasser retweeted
The heretofore silent majority of AI scientists and engineers who
- do not believe in AI extinction scenarios or
- believe we have agency in making AI powerful, reliable, and safe and
- think the best way to do so is through open source AI platforms
NEED TO SPEAK UP !
Max Kleinsasser retweeted
Do language models have an internal world model? A sense of time? At multiple spatiotemporal scales?
In a new paper with @tegmark we provide evidence that they do by finding a literal map of the world inside the activations of Llama-2!
Max Kleinsasser retweeted
A neural network can smell like humans do for the first time!👃🏽
Digital smell is a modality that AI community has long ignored, but maybe one day useful for robot chef 👩🏽🍳? Here's how to do smell2text:
1. Collected 5,000 molecules and ask humans to label "creamy, chocolate, alcoholic, beefy, spicy, citrus", etc. This dataset is one of its kind and a huge contribution from the paper.
2. Train a graph neural network (GNN) to map the molecule to label. Each molecule is a graph of atoms described by valence, degree, hydrogen count, hybridization, formal charge, atomic number, etc.
3. The GNN predictions match well with expert humans on novel smells.
4. The embeddings give us a "Principal Odor map (POM)" that faithfully represents hierarchies and distances among odorants.
