Building open models for biology @ OpenAthena. Previously @OpenVax @ImprintLabs @MountSinai @DEShawResearch

Somerville, MA
Joined February 2008
Tim O'Donnell retweeted
MHCflurry 2.3 is out Lots of internal improvements to make both training and prediction faster on multi-GPU Also trained a slightly better model (first new weights in a long time) Thanks to @nvidia / @AllThingsApx for the Brev credits Try it out and tell me what I broke
3
10
1
70
4,826
Tim O'Donnell retweeted
Excited to share MarinDNA, a 1B gLM that rivals Evo 2 40B on variant effect prediction while being 2,330x faster. With @eczech0, we built around a standard Transformer so we could reuse LLM infra and methods while focusing on data curation and scaling. openathena.ai/blog/marin-dna… 🧵
5
60
12
248
78,597
Tim O'Donnell retweeted
OpenAI
@OpenAIJul 28
Coding agents are helping scientists spend more time advancing research, taking on everything from routine maintenance and targeted optimization to complete redesigns and new systems. While agents can reliably execute on ambitious projects, researchers must still define the scientific questions, verify results, and take a stance on long-term ownership.
213
265
67
3,005
365,011
Tim O'Donnell retweeted
🍹Weekend project: I (with help from claude - budget $40), was able to get OpenFold3 weights running inside AlphaFold3 codebase (jax). Works for proteins, ligands, rna/dna (1/3)!
8
62
4
394
28,154
Tim O'Donnell retweeted
We're training the largest open-source, open-development, AND open-weight base LLMs of any (actual) non-profit. The latest on our journey to the frontier is a 129B-A16B 1T tokens (1e23 FLOPs) MoE. We've improved our training efficiency (i.e. loss per FLOP) by >5x in just the past couple months. Follow along the day to day work in Discord, watch us make mistakes and discuss tradeoffs on GitHub, see our loss curves on Weights & Biases, and watch us consistently hit our preregistered loss targets within 1% @ marin.community
Building momentum at Marin! Upgrading from Dense -> 129B parameter MoEs -> architecture improvements -> optimizer improvements gives our pretraining recipe an estimated 6x cumulative learning speedup, accounting for MFU. Includes community contributions. openathena.ai/blog/pretraini…
1
2
205
Tim O'Donnell retweeted
When you come into biology from CS you scoff at the obscurity of everyone's research, look at all these underpaid postdocs churning out papers on ridiculously niche topics. They don't have the right (startup-adjacent) cultural traits to tackle ambitious goals. Then...
10
14
2
612
97,923
Tim O'Donnell retweeted
To train better open models, we need predictable scaling. Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error. Getting there took some work 🧵
14
79
14
463
141,397
Tim O'Donnell retweeted
MHCflurry 2.2.0rc2 is on PyPI: pypi.org/project/mhcflurry/2… Try it out and let us know if you spot any problems in our transition from TensorFlow to PyTorch
2
2
1
25
4,379
Tim O'Donnell retweeted
I'm rebuilding AlphaFold2 from scratch in pure PyTorch. No frameworks on top of PyTorch. No copy-paste from DeepMind's repo. Just nn.Linear, einsum, and the 60-page supplementary paper. The project is called minAlphaFold2, inspired by Karpathy's minGPT. The idea is simple: AlphaFold2 is one of the most important neural networks ever built, and there should be a version of it that a single person can sit down and read end-to-end in an afternoon. Where it stands today: - ~3,500 lines across 9 modules - Full forward pass works: input embedding → Evoformer → Structure Module → all-atom 3D coordinates - Every loss function from the paper (FAPE, torsion angles, pLDDT, distogram, structural violations) - Recycling, templates, extra MSA stack, ensemble averaging — all implemented - 50 tests passing - Every module maps 1-to-1 to a numbered algorithm in the AF2 supplement The Structure Module was the most satisfying part to build. Invariant Point Attention is genuinely beautiful — it does attention in 3D space using local reference frames so the whole thing is SE(3)-equivariant, and the math fits in about 150 lines of PyTorch. What's next: - Build the data pipeline (PDB structures + MSA features) - Write the training loop - Train on a small set of proteins and see what happens The repo is public. If you've ever wanted to understand how AlphaFold2 actually works at the level of individual tensor operations, this is meant for you. Repo: github.com/ChrisHayduk/minAl…
58
250
7
2,258
83,417
Tim O'Donnell retweeted
new preprint alert! tl;dr we made a global tokenizer for proteins
6
32
1
258
22,454
Tim O'Donnell retweeted
Re-posting Nick Boyd's Mosaic blogpost after his incredible results in the Nipah G competition. blog.escalante.bio/180-lines…
Huge congratulations to Nick Boyd with Mosaic that absolutely killed in the competition! 𝑩𝒊𝒏𝒅𝑪𝒓𝒂𝒇𝒕2 did also pretty well with the second highest hit rate in the competition!
2
13
43
6,404
Tim O'Donnell retweeted
🚀 Just released: Protein Hunter on GitHub! github.com/yehlincho/Protein… Now supports Boltz and Chai with more models coming soon! Use it to: 1️⃣ Design binders from scratch 2️⃣ Optimize your own designs 🔗 Boltz: shorturl.at/0s5Ih 🔗 Chai: shorturl.at/NqrHZ
3
55
1
258
14,356
OpenFold3-preview (OF3p) is out: a sneak peek of our AF3-based structure prediction model. Our aim for OF3 is full AF3-parity for every modality. We now believe we have a clear path towards this goal and are releasing OF3p to enable building in the OF3 ecosystem. More👇
3
73
7
237
36,131
Tim O'Donnell retweeted
I’ve been testing BoltzGen a bit recently and while I haven’t done any experimental testing yet, the quality of the software is very clear. It installs, runs, logs everything, has tons of options. Very excited to test out the designs irl!
Excited to release BoltzGen which brings SOTA folding performance to binder design! The best part of this project has been collaborating with many leading biologists who tested BoltzGen at an unprecedented scale, showing success on many novel targets and pushing its limits! 🧵..
2
7
83
8,117
Tim O'Donnell retweeted
Excited to release BoltzGen which brings SOTA folding performance to binder design! The best part of this project has been collaborating with many leading biologists who tested BoltzGen at an unprecedented scale, showing success on many novel targets and pushing its limits! 🧵..
19
267
41
999
308,150
Tim O'Donnell retweeted
🧵1/ We @tahoe_ai just published a new post on the Tahoe blog—a story of how we used Tahoe-100M, the world’s largest drug-perturbed single-cell dataset, to find compounds that upregulate MHC-I and make tumors more visible to the immune system. Here’s how 🧬🔍👇
5
41
6
216
25,753
Tim O'Donnell retweeted
Protein templates are finally supported in Boltz-2! Next up: updating BoltzDesign1 to use Boltz2 for template-based motif scaffolding and more
Excited to unveil Boltz-2, our new model capable not only of predicting structures but also binding affinities! Boltz-2 is the first AI model to approach the performance of FEP simulations while being more than 1000x faster! All open-sourced under MIT license! A thread… 🤗🚀
1
22
165
9,892
Tim O'Donnell retweeted
Excited to unveil Boltz-2, our new model capable not only of predicting structures but also binding affinities! Boltz-2 is the first AI model to approach the performance of FEP simulations while being more than 1000x faster! All open-sourced under MIT license! A thread… 🤗🚀
47
404
48
1,579
312,791
Tim O'Donnell retweeted
A shared neoantigen vaccine combined with immune checkpoint blockade for advanced metastatic solid tumors: phase 1 trial interim results nature.com/articles/s41591-0… @gritstonebio therapeutic vaccine data in KRAS mutant solid tumors. The overall response rate was 0%! @NatureMedicine
7
26
5
95
28,086
We’re really excited to announce that we’re releasing code for running RFdiffusion! The code is released under an open source license and is free for anyone to use.
Today we're making RF Diffusion, our guided diffusion model for protein design with potential applications in medicine, vaccines & advanced materials, free to use. The software has proven much faster and more capable than prior protein design tools. bakerlab.org/2023/03/30/rf-d…
9
95
3
353
48,222