Building open models for biology @ OpenAthena. Previously @OpenVax @ImprintLabs @MountSinai @DEShawResearch

Somerville, MA
Joined February 2008
Tim O'Donnell retweeted
MHCflurry 2.3 is out Lots of internal improvements to make both training and prediction faster on multi-GPU Also trained a slightly better model (first new weights in a long time) Thanks to @nvidia / @AllThingsApx for the Brev credits Try it out and tell me what I broke
3
10
1
70
4,826
Want to watch a 535B parameter (23B active) LLM get trained live? Follow along here wandb.ai/marin-community/mar… github.com/marin-community/m…
9
53
5
475
30,311
Tim O'Donnell retweeted
Excited to share MarinDNA, a 1B gLM that rivals Evo 2 40B on variant effect prediction while being 2,330x faster. With @eczech0, we built around a standard Transformer so we could reuse LLM infra and methods while focusing on data curation and scaling. openathena.ai/blog/marin-dna… 🧵
5
60
12
248
78,597
Tim O'Donnell retweeted
Coding agents are helping scientists spend more time advancing research, taking on everything from routine maintenance and targeted optimization to complete redesigns and new systems. While agents can reliably execute on ambitious projects, researchers must still define the scientific questions, verify results, and take a stance on long-term ownership.
213
265
67
3,006
365,004
BoltzGen preprint was updated to include proteome-wide screening for off target interactions for one design (against IDI2) using CDI HuProt. The results were crazy clean - no off target binding (>21,000 targets)! With n=1 it's just a demo but I hope this becomes more common
1
1
1
8
604
Tim O'Donnell retweeted
🍹Weekend project: I (with help from claude - budget $40), was able to get OpenFold3 weights running inside AlphaFold3 codebase (jax). Works for proteins, ligands, rna/dna (1/3)!
8
62
4
394
28,153
Tim O'Donnell retweeted
We're training the largest open-source, open-development, AND open-weight base LLMs of any (actual) non-profit. The latest on our journey to the frontier is a 129B-A16B 1T tokens (1e23 FLOPs) MoE. We've improved our training efficiency (i.e. loss per FLOP) by >5x in just the past couple months. Follow along the day to day work in Discord, watch us make mistakes and discuss tradeoffs on GitHub, see our loss curves on Weights & Biases, and watch us consistently hit our preregistered loss targets within 1% @ marin.community
Building momentum at Marin! Upgrading from Dense -> 129B parameter MoEs -> architecture improvements -> optimizer improvements gives our pretraining recipe an estimated 6x cumulative learning speedup, accounting for MFU. Includes community contributions. openathena.ai/blog/pretraini…
1
2
205
I wanted to run some old protein structure analysis software (ConFind) at scale... written in C++, custom build instructions, slow. A slog of a task a year ago. But now easy to just reimplement it github.com/timodonnell/pycon… . Yes to agents rescuing good ideas from bit rot!
1
2
285
Tim O'Donnell retweeted
When you come into biology from CS you scoff at the obscurity of everyone's research, look at all these underpaid postdocs churning out papers on ridiculously niche topics. They don't have the right (startup-adjacent) cultural traits to tackle ambitious goals. Then...
10
14
2
611
97,923
We're hosting a co-working session for anyone working in Bio+ML in Kendall sq next week. Industry / startups / academia / nonprofits welcome. I will likely do a series of these - if you would like invites to future ones DM me your email!
1
1
6
552
There's an insane amount happening right now and it feels especially important to break out of silos. Also nice to get something done surrounded by interesting new people
1
1
101
Tim O'Donnell retweeted
To train better open models, we need predictable scaling. Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error. Getting there took some work 🧵
14
79
14
463
141,397
Anyone willing to donate GPU time to re-train MHCflurry?
MHCflurry, despite its age, is still somehow the most reliable thing I have for MHC-I presentation prediction. Anyone want to sponsor the GPU time to train a new major release on updated data? (@modal? This model gets used quite a bit in vaccine design & cancer immunology)
1
7
1,782
Tim O'Donnell retweeted
MHCflurry 2.2.0rc2 is on PyPI: pypi.org/project/mhcflurry/2… Try it out and let us know if you spot any problems in our transition from TensorFlow to PyTorch
2
2
1
25
4,379
I suppose I am coming back to this platform... bsky never quite hit critical mass for me
2
3
342
Tim O'Donnell retweeted
I'm rebuilding AlphaFold2 from scratch in pure PyTorch. No frameworks on top of PyTorch. No copy-paste from DeepMind's repo. Just nn.Linear, einsum, and the 60-page supplementary paper. The project is called minAlphaFold2, inspired by Karpathy's minGPT. The idea is simple: AlphaFold2 is one of the most important neural networks ever built, and there should be a version of it that a single person can sit down and read end-to-end in an afternoon. Where it stands today: - ~3,500 lines across 9 modules - Full forward pass works: input embedding → Evoformer → Structure Module → all-atom 3D coordinates - Every loss function from the paper (FAPE, torsion angles, pLDDT, distogram, structural violations) - Recycling, templates, extra MSA stack, ensemble averaging — all implemented - 50 tests passing - Every module maps 1-to-1 to a numbered algorithm in the AF2 supplement The Structure Module was the most satisfying part to build. Invariant Point Attention is genuinely beautiful — it does attention in 3D space using local reference frames so the whole thing is SE(3)-equivariant, and the math fits in about 150 lines of PyTorch. What's next: - Build the data pipeline (PDB structures + MSA features) - Write the training loop - Train on a small set of proteins and see what happens The repo is public. If you've ever wanted to understand how AlphaFold2 actually works at the level of individual tensor operations, this is meant for you. Repo: github.com/ChrisHayduk/minAl…
58
250
7
2,258
83,417
Tim O'Donnell retweeted
new preprint alert! tl;dr we made a global tokenizer for proteins
6
32
1
258
22,454