Pinned Tweet
🧬🔮 Single cell foundation models have been a recent hot topic in bio-ML! A few of the recent methods and some thoughts 🧬🔮
1) Geneformer
2) scGPT
3) scFoundation
4) Exceiver
🤌🏼🤌🏼
1/ Most genes and variants matter somewhere but look silent in any one cell model. Can we build conditions that reveal phenotypes? With @kanishkadey, we engineer cell states to expose hidden disease biology and trace how genetic risk reaches disease genes. biorxiv.org/content/10.64898…
Will Connell retweeted
How do lab-in-the-loop ML systems learn most efficiently?
How can you pack as much information as possible into an experiment?
We explored this in our new preprint “Information-Dense Synthesis for Molecular Discovery” (arxiv.org/abs/2610.08495)
If the goal is to simulate a cell’s gene regulatory logic I think a strong test is predicting held out time pts on a dataset of densely time-sampled targeted perts
Or a dataset of dense dose-sampled perts
We’re desperately missing measured changes over close cont intervals
My scoped takeaway is that LLMs can do similarity search "by eye," matching patterns in a way that's independent of statistical methods.
IMO this is more interesting. And this idea generalizes, we formalized it as LLM Kernel: openreview.net/forum?id=Rq5w…
Claude has discovered a previously unknown enzyme system hidden in the DNA of bacteriophages. Beside the enzyme’s gene sits a long array of repeating DNA—a structure that looks somewhat similar to CRISPR.
We don’t yet understand what this system does, but only a handful of known systems share its features, and all of them are able to cut, copy, and paste DNA. Historically, the discovery of such programmable systems has helped revolutionize medicine. CRISPR, for instance, is now the foundation of genetic medicines. But it will take much more work to learn what this system does, and whether it can be put to similar use.
Read more: anthropic.com/news/claude-di…
Imagine culling the tree of knowledge to reproducible conclusions.
IMO one of the first steps to accelerating science‼️
Eliminate entire hypothesis nodes that have misallocated scientific search for decades due to foundations of fraud, weak evidence, poor interpretation.
BixBench3 is out! Our latest eval for biology capabilities in language models. Essentially requires agents to reproduce all work needed to write a paper from raw data. OpenAI is in the lead with 5.6-Sol at 48%, Anthropic is in 4th with Opus 4.8, after Kimi and GLM. Opus 5 has weird instruction following problems that severely degrade its performance. I’m sure Fable is great but can't be evalled on this. Read more below.
Also lest we miss the forest for the trees, we’re in a position now where the models can essentially reproduce entire papers ~50% of the time. Crazy time to be alive.
very comprehensive Review on single-cell foundation models from @JiayuanDing, @Xiaojie_Qiu, @TheodorisLab, and colleagues. After going over it, I would encourage the authors to make it more critical rather than primarily an inventory of models.
First, I think Fig 1 needs some rethinking. The definition of scFM has become too loose. For example, the actual model unit of GET is celltype pseudobulk aggregated from scATAC-seq, hence not single cell model. Also thanks for highlighting our TissueNarrator, but we certainly do not consider it a bona fide scFM - it is a specialized spatial transcriptomics model built by leverging an existing foundation LLM (Qwen). TissueNarrator is very interesting and we re having lots of fun w/ it but it's not a scFM. Anyway - Fig 1 seems to mix FMs, specialized models built on FMs, and task-specific models into one big family tree.
More broadly, I think the more interesting question for a scFM review in 2026 is not how many scFMs now exist or how their architectures differ, but what has the scFM paradigm actually bought us? Since the first wave around scBERT/Geneformer/scGPT, the number of models, parameters, and pretraining cells has exploded, but as a field we are not convinced that capabilities have advanced proportionally.
- Where has large scale pretraining enabled something that a strong specialized model could not do ?
- Where do scFMs consistently beat strong task specific or even simple baselines?
- Where is the broad evidence for true cross-context generalization on biologically interesting tasks?
The Review mentions many of these challenges indeed, but I think they should be much closer to the central theme of the article rather than appearing mainly in a later Challenges section :-)
I might almost suggest to flip the review around and instead ask:
- what have we actually learned after several years and many scFMs?
- What claims have survived independent benchmarking?
- What has scaled and what has saturated?
- What capabilities are genuinely attributable to pretraining and where do we still need much more evidnece?
- What important biological problems remain essentially unsolved by scFMs ?
Given the current rate of model proliferation, there may be another 50 scFMs by this time next year and this inventory / figure will become outdated very quickly. To me, a more useful and timely "state of the field" critique would be much more valuable and long-lasting than another "model zoo" review .
Are single-cell foundation models (scFMs) a true biological breakthrough, or just over-hyped architecture tweaks?
To separate genuine conceptual advances from incremental hype, the lead authors of pioneer models, including Geneformer, Cell2sentence @david_van_dijk , Nicheformer @fabian_theis , GET @raulrabadan, CellPLM @tangjiliang , and our Tabula, have united to publish a systematic, critical synthesis of the scFM landscape.
Designed for the broader scientific community, this guide cuts through the noise to show how emerging model designs can truly support grounded biological discovery.
📄 Paper: lnkd.in/gSxJBEwt
📷 Curated paper list: lnkd.in/gU4R4Rt3
Led by the incredible @JiayuanDing
and @shiyu_jiang23 , Zhaoyu Fang, Yujie Zhang, @xutzhang , @jkobject , Weixu Wang, @alexanderfuxi , Aakash Patel, Syed Rizvi, @Y_Ryan_Lu , @SiyuHe7 , @YixinxinWang , @KejunYing , @peterpaohuang , @YifanLu2024 , @Nanguage , Mengchen Wang, Ziyang Miao, Jianhui Lin, Jimmy Ding, Jerry Wang, @imweio , @TianlongChen4 , Guoxian Yu, Min Li, Jiayi Ma, @feiwang03 , @Yuyingxie , @cmuptx , @PengHeAtlas , Emily B. Fox, @dasongle , @ericxing
Give it a read, we hope you will enjoy it!
We got 3rd place in a AI x Bio hackathon a couple of years ago tackling this problem; we released a model GenPlasmid and a dataset OpenPlasmid; more below
behindbioml.substack.com/p/b…
xitter.cf/wilstc/status/18487928…
By training an AI model using 153,208 plasmids from Addgene, Shao et al.’s PlasmidGPT can annotate and classify existing plasmids as well as generate new functional plasmid sequences from DNA “prompts”. science.org/doi/10.1126/scia…
😂 LFG, love it 🧈🧈
🧬 We are excited to announce the @NeurIPSConf 2026 workshop, I Can’t Believe It’s Not Better: Failure Modes of AI in Biology (ICBINB-BIO), to be held in Sydney, Australia, on Dec. 11 or 12, 2026!
📣 We welcome full (8 pages) and tiny (4 pages) papers that share unexpected challenges in developing, evaluating, and deploying models on biological tasks and data. Sharing these experiences helps the community avoid unproductive paths, clarify failure modes and boundary conditions, and build a culture of transparency. We will also have an exciting lineup of speakers sharing insights from the frontiers of AI and biology.
🌿 ICBINB-BIO is the official biology branch of the broader @ICBINBWorkshop initiative, united by a commitment to candid failure analysis, negative results, unexpectedly strong simple baselines, and benchmarks that genuinely support the claims made from them.
📅 Key Dates (AoE)
Submission Deadline: Aug. 29, 2026
Acceptance Notification: Sep. 29, 2026
🌐 Additional details: icbinb-bio.github.io/
📄 Submission link: openreview.net/group?id=Neur…
We also welcome you to sign up as a reviewer to evaluate submitted manuscripts and provide valuable comments and feedback as part of the selection process: forms.gle/c7bs7nLWm6s3SSkW7
We hope you can join us in Sydney to help shape a more transparent and reliable future for AI in biology! ✨
Will Connell retweeted
No scaling laws for single-cell foundation models: when bigger atlases stop teaching the model anything
In language and vision, the recipe has been simple: more data, bigger models, better performance. Single-cell biology borrowed that playbook. Foundation models for transcriptomics jumped from 1 million cells to atlases of over 100 million, on the assumption that scale would unlock the same gains. Alan DenAdel and coauthors put that assumption to the test, and the result is sobering.
Working from a 22.2-million-cell corpus, they pretrained 400 models across five architectures (from PCA and a variational autoencoder up to the Geneformer transformer) and ran 6,400 evaluation experiments. They varied not just dataset size (1% to 75%) but also diversity, using cell-type re-weighting and geometric sketching to deliberately enrich rare cell types and transcriptional states.
The finding: performance saturates almost immediately. On cell-type classification, batch integration, and perturbation prediction, most models hit their ceiling at roughly 1% of the corpus, about 200,000 cells. Beyond that, adding millions more cells changed essentially nothing. More diversity didn't help. Even spiking in genome-scale Perturb-seq data, to give the models perturbed phenotypes rather than just healthy ones, failed to move the needle. Larger models did score better overall, but they too plateaued early on data.
Two points stood out. Simple baselines (PCA, logistic regression) often matched or beat the transformers. And the strongest model, SCimilarity, won not because of size but because its contrastive training objective is aligned with the downstream task. For single-cell data, what you train on and how you frame the objective matters far more than how much you collect.
This reframes a quiet but expensive habit. In drug discovery, biotech, and any pipeline leaning on cell atlases, the instinct to keep scaling pretraining corpora may be burning compute for no return. The real leverage sits elsewhere: curating high-quality, task-relevant data and matching the training objective to the actual question you're trying to answer.
Paper: DenAdel et al., journal license | doi.org/10.1038/s41592-026-0…
There's a broader pattern here we're also finding success with @transcriptabio: provide structured data as context to elicit prior bio knowledge from an LLM.
Here, there are steps of info restructure / distill via probes but worth asking - which are useful? Are they req? 🧐
We achieved state-of-the-art performance in predicting which of 4.2 million genetic variants cause diseases by interpreting a genomics model, in a new preprint with @MayoClinic.
We're now releasing an open source database for all variants in the NIH's clinvar database. 🧵(1/8)
Will Connell retweeted
New Post: Quantitative Look at Biotech Platforms
Plenty has been written about bio platform strategy but no one's put numbers around it
We used a whole lotta tokens to compile clinical, partnership + financial data on the 100 most successful public platform biotechs of all time
Will Connell retweeted
Auto-research for ML training models is all the rage now, but underrated is: auto-research for data!
Sure, you can squeeze out a bit of model performance by optimizing hyperparameters, but code agents can do data work that has been very labour intensive and required a lot of attention to a lot details effortlessly:
> download data from many different data sources
> bring all the data sources into uniform format
> do detailed EDA: find patterns and outliers
> look at 100s of samples and take detailed notes
> make beautiful infographics rather than mpl plots
> iterate on data filtering by looking at more samples
> make a simple pipelines robust and scalable
It's now possible to write data pipelines for dozens of data sources in hours that would have taken weeks of reading many docs, debugging APIs and data formats, wrangling outliers and missing data.
A few weeks ago we gave Claude access to the CPU partition of our cluster and it iteratively refined filters to retrieve a domain subset of FineWeb. This would have taken me 2-3 days to work through while it took Claude just a few hours with almost no babysitting and with a nice logbook.
Thus the long tail of small, niche data sources becomes more accessible and can be aggregated to even larger high quality datasets for cool applications.
Data has been fuelling LLM progress more than model architecture innovations, so I am very excited about this!
Great to see this out 👏🏼 plz read for a really big idea
Also, I wrote an overview of Variational Synthesis in 2024
open.substack.com/pub/behind…
Manufacturing-aware generative models enable petascale synthesis of designed DNA go.nature.com/3NxXt1I
"We find that intra-complex interactions are largely conserved, whereas inter-complex relationships are extensively rewired, revealing new context-dependent genetic dependencies." 👏
💡rich resource for virtual cell benchmarking to disambig contextual-modeling vs coexpr-modeling
We mapped gene interactions across different environmental conditions (GxGxE) at scale for the first time in human cells. These maps lead to the realization that many genes function in a context dependent manner which provides insight into how humans have relatively few genes but many cell types. Congratulations Ben!
Paper:
cell.com/molecular-cell/full…
I built Scaling Biology 🧬 — a dashboard that live-tracks the volume and growth of key biological data sources across genomics, transcriptomics, and proteomics. The project is open to community contributions, check out the repo linked in footer wconnell.github.io/scaling-b…
Will Connell retweeted
1/13 Excited to share our (@anna_spiro @ChikinaLab @sara_mostafavi) latest preprint! 🧬💻
Personal Genome Prediction isn't just a downstream task—it’s the ultimate end-to-end benchmark for Variant Effect Prediction. We put the new SOTA AlphaGenome to the test and uncovered a striking "Modality Gap" between gene expression and chromatin accessibility. 📄
Link: biorxiv.org/content/10.64898… 🧵👇
Will Connell retweeted
Together with Emma Dann, we are thrilled to present a massive new Perturb-seq atlas of 22M primary CD4+ T cells, from 4 donors, across 3 timepoints – the result of a decade-long collaboration between the Marson (@MarsonLab) and Pritchard (@jkpritch) labs.
🧵👇
Will Connell retweeted
The recent breakthroughs from @nablabio & @chaidiscovery emphasize a split in early biotech strategy.
For the specific range of problems that antibodies address, making the binder, is becoming trivial. This forces a choice between 'fast but competitive' and 'AI intractable'. 🧵
This is a major reason I joined @transcriptabio
We've proved our platform in rare disease – an area that uniquely allows you to:
1) realize the mission of helping people, immediately
2) receive the gold-standard of clinical feedback, immediately
📄: researchsquare.com/article/r…
Will Connell retweeted
Most current drug discovery efforts is structure-based eg. create small molecules or antibodies that best binds X. However, a drug may not drive its efficacy from its strongest binder. Taking a step away from structure-paradigm, we reason that if a CRISPR knockout of a gene mimics a drug's effects across cancer cell lines, that gene is likely the drug's target. This was done in @EytanRuppin in collaboration with @anideshpandelab and @BenDavidLab
Using this principle, we integrated drug and crispr profiles from 1000s of drugs to find their context specific targets (different cancers or when known target is not expressed but drug is yet killing cancer cells).
We call this tool DeepTarget. We show that this approach outperforms current structure based methods (AF3, RF, Chai) to find drug's target in a genome-wide search, when we had no information on what the target might be. We benchmarked in eight gold-standard drug-target pairs. It took us months to get this benchmarks (we hope this benchmark helps the field)
We present two experimentally validated cases and pls see the paper for this (link at the end).
An intriguing observation is that we had many cases where we have many small molecules targeting the same gene (eg. EGFR) and we found that small molecules with higher predicted target specificity show greater clinical advancement.
Very happy to hear your feedback. Here's the free access link: nature.com/articles/s41698-0…
































