
Cofounder, @LeashBio.
Salt Lake City, UT
Joined May 2010
- Tweets7.1K
- Following2.7K
- Followers1.3K
- Likes21.2K
Addendum: a brief taxonomy of agent failure modes
Ian Quigley retweeted
Since the beginning of Leash, the main battle we have fought is generalization vs memorization. It is too easy for models to memorize the patterns that lead to binding in this particular instance, but fail to generalize the next time you use the model.
Last week we released Minos, our framework of premeasured experimental worlds for agentic science.
Minos is not for obtaining the best scientific result, it's for understanding how AI agents make scientific decisions and maybe help them make better ones.
Let's dig in!
1/
We dropped ~4500 transcripts of agents working on these tasks for you to dig through yourself at strelka.leash.bio. We hope folks find a more complete taxonomy of failure that we can all use to improve agentic science.
11/
Ian Quigley retweeted
Pretty clever idea from @allmeasures and @Andrewdblevins at @leashbio to use a dense combinatorial set of experimental data like a DEL screen as an environment to test how well agents can optimize in chemical space.
Medchem optimization in the real world isn't so constrained, but it's plausible to me that ability at the DEL task would transfer to the real world.
I wonder where else the idea can be applied to build RL environments for science.
Blog post: leashbio.substack.com/p/maki…
Preprint: github.com/Leash-Labs/minos-…
The preprint nicely addresses the biggest question I had from reading just the blog post by performing a "shuffle" ablation suggesting that it was measuring actual chemical reasoning ability instead of just "Mastermind" ability.
I would also be interested in seeing some other baselines beyond random, like a fingerprint-based optimizer.
Ian Quigley retweeted
5/ These gels are super soft (stiffest ~330 Pa, 1000x+ below earlier yeast squeezing work), like mucus or tissue. Physical constraint alone slows division, with no detectable change in gene or protein expression.
Ian Quigley retweeted
2/ Sreepadmanabh, Gautam and colleagues grew budding yeast in transparent microgel matrices of tunable stiffness to watch single cells bud in 3D. More confinement = slower budding, for daughters too. Pull them back out into liquid and they snap right back to normal speed.
Ian Quigley retweeted
1/ We discovered something totally unexpected about the neural structure of consciousness that overturns decades of thinking on its biological basis and brings us significantly closer to the recipe for generating conscious awareness.
New preprint: biorxiv.org/content/10.64898… (led by @HesseJanis, with @FrankLanfranchi, @_conor_mcgrory, @Yuelin_Shi, and @EngelTatiana)
Phil's onto something here, but it is trickier than you might think!
AI native biotech companies with wet labs dedicated to data generation have a straightforward path to becoming some of the most interesting RL sandboxes.
Check out this cool whitepaper from @leashbio @Andrewdblevins @allmeasures and colleagues.
Idea goes basically like this:
1. Synthesize a combinatorial library that enumerates a chemical space.
2. Pre-measure binding enrichment for all the combinations for a large number of protein targets.
3. Give Claudes the chemical components of the library and let them try to put them together to get binders with a limited budget and learn from each attempt.
github.com/Leash-Labs/minos-…











