Minta Lu retweeted
1/9 Today, alongside the team at @phylo_bio, we are sharing our first step toward having AI build better AI benchmarks using privileged information—and there is much more to come 👀
Meet BenchGuard: an automated auditing framework for scientific agent benchmarks. 🔬👇
Minta Lu retweeted
9/9 A special thanks to the team at @phylo_bio for the core support and vision in advancing scientific AI evaluation. 🧬
This work was made possible through an incredible collaboration. Grateful to all our co-authors and institute: @TzWangCompbio, @mintaylu, @KexinHuang5, @YuanhaoQ, @sara_mostafavi, @genentech, and @uwcse @phylo_bio
Minta Lu retweeted
Announcing Biomni — the first general-purpose biomedical AI agent. Biomni is a free web platform where biomedical scientists can immediately delegate their tasks to Biomni, starting today!
Biomni automates literature reviews, hypothesis generation, protocol design, bioinformatics analysis, clinical reasoning, and much more — scaling biomedical expertise for 100× the number of discoveries.
Key results:
➡️ Designed a cloning experiment with real-world wet-lab validation; on par with 5+ year expert in a blind test
➡️ Ran 458-file wearable bioinformatics analysis in 35 minutes vs. 3 weeks (800x faster) for human expert
➡️ Uncovered novel hypothesis: new TFs regulating skeletal lineages on a large scRNA+scATAC data
➡️ Human-level performance on LAB-bench DbQA and SeqQA, with SOTA at Humanity’s Last Exam and across 8 new biomedical tasks—ranging from GWAS and rare disease diagnosis to microbiology and drug repurposingPowered by:
➡️ Biomni-E1 – the first unified environment designed for a biomedical agent—encompassing 150 tools, 59 databases, 106 software—systematically curated from 2,500+ bioRxiv papers
➡️ Biomni-A1 – a generalist agent with retrieval, planning, and code as action
Biomni is an open-source initiative: we invite the community to build on it and advance biomedical research at scale.
- Try it now: biomni.stanford.edu
- Paper: biomni.stanford.edu/paper.pd…
- Code: github.com/snap-stanford/bio…
- Join the community: tinyurl.com/biomni-slackWith
Amazing team and collaborators @StanfordAILab @StanfordMed @StanfordCancer @genentech @arcinstitute @UCSF @UW @PrincetonAInews
@KexinHuang5 @serena2z @hcwww_ @YuanhaoQ @mintaylu @yusufroohani @RyanLi0802 @LinQiu0128 Gavin Junze Di Shruti Jennefer Xin Zhou @MWheelerMD Jon Bernstein @MengdiWang10 @PengHeAtlas @SnyderShot @lecong Aviv Regev