Using interpretability to understand, design, and align AI.

San Francisco
Joined August 2024
“I think our job is to sort of try to think like the neural network.” @nickcammarata's interview with @techreview today on why he feels hopeful that neural networks can be understood:
2
7
82
4,772
We identified a gene variant behind hearing loss, a big step for interpretability in medicine. In April, our interpretability system EVEE scored all 4.2M variants in ClinVar, the NIH's variant database. MEPE c.209_212del scored particularly high, so we built a case and had it successfully reclassified as "likely pathogenic." ~2M variants are still "uncertain," but we believe interpretability can help us understand many more!
8
25
10
207
18,584
We discovered a new class of Alzheimer’s biomarkers with @PrimaMente. Today we're taking the next step by opening a global AI competition to discover new treatments. Introducing the Alzheimer’s Translation Challenge, led by @PrimaMente and @AlzData. 🧵
3
16
2
133
8,190
.@farairesearch red-teamed our monitor with a static battery of attacks. In this testing, our monitors cut successful universal jailbreaks to 0, and reduced total jailbroken interactions by 97%.
2
1
21
1,211
Compared to an LLM judge alone, our monitor: - Pareto dominates on recall/FPR - uses 50x less compute, costing <$200 to monitor 1M exchanges - cuts added latency to virtually zero
1
4
1
32
2,455
We built cybersecurity monitors for Kimi K3 and GLM 5.3 — more accurate, 50x faster, & 50x cheaper than an optimized LLM judge. External red-teaming from @farairesearch found that our monitor greatly reduces universal jailbreaks. 🧵
11
42
15
314
26,994
Manifold steering, SAE geometry, and a model that ignores its own clock…our researchers are at @COLM_conf this week to present papers on all three. Meet them at our booth, Tue-Thu, 1:30-2:30pm. Poster times and happy hour RSVP below 👇
1
14
4
93
7,785
Many biosecurity safeguards are oversensitive to requests that aren’t actually harmful, constraining useful biology research. Our method leverages protein model embeddings for more precise and robust real-time screening, at millisecond speeds.
1
1
20
1,022
Biosecurity is the next frontier of AI security. We built SOTA monitors so agents can do more biology, safely. Our monitors outperform frontier model safeguards with fewer refusals on dual-use tasks. They’re fast, real-time, and robust to adversarial attacks. 🧵
7
23
11
138
33,696
Training a good SAE is harder than it looks: choosing layers, collecting activations at scale, tuning width and sparsity, labeling millions of features… Our blog post walks through these choices. Or just describe your goal to Silico, our interp agent, and it handles the rest.
1
28
1,847