interp @ harvard, goodfire
Joined February 2017
- Tweets34
- Following211
- Followers394
- Likes375
Usha Bhalla retweeted
For years, neural networks looked inscrutable, like tangled knots that needed to be unwound. We no longer think that's the case. AI models are far more interpretable than we thought. We think this is great news for interpretability.
Goodfire's @nickcammarata co-authored Zoom In, the paper that introduced polysemanticity and superposition. He spent months revisiting the vision model organisms that much of interpretability grew out of and studying modern language models up to trillions of parameters. His new essay explains what the early field, him included, got wrong, why there's reason for optimism today, and why there might be a bitter lesson for interpretability.
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Usha Bhalla retweeted
We built cybersecurity monitors for Kimi K3 and GLM 5.3 — more accurate, 50x faster, & 50x cheaper than an optimized LLM judge.
External red-teaming from @farairesearch found that our monitor greatly reduces universal jailbreaks. 🧵
come meet us at our posters, the booth, or happy hour and learn more about goodfire, interpretability, and scaling!!
while our posters are on small models i promise we work at the trillion scale too 😉
Manifold steering, SAE geometry, and a model that ignores its own clock…our researchers are at @COLM_conf this week to present papers on all three.
Meet them at our booth, Tue-Thu, 1:30-2:30pm. Poster times and happy hour RSVP below 👇
Usha Bhalla retweeted
Coming to COLM? We’re hosting a happy hour on Wednesday (10/7) for interpretability, alignment, and life sciences researchers. Come meet members of the Goodfire team over vinyl and drinks!
luma.com/timy2nh8
deepseek while it's reward hacking in the most egregious ways possible: hehe just joking 🤪😘
big! model! interp!!
really excited about white-box methods for frontier-scale models!!
Usha Bhalla retweeted
If you are an academic/non-profit working on interpretability, we would love to support your research. Interp still needs rigorous, foundational scientific work if it is to have the impact that it promises
We are giving out grants for this kind of work!
We’re giving out $1M in grants of free Silico usage for academic and nonprofit researchers focused on AI interpretability and alignment.
We feel extreme urgency about advancing interpretability for alignment, and we want to help more researchers push it forward. 🧵
interp should be full stack! you have to smell your data before you can fix your model
i truly had so much fun with this project! there can be a lot of nasty things hiding in internet text and training data -- we've been working on finding them and fixing them at goodfire!
Usha Bhalla retweeted
At CVPR this week for a talk on neural geometry of large vision models. If you’re interested in interpretability or joining @GoodfireAI, come say hi. 🤠
🧵HOW speaker spotlight @CVPR !
Next up we have @thomas_fel_ from @GoodfireAI 🔥
Thomas will talk "Neural Geometry in Large Vision Models", diving into the structure hidden inside vision models.
📅 June 4 @ Room 1Ef | 10:30–11:00 AM
Usha Bhalla retweeted
Super excited to have this paper finally out! So many nuggets here, but a critical highlight: you should *not* interpret SAE features in isolation. The population geometry is where it's all at! Similar to this image of us @GoodfireAI folks playing out the elephant parable. :P
Usha Bhalla retweeted
How do SAEs capture concept manifolds? 🍩
I think this is important work.
we study how SAEs handle the geometric structures we've identified and find they tile/shatter them in a particular way we characterize, letting us recast unsupervised manifold discovery as inverse Ising
Usha Bhalla retweeted
The most popular way to interpret AI is missing the bigger picture.
Models think in curved shapes. But sparse autoencoders (SAEs) work with straight lines.
Can they still capture models’ curved neural geometry? Yes, but not how you might think! (1/7)
---///🥱🥱 ~~~ooo😉😉
interp, now thinking outside of 1d @goodfire! stayed tuned for more from us soon :)
Usha Bhalla retweeted
Neural networks might speak English, but they think in shapes.
Understanding their rich *neural geometry* is key to understanding how they work – and to debugging and controlling them with precision.
Starting today, we’re releasing a series of posts on this research agenda. 🧵
Usha Bhalla retweeted
My team at @GoodfireAI has been cooking up a new way to do interpretability: decompose a language model’s weights, not its activations.
Our decomposition natively handles attention (!) and behaves less like a lookup table and more like a generalizing algorithm. (1/6)
Usha Bhalla retweeted
📣 Excited to announce our oral presentation at #ICLR!
LLMs capture rich semantic structure, as evidenced by their strong performance across a wide range of language and reasoning tasks.
But Sparse Autoencoders (SAEs), a popular interpretability tool, mostly learn local, noisy, token-level features when applied to LLMs (e.g., hundreds of features for the word “the”).
So why aren’t SAEs finding that rich semantic structure?
👉 Because they ignore the sequential nature of language.
We introduce Temporal SAEs to bridge this gap.
arxiv.org/abs/2511.05541
🧵 [1/N]
Usha Bhalla retweeted
Our research with Mayo Clinic was just covered in @TIME!
“If there's some barrier like, ‘Is interpretability useful?’ I think we've been cracking it, and I think we've smashed through it” — @DanJBalsam
We achieved state-of-the-art performance in predicting which of 4.2 million genetic variants cause diseases by interpreting a genomics model, in a new preprint with @MayoClinic.
We're now releasing an open source database for all variants in the NIH's clinvar database. 🧵(1/8)
Usha Bhalla retweeted
We achieved state-of-the-art performance in predicting which of 4.2 million genetic variants cause diseases by interpreting a genomics model, in a new preprint with @MayoClinic.
We're now releasing an open source database for all variants in the NIH's clinvar database. 🧵(1/8)