open source interpretability platform 🧠🧐
the residual stream
Joined July 2023
- Tweets93
- Following14
- Followers1.6K
- Likes58
Pinned Tweet
An average person can't look a CT scan and identify cancer, but radiologists can.
An average person can't look at Llama's model activations and identify lying, but Natural Language Autoencoders sometimes can.
Here, an activation verbalizer shows Llama planning to lie. 🧵
an interpretability "encyclopedia"? for accelerating interp research? 🤯
btw 7M+ neurons for qwen3-32B, gemma 4 31B, and llama 3.3 70B are now on neuronpedia:
neuronpedia.org/qwen3-32b/55…
neuronpedia.org/gemma-4-31b-…
neuronpedia.org/llama3.3-70b…
neuronpedia retweeted
With J-Space, Anthropic has managed to monitor Claude’s internal thoughts.
Again? It feels as though we have heard that claim several times already!
Ah but this time the headlines say they found consciousness. Let's dig into this!
In this post, I'll walk you through the idea behind the so-called J-lens, how you can easily reproduce it, and why actual neuroscientists are cautiously excited by these results.
And no, it does not prove Claude has a soul or inner feelings.
neuronpedia retweeted
Which cultures do LLMs best align with on open-ended questions? Why? Can we change that default?
We study this in our new work: "Steering LLMs for Culturally Localized Generation" (COLM 2026), from my time at Google DeepMind! 🧵
arxiv.org/abs/2603.23301
neuronpedia retweeted
New research: language models develop a distinction between a small set of representations they can report on and reason with, and a much larger volume of automatic processing — a structure that closely mirrors global workspace theory.
Some highlights 🧵
neuronpedia retweeted
I thought this was an excellent paper! Thanks to Anthropic for asking me to write a review of it, linked below
I've long suspected that models have some kind of "working memory" to store intermediate variables during a forward pass and IMO this paper has the best evidence yet
neuronpedia retweeted
LLMs represent information using high-dimensional neural activity. A small bit of this activity appears to be privileged, available to the model to be described, modulated, and reasoned with. I expect that understanding this "workspace" is key to making sense of LLM cognition.
neuronpedia retweeted
We locate a small subset of the representations inside a language model (the J-space), and find that it resembles an internal monologue.
It shows intermediate concepts, as well as judgements about the current text. It even detects prompt injections!
neuronpedia retweeted
New Anthropic research: A global workspace in language models.
Of everything happening in your brain right now, only a tiny fraction is consciously accessible—thoughts you can describe, hold in mind, and reason with.
We found a strikingly similar divide inside Claude.
neuronpedia retweeted
trained the first natural language autoencoder on gpt-2 almost a year ago, now we have one on mythos.🥲
do read the paper/play with the live demo! so excited it's finally out.
Try it yourself at neuronpedia.org/nla - you can freely chat with the models, and click tokens you want the NLA to explain.
You can also share your chats with highlighted explanations like so: neuronpedia.org/nla/cmothttr…
Happy Activation Verbalizing! 🔢➡️🔤
Read about the effectiveness & limitations of NLAs in the paper: transformer-circuits.pub/202…
Train and run NLAs: github.com/kitft/natural_lan…
Our thanks to @KitF_T (Trained NLAs/code) co-authors @thesubhashk @euan_ong plus @saprmarks @ferrum_of_omega & others for making this possible!
neuronpedia retweeted
New Anthropic research: Natural Language Autoencoders.
Models like Claude talk in words but think in numbers. The numbers—called activations—encode Claude’s thoughts, but not in a language we can read.
Here, we train Claude to translate its activations into human-readable text.
neuronpedia retweeted
Announcing new ARENA material: 8 new exercise sets on alignment science, interpretability & AI safety - each containing 1-2 days of structured, hands-on content replicating key papers in the field.
All open source on a public GitHub, and available for study. Here's what's in it:
