Ph.D. student @KhouryCollege, working to make LLMs interpretable
Boston, MA
Joined September 2022
- Tweets73
- Following178
- Followers315
- Likes299
Arnab Sen Sharma retweeted
New COLM paper: Identifying Introspection From the Inside
When an LLM tells us about its decisions, does it ๐ฌ๐ฏ๐ฐ๐ธ what drives its choicesโor is it guessing?
In our setting, we find that faithful models decide and report with the same layers. Unfaithful ones don't. ๐งต
Arnab Sen Sharma retweeted
Excited to kickstart this effort with @glnmario and a star-studded advisory team! ๐
Also, we're hiring for 2 MTS and 1 Ops manager! Come build the future of white-box audits with us: parallx.ai/careers
Introducing Parallax ๐
We're a new UK/EU-based nonprofit research lab building scalable methods and infrastructure to audit the beliefs, goals, and plans that shape model behaviours in long-horizon agentic evaluations.
parallx.ai โ Thread ๐งต 1/
Arnab Sen Sharma retweeted
๐ง ๐ค The 2026 New England Mechanistic Interpretability (NEMI) Workshop will be Aug. 14 at Boston University!
Help spread the word and join the New England mech interp community! Registration and submission info in thread:๐
Arnab Sen Sharma retweeted
Can you tell when an AI model is lying?
Announcing Aletheia's Quest, an AI lie detection challenge running this summer, organized by @cadenza_labs and @ndif_team.
Multiple model organisms to interrogate and probe, $50K prize pool, no local GPU required.
Arnab Sen Sharma retweeted
Can you solve this algebra puzzle? ๐งฉ
cb=c, ac=b, ab=?
A small transformer can learn to solve problems like this!
And since the letters don't have inherent meaning, this lets us study how context alone imparts meaning. Here's what we found:๐งตโฌ๏ธ
Arnab Sen Sharma retweeted
At the #Neurips2025 mechanistic interpretability workshop I gave a brief talk about Venetian glassmaking, since I think we face a similar moment in AI research today.
Here is a blog post summarizing the talk:
davidbau.com/archives/2025/1โฆ
Arnab Sen Sharma retweeted
I am very excited to share that our paper, "One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Models" will be presented at #NeurIPS2025!
@ViaSurkov is presenting it at #MexIPS2025:
๐๐๐ ๐ฒ๐จ๐ฎ ๐๐ซ๐ ๐๐ญ๐ญ๐๐ง๐๐ข๐ง๐ ๐๐๐ฎ๐ซ๐๐๐ ๐ข๐ง ๐๐๐ฑ๐ข๐๐จ ๐๐ข๐ญ๐ฒ, ๐ฉ๐ฅ๐๐๐ฌ๐ ๐ฌ๐ญ๐จ๐ฉ ๐๐ฒ!
Date: Thursday, Dec 4, 2025
Time: 11:00 AM โ 2:00 PM PST
Location: Foyer (Mexico City Poster Session)
Come visit @ViaSurkov it's his first conference and he will be happy to explain his amazing work.
Sadly, #NeurIPS2025 does not allow for parallel presentation in San Diego. However, I am in San Diego and happy to meet up / chat. Please don't hesitate to reach out here or via [email protected].
Once again, a big shout out to our brilliant students Viacheslav Surkov and Antonio Mari who did phenomenal work here and pushed this work (that started as a class project more than a year ago) all the way to pass the high threshold of #NeurIPS2025.
Also, I want to thank manifund.org/ (@andyarditi and @ryan_kidd44 in particular) for helping us to finance Viacheslav Surkov's conference trip.
Please find more information about our work below. We have so many amazing interactive materials (e.g., 3x huggingface demo spaces) for you to check out. Most of our implementations are open-sourced (RIEBench on FLUX, which we added to our appendix during the NeurIPS rebuttal is currently missing but we plan to add it ASAP).
Me demoing the demo attached.
Arnab Sen Sharma retweeted
A key challenge for interpretability agents is knowing when theyโve understood enough to stop experimenting.
Our @NeurIPSConf paper introduces a self-reflective agent that measures the reliability of its own explanations and stops once its understanding of models has converged.
Arnab Sen Sharma retweeted
Who is going to be at #COLM2025?
I want to draw your attention to a COLM paper by my student @sheridan_feucht that has totally changed the way I think and teach about LLM representations. The work is worth knowing.
And you meet Sheridan at COLM, Oct 7!
Arnab Sen Sharma retweeted
How do language models track mental states of each character in a story, often referred to as Theory of Mind?
Our recent work takes a step in demystifing it by reverse engineering how Llama-3-70B-Instruct solves a simple belief tracking task, and surprisingly found that it relies heavily on concepts similar to pointer variables in C programming!
Arnab Sen Sharma retweeted
๐ก New ICLR paper! ๐ก
"On Linear Representations and Pretraining Data Frequency in Language Models":
We provide an explanation for when & why linear representations form in large (or small) language models.
Led by @jack_merullo_ , w/ @nlpnoah & @sarahwiegreffe
Arnab Sen Sharma retweeted
Lots of progress in mech interp (MI) lately! But how can we measure when new mech interp methods yield real improvements over prior work?
We propose ๐ ๐ ๐๐: a Mechanistic Interpretability Benchmark!
ALT Logo for MIB: A Mechanistic Interpretability Benchmark
Arnab Sen Sharma retweeted
๐จ A new architecture of automating mechanistic interpretability with causal interchange intervention! #ICLR2025
๐ฌNeural networks are particularly good at discovering patterns from high-dimensional data, so we trained them to ... interpret themselves! ๐งโ๐ฌ
1/4
Arnab Sen Sharma retweeted
[๐] Are LLMs mindless token-shifters, or do they build meaningful representations of language? We study how LLMs copy text in-context, and physically separate out two types of induction heads: token heads, which copy literal tokens, and concept heads, which copy word meanings.
Arnab Sen Sharma retweeted
In case you ever wondered what you could do if you had SAEs for intermediate results of diffusion models, we trained SDXL Turbo SAEs on 4 blocks for you. We noticed that they specialize into a "composition", a "detail", and a "style" block. And one that is hard to make sense of.
Arnab Sen Sharma retweeted
Why is interpretability the key to dominance in AI?
Not winning the scaling race, or banning China.
Our answer to OSTP/NSF, w/ Goodfire's @banburismus_ Transluce's @cogconfluence MIT's @dhadfieldmenell
resilience.baulab.info/docs/โฆ
Here's why:๐งต โ๏ธ
Arnab Sen Sharma retweeted
DeepSeek R1 shows how important it is to be studying the internals of reasoning models. Try our code: Here @can_rager shows a method for auditing AI bias by probing the internal monologue.
dsthoughts.baulab.info/
I'd be interested in your thoughts.
Arnab Sen Sharma retweeted
why do language models think 9.11 > 9.9?
at @transluceAI we stumbled upon a surprisingly simple explanation - and a bugfix that doesn't use any re-training or prompting.
turns out, it's about months, dates, September 11th, and... the Bible?
Monitor: An Observability Interface for Language Models
Research report: transluce.org/observability-โฆ
Live interface: monitor.transluce.org/ (optimized for desktop)
