@millicent_li

cs phd @ northeastern | interpretability, evaluation, and training dynamics | formerly: @uwcse + @uwnlp, @meta fair, and @MSFTResearch

 Cambridge, MA
millicentli.github.io
Joined November 2021
  • Tweets96
  • Following133
  • Followers288
13Media
NLAs look like they "work"---but compared to what? How can we know that they actually tell us anything about model internals? In my new blogpost (and paper, to be presented at ICML!), we investigate whether activation verbalizers (like NLAs) produce faithful explanations.
Anthropic
@AnthropicAIMay 7
New Anthropic research: Natural Language Autoencoders. Models like Claude talk in words but think in numbers. The numbers—called activations—encode Claude’s thoughts, but not in a language we can read. Here, we train Claude to translate its activations into human-readable text.
4
12
27
2,414
Interpretability stress-testing results can be unreliable if you set up the test wrong. We introduce "How to train your model organism" (w/ @davidbau @byron_c_wallace), where we propose to use more realistic model organisms for interpretability auditing.
Nikita Menon@nikxtacoJul 27
Model organisms (MOs) are frequently used as testbeds for mechanistic interpretability methods. We train 54 of them and find that their interpretability scores vary widely depending on how they were trained. Your interpretability benchmark result may be a lottery. 🧵
1
5
3
45
3,502
VLMs use pointer-like signals for spatial variable binding. But where does the binding actually happen? We find a dual mechanism: the vision encoder is the primary source of binding, with the LM backbone playing a secondary role.
3
18
3
73
7,518
Millicent Li retweeted
Liang Zheng
@LiangZheng_06Oct 9
who are still writing research papers completely manually? am I the only person who thinks this is the only way of staying human and genuine ?
44
30
10
517
32,214
At #COLM2026 tomorrow! If swapping a demographic attribute flips an LLM's answer, is that bias? Compared to what? A paraphrase often does too. Come chat about baselines for counterfactual prompting: 📍 Wed 10/7, 11am–1pm, Imperial #41 w/ @mosh_levy @yoavgo @byron_c_wallace
Zihao (Gavin) Yang
@ZihaoGavinYangMay 11
1/ (New paper!) If swapping the gender in an input prompt makes the AI model give a different answer it means that it has to have a gender bias, right? Wrong. 🧵on counterfactual prompting for LLM evals: Paper: arxiv.org/abs/2605.01048
6
1
17
1,612
Millicent Li retweeted
Emmy Liu@_emliuOct 6
Honoured to receive an outstanding paper award for our work! Couldn’t have done it without all my coauthors @KaiserWhoLearns @millicent_li @wordscompute @lltjuatja @JentseHuang and my advisor @gneubig 🫡
Conference on Language Modeling@COLM_confOct 6
Congratulations to the authors of the 3 outstanding papers! arxiv.org/abs/2607.01077 arxiv.org/abs/2604.08510 arxiv.org/abs/2604.13977
14
26
5
186
12,842
well-deserved award for @_emliu who really orchestrated all of this!! go to her talk, it'll be worth it! :)
Emmy Liu@_emliuOct 6
Honoured to receive an outstanding paper award for our work! Couldn’t have done it without all my coauthors @KaiserWhoLearns @millicent_li @wordscompute @lltjuatja @JentseHuang and my advisor @gneubig 🫡
15
500
Millicent Li retweeted
David Atkinson
@diatkinsonOct 6
New COLM paper: Identifying Introspection From the Inside When an LLM tells us about its decisions, does it 𝘬𝘯𝘰𝘸 what drives its choices—or is it guessing? In our setting, we find that faithful models decide and report with the same layers. Unfaithful ones don't. 🧵
10
72
9
317
22,553
Millicent Li retweeted
Emmy Liu@_emliuOct 5
Excited to head to #COLM2026 to present the Implicit Curriculum Hypothesis and separately, the HalluWorld benchmark at the @SnorkelAI Frontier Data Summit! I’m also on the academic/industry job markets. Happy to chat about pretraining, midtraining, and training dynamics 🧵
4
12
1
82
4,969
Millicent Li retweeted
Sanchit Ahuja@SanchitAhuja7Oct 5
I will be at COLM presenting my first PhD paper with @TerraBlvns as a spotlight Oral at GenAI4World workshop! Happy to connect with people doing anything multilingual (training, data, evals). Also, looking forward to meeting people doing AI safety! arxiv.org/abs/2606.00284
4
37
673
[New preprint] How do images in VLMs align with words? In this work, we found a set of attention heads responsible for OCR. But to our surprise, these heads were actually able to verbalize much more than just text. So, we used them to create a simple logit lens for image tokens!
5
25
4
107
11,995
Millicent Li retweeted
Jacqueline He
@jcqln_hSep 15
Knowledge distillation (KD) is a powerful training objective—but when should we use it? We find that KD behaves differently across training stages: vs. vanilla NTP, it improves reasoning + factual recall in pre-training, but trades off recall for reasoning in mid-training. We introduce Switch Distillation, which improves this tradeoff using teacher uncertainty. Paper: arxiv.org/abs/2609.01532 1/🧵
2
33
147
8,747
Millicent Li retweeted
Emmy Liu@_emliuSep 15
Happy to be heading to #COLM2026 in SF to present this as an ✨ oral ✨! You can catch me at: 📢 Oral Session 3 (Wed morning) 🪧 Poster session 3 (Wed morning), I-3-24 Look forward to interesting discussions! I am also on both academic/industry job markets (more info coming)
Emmy Liu@_emliuMay 20
Copying → morphology/translation → basic arithmetic → complex reasoning & math. Across every model family we tested, LLMs acquire skills in roughly the same order during pretraining. Can we use this to predict what a model will learn next, just from its internals? 🧵
7
110
7,332
highly recommend working with richard :) incredibly fun mentorship + I had a lot of fun as an undergrad at UW working on ideas that mattered to me! ASU is very lucky to have him
Richard Li@lichard49Sep 8
I am excited to share that after 7 incredible years @uwcse , starting January 2027, I will join @ASU as an assistant professor and founding member of the School of Technology for Public Health with a joint appointment in @SCAI_ASU. 1/n
3
193
Millicent Li retweeted
Gabriele Sarti
@gsarti_Sep 9
Excited to kickstart this effort with @glnmario and a star-studded advisory team! 🚀 Also, we're hiring for 2 MTS and 1 Ops manager! Come build the future of white-box audits with us: parallx.ai/careers

Join Parallax

Careers at Parallax, a non-profit research lab developing open methods and infrastructure for white-box auditing of frontier AI systems. Applications for our founding roles are closed.

parallx.ai
Parallax
@parallx_aiSep 9
Introducing Parallax 🔍 We're a new UK/EU-based nonprofit research lab building scalable methods and infrastructure to audit the beliefs, goals, and plans that shape model behaviours in long-horizon agentic evaluations. parallx.ai — Thread 🧵 1/
9
17
78
4,333
Millicent Li retweeted
Richard Li@lichard49Sep 8
I am excited to share that after 7 incredible years @uwcse , starting January 2027, I will join @ASU as an assistant professor and founding member of the School of Technology for Public Health with a joint appointment in @SCAI_ASU. 1/n
13
10
4
42
5,316
Millicent Li retweeted
Xiaoyan Bai
@Elenal3aiAug 27
📢 Deadline extended to Sept. 2, AoE‼️ 🧪 Have some cool work? Submit it to our Interpretability for Discovery workshop and join us in Atlanta! 🔍
Xiaoyan Bai
@Elenal3aiAug 14
📣Call for Papers -- Are AI models discovering things we don’t know yet? 👀Submit to Interpretability for Discovery workshop @NeurIPSConf 2026, Atlanta! Deadline: Aug 29 Website: interpretability4discovery.g… We're looking for submissions on: 🔹Methods for using interpretability to drive discovery in any field 🔹Case studies of using interpretability to discover/validate patterns and mechanisms in scientific models (including failure cases and negative results) 🔹Comparing interpretability to black-box approaches for extraction of model knowledge 🔹Position papers on where this can and can't work Failure cases and negative results are welcome too! ✨We are thrilled to announce our wonderful speakers (with more coming soon!): Gasper Begus @begusgasper Katrin Franke @kfrankelab Nicholas Wang @nkwang24 John F. Wu @jwuphysics Proudly sponsored by @GoodfireAI ✉️Contact us at: [email protected] Looking forward to seeing everyone’s work! Organizers: @boknilev, @EkdeepL, @YNikankin, @ChenhaoTan, @amirtha255
4
7
710
Millicent Li retweeted
Emmy Liu@_emliuAug 17
I'm reflecting on how much research has changed since I've joined the PhD and wrote a short blog post about it (I joined in the tail end of the BERT era!). It seems pretty crazy how different processes are now, and I took the chance to do a retrospective before graduation: nightingal3.github.io/blog/2…
17
138
7
1,098
200,293
Millicent Li retweeted
Koyena Pal
@kpal_koyenaAug 12
Nice independent evidence for cross-model CoT transfer, from a security angle rather than as a property of generalizability. We explored a similar phenomenon in: arxiv.org/abs/2601.11517 Specifically, we transfer CoT from a strong model to weaker models and find that the weaker models' performance and response closely match that of the strong model.

Do explanations generalize across large reasoning models?

Large reasoning models (LRMs) produce a textual chain of thought (CoT) in the process of solving a problem, which serves as a potentially powerful tool to understand the problem by surfacing a...

arxiv.org
Alexander Panfilov
@kotekjedi_mlAug 11
We can finally talk about it: We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company. We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
2
3
14
1,558
Millicent Li retweeted
Xiaoyan Bai
@Elenal3aiJul 20
📣We are excited to announce our 🔭 Interpretability for Discovery workshop at NeurIPS, Atlanta! ⏰Deadline: Aug 29, AoE 🔗Website: interpretability4discovery.g… 🌟We are also looking for reviewers! tinyurl.com/bdkh64af Stay tuned for more details!
Yaniv Nikankin
@YNikankinJul 20
We're hosting the workshop on Interpretability for Discovery🔬 at NeurIPS 2026, Atlanta! AI models predict protein folding, forecast weather patterns, and more. They might be learning new patterns we haven't figured out yet. Can we use interpretability to read out these patterns?
11
70
4,906
Millicent Li retweeted
Emmy Liu@_emliuJul 5
Excited to be heading to #ICML2026 to present 3 papers! ✈️ Happy to talk about developmental LM science, continual learning, or LM creativity! See more details below, and send me a message via email if you'd like to chat! (Twitter chat isn't working for me). Also excited for shopping 👜
4
5
45
4,119
Load more