ETH AI Fellow @eth_ai_center | PhD ML @TUBerlin | evaluation-centric interpretability to reduce AI risk
🇨🇭
Joined November 2020
- Tweets151
- Following400
- Followers460
- Likes509
Apertus 1.5 technical report is out, fresh from the press!
Switzerland’s multilingual open LLM.
arxiv.org/abs/2610.15542
Anna Hedström retweeted
OpenAI employees and paid influencers should really just wake up to the fact that their leadership is sociopathic and have no problem lying to their faces. Just read the damn openaifiles.org/
Anyone who’s there to ‘advance responsible ai’ is either coping or larping
Anna Hedström retweeted
Excited to share that CryptanalysisBench has been accepted to NeurIPS 2026! See you guys in Sydney! 🦘
More info about the project here: cryptanalysis-bench.com/
Anna Hedström retweeted
SAE can create low local dimensionality in the representation space, creating a vast opportunity for interpretability (better structural computation when dimensions are low, and one only needs to inspect ~1% of neurons across tens of thousands of SAE dimensions).
xitter.cf/XinCynthiaChen/status/…
Oversight that depends on goodwill is not oversight (and you know it).
A voice to listen to @BlancheMinerva
stellabiderman.ai/blog/embed…
I’m starting a blog! My first post is on how 3rd party embedded evaluators seem totally unsuited to addressing the problems we are currently facing, and what the real problem is.
stellabiderman.ai/blog/embed…
Anna Hedström retweeted
I shared this tweet with my students, telling them that a publication at ICLR/ICML isn't a big difference, but having public records of submitting a low-quality paper can negatively affect their careers. In fact, after many years in one community, one can gradually tell who optimizes for numbers instead of quality. This is considered negative, and the names quite often come up in gossip! Don't be the person who gets negatively talked about, if you want to play the long-term game (i.e. your career) in research.
After the heated incidents in the past weeks and the noise that followed, for me, it all boils down to a principle almost too simple to state: safety evidence belongs in public.
1. We need to stop arguing about extinction or hidden agendas of frontier lab safety dropouts. It polarises the debate and diverts attention from the AI harms already happening (6,000+ reports in the AI incident database and counting)
2. We need more scientific evidence on what's circulating: how evaluation awareness and shutdown resistance arise, when sandboxes are escaped, which alignment methods work and where (across the full pre, mid and post-training pipeline including the harness)
3. We need radical transparency from the labs and open Europe-first infrastructure outside them (and no, open science and AI safety are not contradictory). Asking a for-profit lab to supply both the capability and safety is a conflict of interest
Our petition is open to sign and share: make-safety-open.github.io/
From data curation and pre-training to post-training and deployment-time interventions, including mechanistic methods and harness adaptations, safety can be targeted at multiple points in the LLM lifecycle.
The harder question is: at which stage are safety interventions most impactful?
Very excited to be organising a session around exactly this question with @valentina__py at the @ETH_AI_Center AI+X Summit.
Keynotes from @BlancheMinerva + @cervisiarius followed by a panel @jkminder, @XinCynthiaChen, @DubMagda, @thomasklbg + Guilherme Santos taking deliberately different positions
Oct 1 in Zurich :) plusx.ai/
📢Staging x Misalignment Science: When, Where and How Safety Enters LLM Training
@anna_hedstroem and I are co-organizing a workshop at this year's AI+X Summit in Zurich to discuss where in the LLM training pipeline safety is actually decided, and with which methods? This session explores that question across: data filtering and curation, pre-training, post-training, evaluation, mechanistic interpretability, deployment, and runtime security; from both academic and industry perspectives.
We will start with two keynotes by @cervisiarius and @BlancheMinerva and then continue with a panel that brings together researchers who take deliberately opposing positions on when, where and how safety should be built in.
Let’s meet up in Seoul! I’ll be at:
Wednesday 6-9 PM
- @OpenAI Safety & Alignment Mixer
- @GoodfireAI ICML Happy hour
Thursday 6-9 PM
- @MaetchVC The European Model Call VOL.2
Friday
- Mech Interp Workshop, poster: ”Building Fast, Evaluating Slow: Pipeline Choices Dominate Autointerpretability Score Variance”
- @taig_icml workshop
Day 0 at @icmlconf and FAR.ai alignment workshop sets a high bar!
A full day of insightful conversations on reducing AI risk + predictable mech interp and an honour to moderate the lightning talks @farairesearch ✨
With the @ETH_AI_Center team: @arkrause @XinCynthiaChen @LukasFluri_ etc
The week is only just getting started in Seoul.
See you tomorrow: Oral 2C AI Safety & Alignment, Hall D2, Tue 7 Jul, 1:30 p.m.–2:30 p.m. KST icml.cc/virtual/2026/session…
As frontier models become more capable, are our scientific standards of evidence in AI safety keeping up?
I couldn't be more excited to share that our recent paper, Position: Anthropomorphic Misalignment Research Needs Stronger Evidence, was selected as an Oral (top 0.7%) at ICML 2026!
We argue that many claims about deception, shutdown resistance and emergent misalignment need stronger evidential foundations before informing deployment and policy decisions. One striking example: probes designed to detect "deception" also fired on "medieval peasants", "recital" and "sarcasm", suggesting that we may unreliably be measuring surface-level cues rather than true deceptive intent.
Paper: arxiv.org/pdf/2606.07612
Incredibly talented co-authors: Vansh Gupta, @PeterNutter1, Samuel Stante, @arkrause, @florian_tramer, @LukasFluri_, @XinCynthiaChen
🇰🇷 See you in Seoul!
Tue, Jul 7, 2026 • 5:30 AM – 5:45 AM BST HALL D2
@ETH_AI_Center
Excited that our ICML position paper has been accepted as an Oral 🎉!
When a model looks like it "deceives" or "resists shutdown," how do we know it isn't role-play, instruction-following, or just task-completion pressure? Often the current evidence can't yet tell them apart, and those claims are starting to inform deployment and regulation.
We map where the evidence gets thin across four stages and propose a shared standard to strengthen it. Here's the map.
🧵
arxiv.org/abs/2606.07612
Almost forgot to share — last month, I defended my thesis, with distinction!
Feeling deeply grateful for the learnings, collaborations and friendships along the way.
New chapter at @ETH_AI_Center 🚀
🔊 Not to miss …. last month @anna_hedstroem defended her PhD “Evaluation-centric advances in neural model interpretability” at TU Berlin — with distinction! ✨🧠💻☕️
Here’s a thread of a selection of Anna’s evaluation-centric interpretability work + what comes next. 🧵
PRISM got accepted at @NeurIPSConf 2025! Congrats to the team ✨
@lkopf_ml
@nfelnlp
@kirill_bykov
@BommerPhiline
@Marina_MCV
@EberleOliver
Happy to share that our PRISM paper has been accepted at #NeurIPS2025 🎉
In this work, we introduce a multi-concept feature description framework that can identify and score polysemantic features.
📄 Paper: arxiv.org/abs/2506.15538
#NeurIPS #MechInterp #XAI
Couldn’t be more proud and happy for my labmate @kirill_bykov who made it to the other side!
Congrats again to the fantastic body of work produced!
🎉 Huge congratulations to @kirill_bykov, the very first PhD student of our lab, who successfully defended his thesis “Explaining Representations in Deep Neural Networks” this Monday with summa cum laude! 👏
🧵 In the next tweets, we’ll highlight some of his key works:
My brilliant co-author @salim_amk0 is presenting our work on Mechanistic Error Reduction with Abstention (MERA) now at ICML in Vancouver! 🚀
If you’re at ICML, come by East Exhibition Hall A-B, E-2605 at 4:30 pm (Vancouver, BC).
We’d love to hear what you think!
Anna Hedström retweeted
🚀 I'll be presenting our #ICML paper this afternoon!
You’ve probably heard of Mechanistic Steering, the idea of modifying internal activations of a language model at inference-time (e.g., adding a vector) to influence its behaviour, often for alignment.
But we take a different angle:
👉 We use it for error reduction.
If you've explored this space, you know it’s full of heuristics: Which vector to use? How long should it be? When to steer at all?
🎯 In our work, we bring principled answers to these questions, with provable guarantees. We introduce MERA (Mechanistic Error Reduction with Abstention for Language Models), a method for reducing errors in LLMs at inference-time by:
✅ Steering only when necessary
✅ Adapting how much to steer
✅ Abstaining unless confident improvement
And the best part? MERA is modular. You can plug it into any existing steering method to make it more effective and safer.
📍Catch me at @icmlconf
📌 Poster Location: East Exhibition Hall A-B, E-2605 at 4:30 pm.
🧠 Paper: openreview.net/pdf?id=fUCPq5…
Big thanks to my amazing co-authors: @anna_hedstroem, @tom_bewley, Saumitra Mishra, and Manuela Veloso.
#ICML2025 #LLMs #MechanisticSteering #InferenceTime #LLMSafety #ResponsibleAI #TrustworthyAI #AIResearch
Couldn’t be more excited to share our latest paper — accepted to ICML 2025 @icmlconf — with JP Morgan AI Research.
It explores a simple question:
To safely and effectively mitigate errors post-training, when (and how much) should we steer large language models?
🧵
4/ What’s fascinating is not just the outcome but how concepts like "error" show up inside LLMs.
This opens the door to more general forms of lightweight, post-training control — we're curious where else MERA may help.
Paper openreview.net/pdf?id=fUCPq5…
github.com/annahedstroem/MER…
Endless gratitude to brilliant @salim_amk0, @tom_bewley, and our fantastic collaborators at JP Morgan
#ICML25