AI Safety Researcher, who has clearly not been very active on X since 2020.
Kochi, India
Joined April 2016
- Tweets196
- Following621
- Followers338
- Likes2.5K
Pinned Tweet
Model organisms (MOs) are frequently used as testbeds for mechanistic interpretability methods. We train 54 of them and find that their interpretability scores vary widely depending on how they were trained.
Your interpretability benchmark result may be a lottery. 🧵
Nikita Menon retweeted
Apply by 20 Sept for the next LASR Labs cohort, running Jan–April in London.
13 weeks to write an AI safety paper on topics like scalable oversight, evaluation awareness and constitutions, with supervisors from @AISecurityInst, @GoogleDeepMind, @ApolloResearch, and more. (1/3)
We recommend that benchmarks include MOs built using many different construction methods, ideally including integrated training.
No single MO’s interp result should be treated as individually meaningful and interp scores should always be aggregated across several MOs.
Paper: arxiv.org/abs/2607.01033
Blogpost: lesswrong.com/posts/frvmrrND…
With @Gabriel2KS, @A_Szablewski, and @_Raffaello_F, supervised by @sheimersheim. Work done at @LASRlabs.
Nikita Menon retweeted
this interpretability history lesson is rare
I don't think I've seen a writeup like this in interpretability research before. Maybe i'm wrong, and i'd love to be educated.
IMO every interpretability researcher, pre-mech-interp and post-mech-interp eras, should read it though this is not an easy read. One needs a lot of background to digest it.
but the same message echoes through many years of interpretability research:
- we keep trying to explain models after the fact, but the deeper problem is that our models were never built to be read and understood in the first place. many of our interpretability discoveries are the artifacts of our imagination or brittle under simple tests.
if you seriously work through the whole piece, you will understand a lot of the unresolved tension behind modern interpretability.
Nikita Menon retweeted
the word you are looking for is karma
Breaking News: President Jair Bolsonaro of Brazil has tested positive for the coronavirus after months of dismissing its seriousness. More than 65,000 Brazilians have died. nyti.ms/2ZFOgHS
