Quang Hưng Phạm retweeted
Preparing for AI Research Scientist interviews? 🔥
We just open-sourced PaperLeet 🐈⬛ LeetCode for AI researchers: implement the core of ~100 key papers (llm, rl, diffusion, flow matching, JEPA, world models), graded by tests that check the math.
paperleet.com
Quang Hưng Phạm retweeted
Anticipating interference does not alter the strength of long-term memory reactivation in working memory, according to two EEG studies. The findings show that contralateral delay activity remains stable even when participants expect perceptual or task-based distractions, challenging the assumption that anticipation modulates memory maintenance.
doi.org/10.1162/JOCN.a.2625
Quang Hưng Phạm retweeted
it’s been an enormous privilege and joy to watch @nickcammarata cook this up over the last year. week after week, experiment after experiment, he unraveled the tangled mess of our thinking about neural nets to reveal the truth.
i never knew quite what to expect on our weekly walks. each conversation brought a torrent of new theories about the nature of AI, some conjectures teetering on the brink of madness, and others that seemed to cut through years of confusion with a single sentence.
nick is one of the great scientists of our time. his intuition and research taste are out of this world, but what sets him apart is his unmatched ability to capture the essence of what something truly is, without being lost in the details. i would highly recommend you read his entire essay if you’re curious about AI. it’s long, but written beautifully.
nick’s research makes me far more optimistic about the future of AI alignment. i hope that it can inspire the same for you.
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Quang Hưng Phạm retweeted
arxiv.org/abs/2610.15965
Dealing with data repetition, one of the most interesting issues in pretraining. For this case maybe old deep learning wisdom like small batch sizes could be useful.
Quang Hưng Phạm retweeted
Deep learning is often taught as a sanitized collection of facts and math. But work at the frontier of AI is messy and empirical.
Our new class, Deep Learning Alchemy, aims to teach the art and science of deep learning in the hands-on spirit of Language Modeling From Scratch.
Students build intuition by running experiments on small LMs, then test it through prediction quizzes: given an unseen code diff, can you predict what happens?
deep-learning-alchemy.github…
My co-instructor @kothasuhas and I think there's significant demand for classes that treat students as serious learners who want to be assessed for what they’ve learned, and so we're trying a new exam-heavy approach to the class (7 exams, 85% of the grade, with the remaining 15% in-person discussion assessments).
We’ve designed the class so students can use AI agents to run their experiments and do their assignment, but they then have to demonstrate that mastery on exams without access to the assignment, and any partial credit on the exam has to come through citations to the assignment, incentivizing them to know what’s in their write-up.
The same assignments and quizzes are public for anyone to try (lecture recordings will be released through @StanfordOnline with some delay). We have some very fun assignments and quizzes covering both classic topics (like NQM, run-to-run stochasticity, and scaling laws) and new and timely topics like muP or hyperball. Made by some real experts like @TristanThrush and @wen_kaiyue
Try some quizzes below!
Quang Hưng Phạm retweeted
This is a message that I prepend to all my conversations with coding agents
Coding agents love bloating the code, leading to context rot, bugs and higher token usage
Quang Hưng Phạm retweeted
This is a beautiful analysis of the order in which diffusion models resolve uncertainty about information at different abstraction levels in different representation spaces, with significant implications for latent diffusion.
panjiashu.github.io/feature-…
Quang Hưng Phạm retweeted
I wrote a blog post detailing the writing approach I took while working on my recent paper: sophielwang.com/blog/paper-w…
I describe the behind the scenes for how this paper came to be, why and how I storyboarded figures before running experiments, and some figure design tips! :)
Can “chicken” make a base model reason? 🐔
Yes!
Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too.
blog: sophielwang.com/cues
Quang Hưng Phạm retweeted
Crazy that this works. But very cool.
(Also, a bit of a palate cleanser to see people posting AI results that come from skillful application of novel ideas rather than just scaling.)
DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair.
It even works when the images and the captions come from different datasets.
Project page: dominik-schnaus.github.io/un…
⬇️
Quang Hưng Phạm retweeted
I've been doing some thinking about multimodality lately, specifically the semantic gap between common representations of language and perceptual signals. In that context, this is an exciting result!
(These thoughts are likely to land on my blog in written form at some point✍️)
DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair.
It even works when the images and the captions come from different datasets.
Project page: dominik-schnaus.github.io/un…
⬇️
Quang Hưng Phạm retweeted
excited to partner with @ravi_sola & the @PrimaMente team on pushing forward new mechanistic hypotheses for Alzheimer's
We discovered a new class of Alzheimer’s biomarkers with @PrimaMente. Today we're taking the next step by opening a global AI competition to discover new treatments.
Introducing the Alzheimer’s Translation Challenge, led by @PrimaMente and @AlzData. 🧵
Quang Hưng Phạm retweeted
Meet Mistral Large 4, aka Le Chonk.
• 1T parameters, natively multimodal. 49B active.
It is the best open weights model from US or Europe on aggregated benchmarks.
• State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding.
• Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure.
• Available to all via API today. Working with cybersecurity partners privately.
Open weights release end of October.
🤖 Made with AI
Quang Hưng Phạm retweeted
Our second call for NeurIPS Paris 2026 volunteers will open tomorrow TUE Oct 6th, 2pm CET.
We are still seeking for PhD students and postdocs to assist during the conference in exchange for complimentary registration.
Application form:
forms.gle/bBFUyUNngtCcEN3z8
Quang Hưng Phạm retweeted
💥 New paper: AI safety is full of "forbidden techniques" (using CoT to detect reward hacking, using model internals for training, etc). But do they really have a clear scientific basis? I'm not sure.
In this paper, we directly optimize models against harmlessness and honesty probes, and it works just fine (if you continuously update the probe!). The models learn how to generate harmless responses to harmful queries and honest responses under pressure to lie.
Figuring out how to correctly use interp techniques for training is becoming increasingly important: it's very likely that soon we won't be able to align models using output-based supervision. Future models will just max out all alignment training scenarios, but for the wrong reasons due to their general reward-seeking behavior. To have a chance of aligning future models, we need to do much more research on supervising model training using their internals *without losing monitorability*!
Quang Hưng Phạm retweeted
I really like Goodfire and think they will succeed. They won't get a watertight recipe for "alignment" in all generality, but will show that it's surprisingly easy to preclude training on signals that lead to unwelcome generalization.
Quang Hưng Phạm retweeted
I’ll take the opposite point, that for the vast majority of people, it probably doesn’t make sense to do a PhD in AI right now, and is better to join industry. I say this with love for Phillip, whose work and philosophy I do admire, and respect for putting this out there!
Everything in AI is, generally speaking, working well. Most things come from stacking wins and good ideas over and over again, and if you’re in industry, you have a much better opportunity to learn what is actually working and what doesn’t work, and then push the forefront of what’s possible at a much larger scale. Internships also don’t really feel the same because you’re certainly not working on the most important projects for the company and often come in with a high risk agenda and work alone. In industry, you learn a lot about how to manage budgets, work in large teams, and push projects that require a lot of effort and work to get done.
Academic project budgets are generally quite minuscule, maybe being O($1 - $100K) for many projects and you’re often working alone. If you really want to push novel ideas, another route I don’t think most people consider is also a startup, where you may be able to raise 1-100x more money to do the same project you may do in a PhD, with more resources and the ability to hire other people. And, if it goes well, the ability to scale it dramatically.
When it comes to hiring, often times, it’s easier to get an offer having worked well with other people in industry and then having them change jobs and bring you over. Doing a PhD can get your foot in the door, but often it’s still for entry level positions after doing a 5-year PhD. In industry too, you also will make a lot of strong connections, similar to those you’d make in academia. Though it might be the case that it’s good to do 1 year of a PhD and then drop out to work in industry at this point to get the benefit of both.
Doing a PhD right now is on extremely hard mode, where not much work ends up mattering and even if it works, there’s high cost to change the ways of industry to adapt something that might provide incremental gains when everything generally is working well with enough resourcing and effort.
Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI.
Perhaps counterintuitively, I think it's a great time to do so.
I wrote up some thoughts on this here: web.mit.edu/phillipi/www/wri…
Quang Hưng Phạm retweeted
Very good read!
Wrote a short blog about the "shape" of language models, and the tradeoffs they may present in the future. I genuinely think it's a valuable research direction to start thinking about now, especially w.r.t. harness design.
alexzhang13.github.io/blog/2…
Quang Hưng Phạm retweeted
I’m as excited about the possibilities AI brings as anyone. I should be because I’ve worked on it since 1985. But I want to balance this excitement with a dose of humanity and what still makes people both special and necessary.
Despite its nearly infinite knowledge, I still write better than Claude. Why? Precisely because I’m human. I understand my audience in a way Claude can’t and maybe never will. I understand what it means to be bored, confused, and distracted because I’ve been board, confused, and distracted. I’ve been moved to tears by words on a page. I have empathy for my reader because I too am a reader with a limited human brain.
So, when I write, I write for you, dear reader. Your time is precious. I need to earn your attention. Writing is a very imperfect process of communication from one human to another.
In contrast to my writing, Claude is brutalist. It assaults with declaratives. It fires off staccato jargon with no sympathy for a reader who is lost. Sure, you can coax it to behave but it doesn’t do this out of empathy for the reader because it lacks this. Claude’s default style reveals the true heart of the machine.
I hope my writing also has a bit of style, a little be of me, in it. A trail of words guiding you into a little part of my mind. The words are a fingerprint left behind. Claude leaves a big greasy fingerprint that is unmistakable. In that sense it has style. But that style reveals its lack of empathy for the reader. The best word for it is “condescending”.
What drives me to write this missive is some small creative spark and a human drive to share it. I woke up this morning with “a bee in my bonnet” about our societal enthusiasm and fear about AI. I need to write to articulate my thoughts to myself and hopefully to you.
AI is getting very very good. But I’m still better at this and other things. I’m sure that you too are better than AI at what you do. Real humans are still more creative and more interesting than AI. I want facts from AI, but I want to know what humans think and feel. I hope we blunder onwards into the future writing, creating, and sharing in our wonderfully imperfect human way.
If you read something written by me, know that it was written by me. It is my brain reaching out to yours without the filter of AI. My words. Imperfect. Like me and like you.
Quang Hưng Phạm retweeted
incredibly based that Xiaomi MiMo released their RL dataset
link: huggingface.co/datasets/Xiao…
