Research @Microsoft - Frontier Tuning | Scalable & Efficient RL for Frontier LLMs | PhD @ KAIST | ex-@mila_quebec (Bengio Lab) | Views are my own.
Redmond, Washington, USA
Joined March 2025
- Tweets113
- Following161
- Followers275
- Likes448
Minsu Kim retweeted
At least some settings (model-data combinations) benefit from IS on top of this kind of regression objective, but in others IS isn't needed.
(IIUC, the TBA objective below matches CORO when you set the reference to be the inference policy and drop IS).
xitter.cf/bartoldson/status/1996…
Minsu Kim retweeted
Highly recommended read from Google DeepMind and colleagues.
There is a lot of interest in self-evolving agents, but they're hard to get right.
This work highlights how important and difficult memory and retrieval are in agents.
The big question in this paper is whether self-evolving agents keep learning without forgetting.
They tested ReasoningBank, GEPA, and Meta-Harness on four task streams that drift in topic, workflow, or domain, built from BIRD, BFCL, and ScienceWorld.
The weights stay frozen, and the agents adapt by rewriting prompts, memory, and skills.
Self-evolution improves overall scores on most streams. But more adaptation consistently means agents forget more.
Methods that rewrite a shared prompt or workspace make global edits, so they interfere with unseen tasks and overwrite older knowledge when the stream shifts.
Memory retrieval keeps updates more local, but it still pulls in memories from other domains and collides when similar queries need different workflows.
Static benchmarks hide both problems, so test self-evolving agents on a stream that changes over time.
arxiv.org/abs/2610.25858
Chat with Paper: academy.dair.ai/papers/are-s…
Minsu Kim retweeted
“All roads to off-policy learning go through path consistency”
The CORO objective used here is equivalent to the TBA objective proposed in our paper from last year. All of these are variants of PCL and in the on-policy trajectory case yields same gradient as REINFORCE
arxiv.org/abs/2503.18929
Minsu Kim retweeted
New Google DeepMind paper finds that self-improving agents forget as they learn, and the method that learned new tasks fastest broke old ones the most.
Self-evolving agents improve without retraining by editing their system prompt, memory, or skill files. In real use tasks keep shifting, and each edit made for 1 task also applies to every task that follows.
The team ran ReasoningBank, GEPA, and Meta-Harness on task streams that changed topic, workflow, or domain. They tracked how much each method learned on the current task and lost on the others.
GEPA learned the most, 6.2 points per block on average, and also lost the most elsewhere, 4.3 points. After it learned to ask users for missing details, a later block rewrote that rule and its 15.5-point gain fell to 2.5.
If your agent edits its own prompt or memory, keep task-specific rules separate and retest old tasks after every update.
Minsu Kim retweeted
The CORO objective used here is equivalent to the TBA objective proposed in our paper from last year. All of these are variants of PCL and in the on-policy trajectory case yields same gradient as REINFORCE
arxiv.org/abs/2503.18929
No one changes the world alone
Introducing Ampersand - the first model built for us. Ampersand is a collaborator trained to help people achieve their individual and collective goals. In multi-month simulations, users collaborating with Ampersand accomplished more than with Fable and other frontier models
Available to everyone through our API
Minsu Kim retweeted
context-beyond-window.github…
If you're attending COLM, do not miss @Yoshua_Bengio's 9am talk at our workshop on Friday!
Recursive self-collapse of AI academia: Many great papers offer deeper insights into paths to superintelligence than industry does. But curation has become sampling from a slop-dominated distribution. AI-assisted curation feeds slop back to researchers, generating even more slop. Industry could be next.
Minsu Kim retweeted
In LLM post-training, policy gradient is "the RL loss".
When it fails, we blame the RL: exploration, credit assignment, sampling noise.
Image classification has none of these, and the policy gradient is exact. Following it is terrible...
ImageNet, 4% vs 62% for cross-entropy.
Recursive reasoning adds a new scaling axis: loop depth. Small models can solve hard problems, with more headroom as models scale.
GRAM turns recursive reasoning model into a sampler, adding width scaling through diverse parallel reasoning trajectories. #NeurIPS2026
GRAM is accepted to #NeurIPS2026! 🎉
Turning recursive reasoning into a stochastic latent trajectory — multiple hypotheses, and inference-time scaling in both depth and width.
A revised version of the paper and the code will be released soon!
🌐 ahn-ml.github.io/gram-websit…
General intelligence is compute-inefficient.
At the same compute budget, specialized intelligence will almost always beat general intelligence on the task it specializes in.
Or put differently: a small specialist can often match a much larger generalist.
And a mixture of specialists may be a much better scaling path than one monolithic intelligence.
There is no free lunch for general intelligence. Generality has a compute tax.
Value functions are severely underestimated in LLM post-training. People often count on compute scaling to compensate for the high variance of Monte Carlo estimators without credit assignment—broadcasting the same reward to all tokens.
Value functions should be revisited to push the frontier toward longer-horizon agentic tasks. We need to make value functions work, not simply throw them away.
Nice work from @MistralAI and @Siddarth on privileged value functions and TETHER.
LE CRITIQUE: Privileged Value Functions for LLM Reinforcement Learning
Value functions for LLMs have been grossly underutilized. I'm excited to finally share my work on Privileged Value Functions, done @MistralAI !
Our paper includes 2 massive improvements to value functions:
1) Privileged Value Functions: Turns out value functions can use hidden context unavailable to the policy for improved token-level RL signal! Condition them on reference solutions, verifier rubrics, or even other group samples and rewards. No more hacky self-distillation necessary! (being a tad hyperbolic)
2) TETHER: Do not trade off group and value baselines, you can just do both! We introduce a new baseline that adaptively trades off between the group mean and value baselines adaptively based on value function reliability.
🌐 privileged-value-functions.g…
📄 arxiv.org/pdf/2608.16739
Code: github.com/HyperPotatoNeo/pr…
🧵below
Minsu Kim retweeted
Let's bootstrap reasoning models by training on tasks with expert-level solutions (e.g., game solvers, robotic planners, theorem provers).
But, where do we get the CoT labels? Just asking another LM to impute the CoT doesn't work.
Solution: variational inference over CoT!
We've been thinking can we distill the knowledge humans encoded in expert systems into LLMs, and make it general?
Decades of human engineering live inside expert systems. Solvers, planners, schedulers: each one encodes real decision-making knowledge, but as a program that acts in a single domain and explains nothing.
Paper: arxiv.org/abs/2607.21856
More👇:
Minsu Kim retweeted
Can LLM agents figure out that their "private" chain of thought is being monitored, with no hints, no training, just from getting blocked?
Yes. And some then explicitly reason to hide their reasoning.
New paper with @FlemmingKondrup (co-first authors) & @Yoshua_Bengio: "Noticing the Watcher" 🧵
Minsu Kim retweeted
New work led by @FlemmingKondrup and @tomjiralerspong highlights an important vulnerability in chain-of-thought monitoring for agents, I highly recommend giving it a read.
Link to the paper: arxiv.org/pdf/2603.16928
Can LLM agents figure out that their "private" chain of thought is being monitored, with no hints, no training, just from getting blocked?
Yes. And some then explicitly reason to hide their reasoning.
New paper with @FlemmingKondrup (co-first authors) & @Yoshua_Bengio: "Noticing the Watcher" 🧵
Minsu Kim retweeted
Our Learning to Theorize paper received an award from the ICML Workshop in Compositional Learning
Minsu Kim retweeted
🇰🇷 If you’re at ICML: come check out our Meta FairChem + TUM poster exploring an idea:
Speculative decoding, but for simulation trajectories!
🗓️ Tue, Jul 7, 2026 • 2:00 PM – 3:45 PM KST
📍 HALL A #1312
w/ @guennemann, Ray (Meng) Gao, @bwood_m. 🙏 to the entire FAIRChem team!
Minsu Kim retweeted
at icml this week in seoul, please reach out for coffee, beer, etc. especially if you're a student (or a group of stduents) or postdoc (or a group of postdocs).
Minsu Kim retweeted
I'll be at ICML next week (7/6~11)!
Presenting "Learning to Theorize the World from Observation": oral Jul 9 16:15 Hall D2, poster 5 pm. Come by, I'd love to discuss.
Lately, I'm into world models, fixed-point dynamics, looped models. Happy to chat or catch up. DM me!☕️
🚀 We introduce Neural Theorizer (NEO) — a new type of world model that learns to theorize the world from observation, without language or LLM supervision.
Selected as an ICML 2026 oral presentation — 0.7% of submitted papers.
The paper asks:
"What does it mean to understand the world and build a world model?"
Today’s world models are often trained to predict the future: the next frame, next latent state, or next observation.
But is prediction enough?
We argue that a world model should be a theory-building system: one that discovers reusable primitives, composes them into executable explanations, and transfers those explanations to novel phenomena.
NEO is our first step toward this vision — a World Theory Model that learns explicit, compositional theories from raw observation.
This work was led by my wonderful students: Doojin Baek*(@doojin_a_baek), Gyubin Lee* (@gyubin0521), Junyeob Baek (@JunyeobB), and Hosung Lee (@HosungLee_).
For more details, take a look at the paper — and if you’re attending ICML, let’s talk there!
📄 arXiv: arxiv.org/abs/2605.03413
🌐 Project page: doojinbaek.github.io/publica…
Minsu Kim retweeted
Come and chat with us in Seoul at ICML!
Agentic Learning AI Lab @agentic_ai_lab presents at #ICML2026 in Seoul✈️🇰🇷 Come check out our latest research!
📍Main Conference Papers
Temporal Straightening for Latent Planning
Tue Jul 7 10:30am Hall A #1509
Context Tuning for In-Context Optimization
Wed Jul 8 5pm Hall A #2510
📍Workshop Papers
The Surprising Effectiveness of Deleting Weights in LLM Reasoning and Adaptation
Fri Jul 10 Rm 318 | Foundations of Deep Generative Models Workshop
Seeking the Unfamiliar but Memorable: Conceptual Creativity as Meta-Learning
Sat Jul 11 Rm 401 | Human-AI Co-Creativity Workshop
Aligning LLMs with Human Uncertainty: A Beta-Bernoulli Calibrator for LLM Forecasting
Sat Jul 11 Gr. Ballroom 104–105 | Forecasting as a New Frontier of Intelligence Workshop
Alive and Predicting: A Live Evaluation of Multi-Step Forecasting Agents
Sat Jul 11 Gr. Ballroom 104–105 | Forecasting as a New Frontier of Intelligence Workshop
See threads below for details!🧵👇