MTS @MicrosoftAI SuperIntelligence, Senior Researcher @ms_aifrontiers @MSFTResearch; Ex @HUMAIN @DreamFoldAI PhD @Mila_Quebec, World Models, RL Reasoning, VLMs
New York, USA
Joined November 2016
- Tweets624
- Following1.3K
- Followers1.8K
- Likes2K
Pinned Tweet
Excited to share that Agent-Controller representations for offline RL in presence of rich exogenous information is now accepted at #ICML2023 (arxiv.org/abs/2211.00164)
This is a follow-up of our recent work on latent state discovery (#TMLR'23) arxiv.org/abs/2207.08229 – at Montréal, Québec
This is a really interesting piece of work.
Been thinking about how to extract more signal out of the agent experience, and this work fits in nicely providing great insights on agent's affordances to learn from experience.
Riashat Islam retweeted
The connection between nextlat and FBT is stronger than one leads to the other: they structurally enable each other when you fold the former on top of the latter.
when i chat with @xidulu about full-bandwidth transformers back in july, i remember the idea being vaguely familiar. after digging through some messages, it turns out @JohnCLangford and I were chatting about this idea already back in November while writing NextLat!
NextLat was always a bit unsatisfactory to me. Yes, it can only teach the transformer to inherit the inductive bias of an RNN, and it can also be used to train a RNN without recurrence. However, it doesn’t actually provide the transformer teacher the circuit needed to escape its fixed-depth computation—to actually perform recurrence across time. we wanted a way to imbue transformers with recurrence without relying on implicit differentiation or Jacobian linearization. the Jacobi iteration approach of introducing recurrence in FBT turns out to be an extremely appealing to doing so! in my opinion, FBT will pan out better than the current ‘research local minima’ of looped transformers, i.e., models that loop in the layer dimension (as opposed to the time dimension).
it’s really nice to see FBT come into fruition, @xidulu did an extremely good work with the paper, and the updated results, and everyone should really go read it!
You all will miss the amazing works that were coming out of @ms_aifrontiers!
ECHO is going to #NeurIPS2026 as a spotlight! 🎉
TL;DR: When an agent acts, the environment responds. ECHO learns from that signal during RL.
Paper: arxiv.org/abs/2605.24517
Code: github.com/microsoft/echo-rl
Riashat Islam retweeted
NextLat is going to NeurIPS 🎉
Next-token prediction is myopic. What if transformers learn to predict their own next latent state?
🌠 We present 𝗡𝗲𝘅𝘁-𝗟𝗮𝘁𝗲𝗻𝘁 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 (𝗡𝗲𝘅𝘁𝗟𝗮𝘁): a self-supervised learning method that teaches transformers to form compact world models for reasoning and planning. It also unlocks up to 3.3x faster inference via self-speculative decoding! 🚀
ALT illustration of next-latent prediction vs. other predictive mechanisms
This is an awesome work from @murefil and other folks at MSFT Research Montreal!
(I have fond memories of this lab)
"On-policy" data: generating synthetic RL environments targeted to the learnability zone of the current policy and based on feedback from its rollouts will be a huge deal.
By continuously adapting synthetic generation of RL envs while training, we can get quite a strong coding model for a 4B dense backbone, without SFT from any teacher trajectories. I was surprised by the generalization to TB2, SwePro and PatchEval.
We stopped after 1500 tasks to put this report out. It remains to understand if and why slope will eventually flatten.
Great work by @kim__minseon @Zhengyan_Shi @Cote_Marc @ericxyuan and team
This is a really cool work!
What if a foundation model could tell us how to modify its architecture to boost inference and reasoning instantaneously—without retraining? What if that tweak incurred near zero latency cost during generation and supported indefinite state tracking? arxiv.org/abs/2608.17981
Really cool work from our team at MSR AI Frontiers!
1/ Sharing a new, interesting project I did during my internship at Microsoft AI Frontiers w/ @JohnCLangford
TL;DR: At decoding time, we feed **previous hidden state** into the input together with token embedding, and it boosts performance for free.
arxiv.org/abs/2608.08888
Riashat Islam retweeted
Kimi K3's RL loss has a pretty clean derivation!
Starting from a Mirror Descent formulation and plugging in a softmax form for the policy -- which is exactly how token probabilities are computed -- it naturally leads to a symmetric loss function that uses both clipping (for async train/inference mismatch) and KL penalization (for maintaining trust region properties).
Here's a single page note on this. I find it cool that a frontier level model is using a principled recipe without needing many bells and whistles :)
After Energy-Based Transformers, @AlexiGlad comes back with another really cool work!
We discovered a third pretraining axis beyond parameters and data: exploration.
Scaling exploration monotonically improves existing models across images/video/language, and unlocks end-to-end generation.
In the simplest case, it's just a for loop.
Introducing Explorative Modeling.
TLDR:
- Gains from exploration grow with scale: 7%→36% as data scales, 13%→23% as parameters scale, and gains double at 3× the compute
- Adding exploration to ~SOTA baselines improves data efficiency by 6.2×, FLOP efficiency by 4.1×, parameter efficiency by 47%, and hits a near-SOTA 1.43 unguided FID on ImageNet
- Exploration lets you trade training compute for generalization, and scales how end-to-end your generative model is
- End-to-end Explorative Models (XMs) match diffusion performance on control tasks with up to 256× less inference compute
🧵Thread:
Check out our poster on h1 and find @sumeetrm for a chat!
I'll be presenting 3 papers at ICML 2026🫡
h1 (Spotlight) trains models to reason over longer horizons using curriculum RL over composed short-horizon data. This allows models to generalize to harder tasks and improves performance even at high pass@k.
LongCoT isolates and benchmarks long-horizon CoT capabilities, pushing models to reason over tens and hundreds of thousands of tokens in an output.
Rubric Curriculum RL uses the generation-verification gap in non-verifiable domains to allow models to self-improve on tasks like creative writing and planning. More about this project to come soon!
Very grateful for amazing collaborators @CharlieLondon02 @bartoldson @DanielNichols10 @AmyPrb @hammh0a @jackcai1206 @bkailkhu @casdewitt @sytelus @riashatislam @philiptorr, and especially co-leads Alesia and Tejas!
If you're interested in these topics, my DMs are open :)
This is really cool work - you know this work has been thought through when you see @EfroniYonathan on it! Congrats to @Ankur_Samanta_ for the hard work in pulling it off too!
Riashat Islam retweeted
New work from our lab on Next-Latent Prediction and stronger world representations for agents, paving the way for more capable long-horizon reasoning and decision-making.
Well done @jayden_teoh_ @manan_tomar @KwangjunA @edward_s_hu @Tea_Pearce @pratyusha_PS Akshay Krishnamurthy @riashatislam @lamblabtsinghua @JohnCLangford
Next-token prediction is myopic. What if transformers learn to predict their own next latent state?
🌠 We present 𝗡𝗲𝘅𝘁-𝗟𝗮𝘁𝗲𝗻𝘁 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 (𝗡𝗲𝘅𝘁𝗟𝗮𝘁): a self-supervised learning method that teaches transformers to form compact world models for reasoning and planning. It also unlocks up to 3.3x faster inference via self-speculative decoding! 🚀
ALT illustration of next-latent prediction vs. other predictive mechanisms
Riashat Islam retweeted
Next-token prediction is myopic. What if transformers learn to predict their own next latent state?
🌠 We present 𝗡𝗲𝘅𝘁-𝗟𝗮𝘁𝗲𝗻𝘁 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 (𝗡𝗲𝘅𝘁𝗟𝗮𝘁): a self-supervised learning method that teaches transformers to form compact world models for reasoning and planning. It also unlocks up to 3.3x faster inference via self-speculative decoding! 🚀
ALT illustration of next-latent prediction vs. other predictive mechanisms
A huge loss for the RL, Control and Optimization communities. A legend passed way.
Deeply saddened at the passing of my dear colleague, Dimitri Bertsekas.
Everyone in RL, OR and control theory already knows of his monumental contributions. Over the past seven years, we at @SCAI_ASU also got to know him as an unwaveringly kind and gracious man of science.
He truly enjoyed his research and has remained active all through; uploading two pre-prints to arXiv just in the past month!
While I fully expected him to continue working for years to come, I also know that his contributions and books will be cherished by generations..
RIP 🙏
Great work @sumeetrm!
Riashat Islam retweeted
LLMs can learn to manage their own context on some quite difficult thinking-required problems.
Our work on Next Latent Prediction, led by @jayden_teoh_ is a step towards this new pre-training paradigm.
Jayden did an amazing internship with MSR AI Frontiers!
arxiv.org/abs/2511.05963
Please apply to join our team at MSR AI Frontiers
We are hiring for a researcher in the foundations of generative AI: apply.careers.microsoft.com/… .
Please share with whoever may be interested.
Riashat Islam retweeted
Think of orchestration as search over thoughts. Then train the model to match that orchestration. This allows shifting the accuracy↔latency/context tradeoff by compressing search into weights.
I'm glad this paper of ours is getting attention. It shows that there are more efficient and effective ways for models to use their thinking tokens than generating a long uninterrupted thinking trace. Our PDR (parallel/distill/refine) orchestration gives much better final accuracy, while avoiding context bloat. (So it might be much cheaper to serve than today's thinking models.) I'm guessing that so-called "deep research models" rely on such orchestrations.