Pinned Tweet
How can we predict multiple plausible targets from a single context in joint-embedding self-supervised learning (SSL)?
Check out our paper titled “Self-Supervised Learning from Structural Invariance” accepted at #ICLR2026! Previously Best Paper Award at @unireps 2025.
arxiv.org/abs/2602.02381
We introduce AdaSSL, which models the target uncertainty and relaxes the standard assumption that the positive pair share the same semantic features.
Derived from first principles, we realize @ylecun’s JEPA with a learned latent variable for jointly learning better representations and world models, extending SSL’s utility to a broader range of data types.
1/🧵
Yipeng Zhang retweeted
Mila's annual supervision request process is now open to receive MSc and PhD applications for Fall 2027 admission! For more information, visit mila.quebec/en/prospective-s…
Yipeng Zhang retweeted
LE CRITIQUE: Privileged Value Functions for LLM Reinforcement Learning
Value functions for LLMs have been grossly underutilized. I'm excited to finally share my work on Privileged Value Functions, done @MistralAI !
Our paper includes 2 massive improvements to value functions:
1) Privileged Value Functions: Turns out value functions can use hidden context unavailable to the policy for improved token-level RL signal! Condition them on reference solutions, verifier rubrics, or even other group samples and rewards. No more hacky self-distillation necessary! (being a tad hyperbolic)
2) TETHER: Do not trade off group and value baselines, you can just do both! We introduce a new baseline that adaptively trades off between the group mean and value baselines adaptively based on value function reliability.
🌐 privileged-value-functions.g…
📄 arxiv.org/pdf/2608.16739
Code: github.com/HyperPotatoNeo/pr…
🧵below
Yipeng Zhang retweeted
I’ll be presenting this work during ICML this upcoming week🇰🇷
Reach out if you’d like to chat anything to do with RL post-training, compaction and agents!
Or come say hi Wednesday 2:30-4:30
Remember all the self-distillation papers that came out last week. Well, we also propose it 😅, but…
But alongside something better 😎 π-Distill
We show that with this method, you can distill closed-source frontier models even tho their traces are hidden 🔒.
Both our methods can reach and even surpass the performance of the industry-standard SFT + RL with access to reasoning traces 🤯.
🔬And we spent ~100,000 hours GPU hours on a comprehensive analysis, not because the method is finicky, but because we wanted to understand why it works so well.
🧵
1/10
Yipeng Zhang retweeted
We’re coming out of stealth with some exciting AI-optimized compute results.
Some highlights:
1) Works on industrial-grade and consumer-grade GPUs.
2) It can optimize different compute layers, from the kernels to tokens, for higher tokens-per-watt on your GPU.
The entire industry is racing to make AI cheaper. Most people assume the answer is better chips or bigger GPU clusters. However, while the industry is buying its next 2× when there's a 1.78× speedup is sitting in the software.
We've been hard at work @unifiedsciences, building an autonomous optimization engine toward radically more efficient compute.
Today I'm very happy to share some early results where our unified engine:
1) wrote faster GEMM kernels for cuBLAS (up to 1.78x faster)
2) made an efficient AI model 6x smaller and 5x faster without any quality loss
3) optimized diverse everyday AI workloads on a consumer GPU!
Yipeng Zhang retweeted
Excited to share Gigapixel: a high-throughput driving simulator for scalable self-play training of E2E driving models: montrealrobotics.ca/gigapixe…. To make E2E training feasible, we propose self-play DAgger, which trains robust policies in ~3000x fewer steps than self-play RL, achieving SOTA on closed-loop driving benchmarks without human trajectory supervision!
Yipeng Zhang retweeted
It’s surprisingly easy to improve intuitive physics in video generative models with a powerful latent world model ;)
Talk to @DYDYYDYYYD @adri_romsor @michal_drozdzal if you are at #CVPR
1/6 📌 Excited to share our #CVPR2026 Highlight: "Inference-time Physics Alignment of Video Generative Models with Latent World Models"
We introduce WMReward an inference-time framework that steers video generators toward physically plausible outputs
📄arxiv.org/abs/2601.10553
Yipeng Zhang retweeted
Giving a talk at ICLR MemAgents Workshop tomorrow, 11:25am local, Room 205: "Does Your LLM Agent Have a Self?" Spoiler: probably not — but the reason is more nuanced than "they're just LLMs." Hope to see you there!
Yipeng Zhang retweeted
I’ll present our paper on how prompt complexity change the generated distribution in T2I models (arxiv.org/abs/2510.19557) at ICLR 26!
Come and chat at Pavillon 4 - #3114 tomorrow 23 April 3PM :)
Yipeng Zhang retweeted
Presenting Future Summary Prediction at #ICLR2026 🇧🇷
📌 Friday, 24th April, 10.30 am
📍Pavilion 3 (521)
Come over to chat about novel pretraining objectives for LLMs!
[1/9] While pretraining data might be hitting a wall, novel methods for modeling it are just getting started!
We introduce future summary prediction (FSP), where the model predicts future sequence embeddings to reduce teacher forcing & shortcut learning.
📌Predict a learned embedding of the future sequence, not the tokens themselves
Yipeng Zhang retweeted
I’m in Rio presenting this work at the SPOT workshop 🇧🇷☀️!
Feel free to reach out to chat about self distillation/privileged information and/or anything to do with post-training/long horizon agents.
Remember all the self-distillation papers that came out last week. Well, we also propose it 😅, but…
But alongside something better 😎 π-Distill
We show that with this method, you can distill closed-source frontier models even tho their traces are hidden 🔒.
Both our methods can reach and even surpass the performance of the industry-standard SFT + RL with access to reasoning traces 🤯.
🔬And we spent ~100,000 hours GPU hours on a comprehensive analysis, not because the method is finicky, but because we wanted to understand why it works so well.
🧵
1/10
Yipeng Zhang retweeted
JEPA are finally easy to train end-to-end without any tricks!
Excited to introduce LeWorldModel: a stable, end-to-end JEPA that learns world models directly from pixels, no heuristics.
15M params, 1 GPU, and full planning <1 second.
📑: le-wm.github.io
Yipeng Zhang retweeted
Come check out the talk for a deep breakdown of my recent work/blog :)
NICE Talk 148🌟 invites @emilianopp_, a PhD student at Mila-Quebec & Université de Montréal, to discuss how LLMs can learn from privileged information during training — without needing it at test time.
📖 Paper: Privileged Information Distillation for Language Models — [arxiv.org/pdf/2602.04942]
⏰ Time:
3.20 (Fri) 9:00 PM - 10:00 PM EDT
3.20 (Fri) 6:00 PM - 7:00 PM PDT
📌 Register: luma.com/dll9x6f5
📌 Watch live: youtube.com/watch?v=SUb4M73q…
✨This talk is hosted by @Haolun_Wu0203, Ph.D. at Mila & McGill
What if your model could train with a "cheat sheet" — but still ace the test without it? Emiliano presents Privileged Information Distillation, a unified post-training framework that bridges the gap between hinted training and non-privileged inference.
⭐ Key findings:
🧐 Privileged information during training significantly boosts LLM performance — but design choices matter enormously for generalization;
🤠 A variational framework + on-policy distillation outperforms strong baselines including SFT + GRPO;
🤪 Most surprisingly, not all privileged information is equal — the right hints incentivize generalization, while the wrong ones don't.
#AI #LLM #PrivilegedInformation #Distillation #PostTraining #Reasoning #NICE #NexusForIntelligence
Yipeng Zhang retweeted
Are frontier LLMs trained across datacenters? One thing is certain: if the pre-training optimizer’s critical batch size is too small, they are NOT! Excited to announce MuLoCo, a pre-training optimizer that can efficiently pre-train across datacenters while having large enough batch sizes to warrant doing so. 🧵1/N
Yipeng Zhang retweeted
World Modeling research needs fast iteration, reproducibility, optimized baselines, open-source, and precise zero-shot stress testing. Here comes stable-worldmodel!
Paper: arxiv.org/abs/2602.08968
Code: github.com/galilai-group/sta…
Come stress-test your model/idea!
DINO-WM results ⬇️
Yipeng Zhang retweeted
I had a lot of fun meeting all the smart people at this workshop and presenting my work "On the Identifiability of Latent Action Policies" as an oral! A huge thanks to the organizers!
Paper: arxiv.org/abs/2510.01337
What an awesome first day! Thank you all for joining and listening to our amazing speakers: @SchmidhuberAI, @sherryyangML, @cosmo_shirley, @Yoshua_Bengio, @ylecun, @mido_assran
World Models have beautiful days ahead. This is just the beginning 🫡
Yipeng Zhang retweeted
Remember all the self-distillation papers that came out last week. Well, we also propose it 😅, but…
But alongside something better 😎 π-Distill
We show that with this method, you can distill closed-source frontier models even tho their traces are hidden 🔒.
Both our methods can reach and even surpass the performance of the industry-standard SFT + RL with access to reasoning traces 🤯.
🔬And we spent ~100,000 hours GPU hours on a comprehensive analysis, not because the method is finicky, but because we wanted to understand why it works so well.
🧵
1/10
Yipeng Zhang retweeted
What an awesome first day! Thank you all for joining and listening to our amazing speakers: @SchmidhuberAI, @sherryyangML, @cosmo_shirley, @Yoshua_Bengio, @ylecun, @mido_assran
World Models have beautiful days ahead. This is just the beginning 🫡
I'm at @worldmodel_26 now through Friday. Lmk if you want to chat!
How can we predict multiple plausible targets from a single context in joint-embedding self-supervised learning (SSL)?
Check out our paper titled “Self-Supervised Learning from Structural Invariance” accepted at #ICLR2026! Previously Best Paper Award at @unireps 2025.
arxiv.org/abs/2602.02381
We introduce AdaSSL, which models the target uncertainty and relaxes the standard assumption that the positive pair share the same semantic features.
Derived from first principles, we realize @ylecun’s JEPA with a learned latent variable for jointly learning better representations and world models, extending SSL’s utility to a broader range of data types.
1/🧵