Nick Stracke ✈️ CoRL 26 (@rmsnorm) profile picture

PhD Student at Ommer Lab (Stable Diffusion) Working on video analysis and world understanding...

Munich
Joined July 2024
Photo by Nick Stracke ✈️ CoRL 26 (@rmsnorm)Photo by Nick Stracke ✈️ CoRL 26 (@rmsnorm)Photo by Nick Stracke ✈️ CoRL 26 (@rmsnorm)Photo by Nick Stracke ✈️ CoRL 26 (@rmsnorm)Photo by Nick Stracke ✈️ CoRL 26 (@rmsnorm)Photo by Nick Stracke ✈️ CoRL 26 (@rmsnorm)Photo by Nick Stracke ✈️ CoRL 26 (@rmsnorm)Photo by Nick Stracke ✈️ CoRL 26 (@rmsnorm)Photo by Nick Stracke ✈️ CoRL 26 (@rmsnorm)Photo by Nick Stracke ✈️ CoRL 26 (@rmsnorm)
Nick Stracke ✈️ CoRL 26 retweeted
Directly predict the distribution over possible futures in one forward pass. To find out how, come by our poster at #ECCV in #Malmö today. Poster session 3 - Poster 101. See you at 10:30 am!
People often think of prediction as choosing a likely outcome. But that’s not how the world works! Given partial information, the future is a distribution of possibilities. And more knowledge collapses the distribution. We built a representation that models exactly that 🧵
4
1
10
4,174
Nick Stracke ✈️ CoRL 26 retweeted
People often think of prediction as choosing a likely outcome. But that’s not how the world works! Given partial information, the future is a distribution of possibilities. And more knowledge collapses the distribution. We built a representation that models exactly that 🧵
1
8
1
15
1,479
Nick Stracke ✈️ CoRL 26 retweeted
Most representation learning methods produce generic embeddings that try to preserve everything in an image. We introduce VICIS: a way to use example sets to define a tailored embedding space for what matters. This is useful whenever the desired visual signal is easier to show than to describe. Code & weights available now 🧵👇
1
15
19
1,348
Nick Stracke ✈️ CoRL 26 retweeted
datasets .bot now supports filtering by dataset size! There are 10 datasets with 1,000+ hours of data. Happy training 🏋🏽 shoutout @rmsnorm for the feature request
4
3
25
5,081
Nick Stracke ✈️ CoRL 26 retweeted
Now accepted to #ECCV2026 🎉
The internet is full of video. So why can't novel view synthesis just scale on it? Real-world video is simultaneously unposed, messy, and dynamic, breaking self-supervised NVS. We fixed that. RayDer learns static-scene NVS from dynamic internet video, scaling like an LLM. A🧵
4
7
69
10,284
Nick Stracke ✈️ CoRL 26 retweeted
Another crazy CVPR 2026 world model result. “Envisioning the Future” forecasts where points in a scene will move, step by step, from a single image. No dense video needed. - 3,000x faster than video models, 10x fewer parameters, and 5x more accurate under a fixed compute budget. - An autoregressive diffusion model rolls sparse point trajectories forward through short, predictable steps, modeling uncertainty as it grows. - Why it matters: Rollouts get cheap enough to simulate thousands of futures and plan over them, hitting 78% billiard planning accuracy vs 16% for the best dense video baseline.
33
231
12,134
Nick Stracke ✈️ CoRL 26 retweeted
second runner up is a super cool paper from Björn Ommer’s group at LMU. they first encode point trajectories into a latent using a VAE. then they learn to denoise future trajectories conditioned on past. they do planning experiments on Libero by training a small decoder to convert denoised latents to robot action. I joked that this is latent stable diffusion all over again by Björn but this time with point trajectories 😄
1
3
23
1,498
Nick Stracke ✈️ CoRL 26 retweeted
ZipMo hands a robot its plan by daydreaming how the scene should move. No pixels, no rendering, no hour-long video to sit through. 10,000x faster than a top video model! Just a fast read on where everything's headed, and the robot runs with it. Give it a start frame and a goal in plain language. It predicts how the arm and the objects around it should move to get there. A thin policy head reads that motion plan and turns it into the next arm command. The head never sees the task, only the predicted motion. So all the reasoning lives in the motion model. The head is pure inverse dynamics, motion in, action out. On the LIBERO benchmark it replans on every new frame, and it beats the trajectory-based policy methods it lines up against: ATM, Tra-MoE, Amplify. It pulls this off by never touching pixels. It learns a compact motion space from tracked trajectories, then generates motion straight inside that space. And stranger still, compressing that space 64x makes the motion sharper, not worse. Congrats to the team, lovely work. @rmsnorm, @KoljaBauer, @StefanABaumann
Come visit our poster today (Sunday) 3:30 pm at CVPR poster 595! We'll also show some new results on translating the generated motion embeddings back to video 👇
1
6
1
11
1,500
Nick Stracke ✈️ CoRL 26 retweeted
You don't need a video model to predict motion. We'll talk about how to be 3000× faster at CVPR Poster 634, this morning at 10:45. Drop by and check it out!
You don't imagine the future by mentally rendering a movie. You trace how things move -- abstractly, sparsely, step by step. We built a model that does exactly this. It predicts motion, not pixels -- and it's 3,000× faster than video world models. Myriad, accepted at @CVPR 2026
1
6
1
44
5,402
Nick Stracke ✈️ CoRL 26 retweeted
⚠️ Standard first stages are not sufficient for safety-critical applications! The most extreme weather events are often the hardest to decode. One latent → many plausible reconstructions Deterministic decoders hide that uncertainty. Meet FREUD 🧵👇
2
7
2
10
1,450
Nick Stracke ✈️ CoRL 26 retweeted
Cool stuff
Video diffusion models learn motion indirectly through pixels. But motion itself is much lower-dimensional. We introduce 64× temporally compressed motion embeddings that directly capture scene dynamics. This enables efficient planning -> 10,000× faster than video models. 🧵👇
4
33
7,733
Nick Stracke ✈️ CoRL 26 retweeted
Amazing work led by @rmsnorm @KoljaBauer and our collaborators at LMU, to be presented at @CVPR! Personally, I find this question of "what's the right level of abstraction for planning in physical space?" to be very intriguing. Pixels over time are very low SNR (ie. the argument behind JEPA) but motion/trajectories carries a lot on information while being extremely compressible. I believe there's a lot more to uncover from this direction. Very glad to be part of this one!
Video diffusion models learn motion indirectly through pixels. But motion itself is much lower-dimensional. We introduce 64× temporally compressed motion embeddings that directly capture scene dynamics. This enables efficient planning -> 10,000× faster than video models. 🧵👇
1
3
14
1,900
Nick Stracke ✈️ CoRL 26 retweeted
A massive step forward for AI video!
Video diffusion models learn motion indirectly through pixels. But motion itself is much lower-dimensional. We introduce 64× temporally compressed motion embeddings that directly capture scene dynamics. This enables efficient planning -> 10,000× faster than video models. 🧵👇
3
25
5,144
Nick Stracke ✈️ CoRL 26 retweeted
I have been saying
Video diffusion models learn motion indirectly through pixels. But motion itself is much lower-dimensional. We introduce 64× temporally compressed motion embeddings that directly capture scene dynamics. This enables efficient planning -> 10,000× faster than video models. 🧵👇
1
22
5,374
Nick Stracke ✈️ CoRL 26 retweeted
1️⃣xitter.cf/neerjathakkar/status/2… Also, shoutout to two other recent works that explore how to use point tracks for world modeling. 👇...
What’s the right representation for a world model? 3D, pixels, or something else? Excited to release our new paper “Forecasting Motion in the Wild” where we propose point tracks as tokens for generating complex non-rigid motion and behavior From @GoogleDeepmind @Berkeley_AI @TTIC_Connect
1
4
20
2,208
Nick Stracke ✈️ CoRL 26 retweeted
Do we really need pixel generation to model motion? 🤔 We show how directly representing motion in a compact space enables efficient, scalable planning. 10,000× faster than video models, enabling planning and reasoning in open-world and robotics settings. Check it out ⬇️
Video diffusion models learn motion indirectly through pixels. But motion itself is much lower-dimensional. We introduce 64× temporally compressed motion embeddings that directly capture scene dynamics. This enables efficient planning -> 10,000× faster than video models. 🧵👇
1
4
31
3,759
Nick Stracke ✈️ CoRL 26 retweeted
You don't imagine the future by mentally rendering a movie. You trace how things move -- abstractly, sparsely, step by step. We built a model that does exactly this. It predicts motion, not pixels -- and it's 3,000× faster than video world models. Myriad, accepted at @CVPR 2026
4
58
6
357
32,591
Nick Stracke ✈️ CoRL 26 retweeted
14
52
3
640
49,155
Nick Stracke ✈️ CoRL 26 retweeted
🤔 What if you could generate an entire image using just one continuous token? 💡 It works if we leverage a self-supervised representation! Meet RepTok🦎: A generative model that encodes an image into a single continuous latent while keeping realism and semantics. 🧵👇
8
23
4
111
20,330