Pinned Tweet
Excited to introduce PolaRiS, a real-to-sim recipe for turning short real-world videos into high fidelity simulation environments for scalable and reliable zeroshot generalist policy evaluation.
polaris-evals.github.io
(1/N 🧵)
sim2real goat
This robot has never seen a real-life gear, but assembles one with a 94% success rate! It can also thread an M16 nut and insert a 16mm peg. All trained with RL and transferred zero-shot sim2real from RGB. 🔈
Our method, Success-Guided Sampling (SGS), makes a small change to how we sample environment resets. More on SGS in @octi_zhang's thread, and more on sim2real transfer down below! 🧵
Website: sgs-rl.github.io/
Arhan Jain retweeted
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before.
The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices.
Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL.
No demonstrations, no per-task reward tuning. Just PPO at scale.
Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
Arhan Jain retweeted
Our manipulation results were mind-blowing the first time I saw them, but earlier on, my favorite result was this locomotion run. Many great people have done parkour-type skills with locomotion already, but as far as I am aware, most, if not all, of these works trained separate policies for each terrain, and then distilled into a single policy. They also typically contained tons of hand-engineered rewards and curriculums.
A very early motivation for SGS was to simply figure out if we could get similar results but get rid of as much of this hand-engineering as possible, and we managed to achieve that! This video shows a continuous 1-min run in much harder terrains than ever before, all solved by the same policy.
Arhan Jain retweeted
I'll be presenting TMRL at #RSS2026!
Thu 7/16, 3:15 PM @ Imitation Learning 3 Session
Fri 7/17, 5:30–6:00 PM @ Diffusion for Robot Learning Workshop(rss2026-diffusion-robot-lear…)
Stop by our poster after Thursday's session, happy to talk real-world RL, or pre-training for post-training broadly. Come say hi 👋
xitter.cf/matthewh6_/status/2075…
RL can't teach an LLM to solve problems it never solves. Robot policies suffer from exactly the same limitation.
The fix turned out to be almost embarrassingly simple: train the policy with diffusion noise during pre-training.
That's it.
The policy covers a much wider action distribution, RL finally has somewhere to search, and we fine-tune VLAs on real robots in under an hour.
Introducing TMRL.
🧵(1/9)
amazing progress on real2sim evals 🤠
Today, we're introducing SimFoundry, our real2sim2real framework at NVIDIA GEAR that automatically turns real-world scenes into simulation-ready worlds from a single image or video.
Website: research.nvidia.com/labs/gea…
Paper: arxiv.org/abs/2606.28276
This work marks a major step for our team toward leveraging simulations and synthetic data for foundation model training and systematic policy evaluation at scale. Code will be open-sourced soon. Stay tuned!
great work by @yu_raymond5 and @willhuey9 on sim2real & posttraining 😼
Policies trained on real robot data via imitation can be surprisingly capable. But for domains like dexterous manipulation, they are often not quite good enough: they move slowly, miss grasps, make unreliable contact, and fail under small perturbations.
Can we improve them without any additional data collection on the real robot?
In SCORE, we show that we can improve real-world diffusion/flow policies cheaply by using simulation to simply learn how to steer them on deployment. This leads to large gains in real-world success and speed across a variety of tasks, without requiring additional real-world experience:
weirdlabuw.github.io/score/
🧵 (1/10)
Arhan Jain retweeted
🔥 Coding agents have become one of the hottest LLM workloads. But serving them looks nothing like serving a chatbot: 294× more input than output, hundreds of thousands of tool calls, and extremely long-tailed latency.
🚀 We are releasing the SyFI Coding Trace: ~4,300 real-world coding-agent sessions from our daily use, plus TraceLab, an open-source pipeline to collect, sanitize, analyze, and replay your own traces.
More in the thread below 🧵👇 (1/n)
Arhan Jain retweeted
Remember action recognition? The days of trying to climb on Kinetics?👻
Announcing VideoNet, a CVPR 2026 Highlight 🎉 which revitalizes action recognition in the VLM era
Explore our data with this fun, interactive demo: tanu.sh/videonet/data
(1/8) 🧵
Arhan Jain retweeted
We’re releasing OmniReset, a framework for training robot policies using large-scale RL and diverse resets for contact-rich, dexterous manipulation.
OmniReset pushes the frontier of robustness and dexterity, without any reward engineering or demonstrations.
Try the policies yourself in our interactive simulator! weirdlabuw.github.io/omnires…
(1/N 🧵)
Very cool use of PolaRiS by @jackyk02 and @XilunZhang1999 as a testbed for their new work in scaling verifiers for robot policies‼️
Arhan Jain retweeted
Excited to share CoVer-VLA—a fully self-supervised action verifier for VLA models and the first work of my PhD! 🤖
We developed a lightweight verifier that assesses VLA action quality by aligning actions with text-visual features. Best of all? It requires zero failure data and scales seamlessly to large robotics datasets.
Beyond verification, CoVer learns aligned action representations via contrastive learning—opening doors for more downstream robotics tasks such as data curation and OOD detection! 🚀
Huge thanks to my amazing collaborators and advisors, and a special shout-out to @prodarhan for the help with PolaRis! Truly an incredible platform.
Please check out more details in the post, and try to CoVer your VLA policy!
Introducing CoVer-VLA💫— a contrastive verifier + hierarchical test-time scaling framework for VLAs!
- Lightweight 1B verifier 🧠
- Outperforms π₀ & π₀.₅ 🦾
- Trained on Bridge & DROID 🤖
Turns out scaling verification > scaling policy learning for VLA alignment!
🧵👇
🌐 Website: cover-vla.github.io
📄 Paper: arxiv.org/abs/2602.12281
🤗 Models: huggingface.co/cover-vla
💻 Code: github.com/cover-vla/cover-v…
Arhan Jain retweeted
How can robot policies be trained to best leverage VLMs' CoT reasoning and in-context learning for generalization?
The key is Steerable Policies: vision-language-action models that can be flexibly controlled in many ways!
steerable-policies.github.io
1/9
Arhan Jain retweeted
Meet SceneSmith: An agentic system that generates entire simulation-ready environments from a single text prompt.
VLM agents collaborate to build scenes with dozens of objects per room, articulated furniture, and full physics properties.
We believe environment generation is no longer the bottleneck for scalable robot training and evaluation in simulation.
Website: scenesmith.github.io/
👇🧵(1/8)
Arhan Jain retweeted
Building sim environments is hard -- especially ones that are useful for assessing progress on real-world performance. With PolaRiS, you can scan an environment for 2-5 minutes and use 2d gaussian splatting + some tools to create a high fidelity simulation.
Evaluating robot policies is hard. Ideally, instead of testing every new policy on a real robot, you could test in simulation; but simulations rarely correlate well with real-world performance. In order to make good, useful simulations, you need to spend a great deal of time and effort.
That’s where PolaRiS comes in: it’s a toolkit that lets you take a short video of a real scene and turn it into a high-fidelity simulation. It provides what you need to build a good evaluation environment, and it “ships” with off-the-shelf environments that already show strong sim-to-real correlation, meaning that they can be used to inform policy performance.
@prodarhan and @KarlPertsch join us to talk about what they have built, why, and how you can use it.
Watch Episode #62 of RoboPapers, with @chris_j_paxton and @DJiafei, now!
Checkout our deep dive with @chris_j_paxton and @DJiafei on using simulation to faithfully evaluate generalist policies in digital twins!🤠
Evaluating robot policies is hard. Ideally, instead of testing every new policy on a real robot, you could test in simulation; but simulations rarely correlate well with real-world performance. In order to make good, useful simulations, you need to spend a great deal of time and effort.
That’s where PolaRiS comes in: it’s a toolkit that lets you take a short video of a real scene and turn it into a high-fidelity simulation. It provides what you need to build a good evaluation environment, and it “ships” with off-the-shelf environments that already show strong sim-to-real correlation, meaning that they can be used to inform policy performance.
@prodarhan and @KarlPertsch join us to talk about what they have built, why, and how you can use it.
Watch Episode #62 of RoboPapers, with @chris_j_paxton and @DJiafei, now!
Arhan Jain retweeted
A key bottleneck to fast deployment is understanding how you'll fail in a new env.
We need ways to spin up rapid simulations / evaluations of robot policies in new environments if we want to keep high iteration speed.
@prodarhan and @KarlPertsch are building something cool.
Evaluating robot policies is hard. Ideally, instead of testing every new policy on a real robot, you could test in simulation; but simulations rarely correlate well with real-world performance. In order to make good, useful simulations, you need to spend a great deal of time and effort.
That’s where PolaRiS comes in: it’s a toolkit that lets you take a short video of a real scene and turn it into a high-fidelity simulation. It provides what you need to build a good evaluation environment, and it “ships” with off-the-shelf environments that already show strong sim-to-real correlation, meaning that they can be used to inform policy performance.
@prodarhan and @KarlPertsch join us to talk about what they have built, why, and how you can use it.
Watch Episode #62 of RoboPapers, with @chris_j_paxton and @DJiafei, now!
Arhan Jain retweeted
Fun chat and great insights from @prodarhan and @KarlPertsch on evaluation!
Full episode dropping soon!
Geeking out with @prodarhan @KarlPertsch on PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies polaris-evals.github.io/
Co-hosted by @chris_j_paxton @DJiafei
Arhan Jain retweeted
Full episode dropping soon!
Geeking out with @prodarhan @KarlPertsch on PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies polaris-evals.github.io/
Co-hosted by @chris_j_paxton @DJiafei
