Robotics Research Intern @NvidiaAI; Robot Learning PhD student @UMRobotics; undergrad @imperialcollege
Joined April 2023
- Tweets168
- Following382
- Followers348
- Likes2.1K
Pinned Tweet
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗adept-dexterity.github.io with Dex team @NVIDIAAI
Jayjun Lee retweeted
Teaching a robot to insert a peg shouldn't erase its ability to pick the peg up. Yet that's exactly what standard RL fine-tuning did.
ADEPT, from @jayjunleee, @ankurhandos and the @nvidia /@UMich dexterity team, has one of the more useful ablations I've seen on robot skill transfer.
First they spend 8 BILLION simulated environment steps teaching a 23-DoF arm-hand system a deliberately generic task: pick up, reorient and reposition 16 randomized primitive shapes. No peg-insertion demonstrations.
The resulting policy already knows how to reach, grasp and rotate unfamiliar pegs. But it doesn't know how to insert them.
Now the nasty bit. Change the reward to insertion and run ordinary PPO fine-tuning. The pretrained grasping behavior collapses almost immediately.
Their repair is surprisingly surgical: distill the old actor into the new observation space, freeze it while a fresh critic learns the new reward, then update the actor VERY slowly.
In five controlled training seeds:
Standard PPO actor LR (10⁻³): 0/5 reached the final insertion curriculum.
Conservative LR (10⁻⁵), with distillation + critic warm-up: 5/5 reached it.
And the ablation shows the smaller learning rate mattered more than tightening PPO's clipping threshold. This is not a generic argument for turning every hyperparameter down.
They then distill the simulated teacher into camera-based policies and deploy with NO real-robot fine-tuning. On a physical 29-DoF Flexiv-Sharpa hand, adding five fingertip tactile sensors changed peg-insertion success from 3/10 to 8/10. The tactile version could tell when it had actually secured the object instead of repeatedly reopening a successful grasp.
Caveats: only 10 hardware trials per condition, each task still gets its own specialist, and the deployed policies run on an external workstation.
The bigger idea: robot pretraining is only valuable if post-training doesn't bulldoze the skills it paid billions of simulation steps to acquire.
Maybe we should measure how much dexterity survives fine-tuning, not just how fast the new benchmark improves.
adept-dexterity.github.io/
#RobotLearning #DexterousManipulation #Sim2Real
Jayjun Lee retweeted
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?
That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.
Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.
Paper: arxiv.org/abs/2610.13354
Website: alpha-nist.github.io
Jayjun Lee retweeted
This robot has never seen a real-life gear, but assembles one with a 94% success rate! It can also thread an M16 nut and insert a 16mm peg. All trained with RL and transferred zero-shot sim2real from RGB. 🔈
Our method, Success-Guided Sampling (SGS), makes a small change to how we sample environment resets. More on SGS in @octi_zhang's thread, and more on sim2real transfer down below! 🧵
Website: sgs-rl.github.io/
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before.
The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices.
Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL.
No demonstrations, no per-task reward tuning. Just PPO at scale.
Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
Jayjun Lee retweeted
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before.
The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices.
Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL.
No demonstrations, no per-task reward tuning. Just PPO at scale.
Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
Jayjun Lee retweeted
Super interesting to see that the #SharpaWave hands beat parallel grippers on peg insertion: 5-10s vs 20-70s per trial. Less need for precise orientation at grasp, faster in-hand rotation, faster overall. And also that it confirms the trend we're seeing with #TacMap (co-authored with Sharpa) and T-Rex (from @Dantong_Niu
): tactile policies hit 8/10 success vs 3/10 vision-only. Wave's per-fingertip sensors simulate well in IsaacLab, rich contact data in training.
Our new work with Jayjun explores sim-first approach to pre-training and post-training. There is a lot to squeeze out of simulations and we still don't know the ceiling and this work is a step in that direction. Ideally, you want to work on problems that are compute bound or alternatively figure out a way to turn your problems into compute bound problems and training in simulations at scale is a compute bound problem with lots of opportunities.
This work also explores training visuo-tactile policies in simulation. @SharpaRobotics hand has 5 tactile sensors at the finger-tips and we simulate them in IsaacLab to provide rich interactive data in the loop.
We transferred policies zero-shot on two different arm-hand combinations and importantly we didn't tune any hyperparams associated with RL training or reward. Many tasks can be defined as goal reaching problems where the robot has to reach for the object, lift, transport and bring it to a desired goal and this is embodiment agnostic and in simulations, you can both generate and verify goals.
Jayjun Lee retweeted
Our new work with Jayjun explores sim-first approach to pre-training and post-training. There is a lot to squeeze out of simulations and we still don't know the ceiling and this work is a step in that direction. Ideally, you want to work on problems that are compute bound or alternatively figure out a way to turn your problems into compute bound problems and training in simulations at scale is a compute bound problem with lots of opportunities.
This work also explores training visuo-tactile policies in simulation. @SharpaRobotics hand has 5 tactile sensors at the finger-tips and we simulate them in IsaacLab to provide rich interactive data in the loop.
We transferred policies zero-shot on two different arm-hand combinations and importantly we didn't tune any hyperparams associated with RL training or reward. Many tasks can be defined as goal reaching problems where the robot has to reach for the object, lift, transport and bring it to a desired goal and this is embodiment agnostic and in simulations, you can both generate and verify goals.
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗adept-dexterity.github.io with Dex team @NVIDIAAI
Jayjun Lee retweeted
Fantastic work by @jayjunleee and the @NVIDIARobotics Dex research group. Always great to see new foundational sim2real work leveraging Isaac Lab!
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗adept-dexterity.github.io with Dex team @NVIDIAAI
Jayjun Lee retweeted
Dexterity should be a prior, not a per-task cost!
ADEPT learns reach-grasp-reorient-transport once in sim, then post-trains specialists that deploy zero-shot on real hands from pixels and touch: 3B steps per new task vs ~9B from scratch.
Great work @jayjunleee & @NVIDIAAI Dex
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗adept-dexterity.github.io with Dex team @NVIDIAAI
Jayjun Lee retweeted
It’s incredible how close we r coming to zero shot transfer on hardware!!
Hoping one day we can zero shot transfer for any task or morphology. This is one step towards that!
Great job guys!
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗adept-dexterity.github.io with Dex team @NVIDIAAI
Jayjun Lee retweeted
Amazing! Sim-to-real will be critical for dexterous hands I am sure of it.
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗adept-dexterity.github.io with Dex team @NVIDIAAI
Jayjun Lee retweeted
Incredible work! DextrahRGB proved visuals-action could work, so this is the obvious progression to tactile and a meaningful one!
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗adept-dexterity.github.io with Dex team @NVIDIAAI
Jayjun Lee retweeted
Always exciting to see sim2real RL working 🔥
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗adept-dexterity.github.io with Dex team @NVIDIAAI
Jayjun Lee retweeted
Simulation will enable super-human dexterity.
Our robots learn 23/29-DoF hand/arm policies from scratch, entirely in simulation, and transfer 0-shot to real.
Pre-training on a reorientation task accelerates dexterity learning on other tasks.
This is just the beginning, so many exciting techniques and results to share soon.
Great work by @jayjunleee and Dex team at NVIDIA
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗adept-dexterity.github.io with Dex team @NVIDIAAI
Jayjun Lee retweeted
Awesome new work from the dexterity team!
1. It's not always trivial to collect data that matches your morphology.
2. It's even harder to collect recovery data.
Object reposing is a great way for learning a generative prior --> fine-tune to handle new and unseen tasks.
What if robot hands could acquire dexterity once, then reuse it to learn downstream tasks much faster?
Introducing ADEPT: a pre-training & post-training paradigm using RL entirely in sim, then zero-shot deploying visuo-tactile policies in the real world.
🔗adept-dexterity.github.io with Dex team @NVIDIAAI
Jayjun Lee retweeted
ADEPT: Accelerating Dexterity via Pre-Training.
What if dexterous robots could learn foundational manipulation skills once, and then reuse them across many different tasks?
ADEPT pretrains a dexterous RL policy on a generic object reposing task, learning useful motor primitives that can later be reused as a prior for downstream tasks.
The goal: make learning complex dexterous behaviors more efficient and transferable.
The resulting policies solve complex manipulation tasks in just 5–10s with a multi-fingered hand, 2–14× faster than parallel jaw grippers, without external fixtures, pose trackers, or scripted task decomposition.
We also find that touch can signifcantly improve the real-world success rates for sim2real.
This work is done during my internship at NVIDIA and this project wouldn't have been possible without my collaborators at the Dex Team and UMich: Jessica Yin, Asif Rana, @nmblauch, Sam Mady, @mostlymohak, @NimaFazeli7, @robot_trainer, Karl Van Wyk, @ankurhandos.
Paper: arxiv.org/abs/2608.19182
Webpage: adept-dexterity.github.io
@NVIDIARobotics @UMRobotics
