Intern @ Amazon FAR | CS PhD student @UofIllinois | Previously @Tsinghua_IIIS and XJTU | robotics & ai

San Francisco, CA
Joined April 2020
Thrilled to announce that HERO has been accepted to @corl_conf 2026🤠! Congrats to every contributor! I will be attending CoRL in person, and see you guys at Texas and let's talk about humanoids!
Want to ask your humanoid to get novel objects you want from a novel table? We introduce HERO, first achieving open-vocabulary visual loco-manipulation using human language queries! Now we can ask the humanoid to see and grasp the target object (eg, a carrot instead of a tissue box) with whole-body coordination. How do we do it? Key designs and findings in this work: We propose a residual-aware end-effector tracking policy that tracks the target accurately in a closed loop. We find that the robot's forward kinematics are quite inaccurate, and thus we propose residual neural forward models that correct the end-effector FK and base leg odometry. We design a modular system powered by visual foundation models that achieves generalizable grasping capabilities with 83% success rate on novel daily objects and daily scenes. Check out our new project: Learning Humanoid End-Effector Control for Open-Vocabulary Visual Loco-Manipulation Project page: hero-humanoid.github.io/ Paper: arxiv.org/pdf/2602.16705
3
4
32
3,592
RPL code is released at github.com/amazon-far/RPL, including experts training, multi-depth distillation and sim2sim. If you have an Xbox controller lying around, it’s a lot of fun playing it in Mujoco. Its perceptive features were adopted by some cocurrent work like LadderMan (ladderman-robot.github.io/) and PHP (php-parkour.github.io/) with promising results. RPL was also accepted to CoRL2026. See you in Austin.
Robust humanoid perceptive locomotion is still underexplored. Especially when different cameras see different terrains, paths get narrow, and payloads disturb balance... Introduce RPL, tackling this with one unified policy: • Challenging terrains (slopes, stairs and stepping stones); • Multiple directions; • Payloads; Trained in sim. Validated long-horizon in the real world. Watch the robot walk it all🦿 Details below👇
3
27
2
165
7,433
Runpei Dong retweeted
Your navigation model already found the goal. It just doesn't know when to stop. Introducing StopWatch, a navigation agent that grounds the goal explicitly before every action. It sets a new state of the art on six object-goal navigation benchmarks. Watch the flower vase episode below. Both agents reach the goal. StopWatch stops there. LightNav-0 turns away and keeps going. Code, models and data are open source 👉 stopwatch-nav.github.io/ Work done at Amazon FAR w/ @yunshengtian @QiuYuhengQiu @pabbeel @peterxichen @GuanyaShi @simonforrest 🧵 [1/n]
2
8
2
24
1,455
Runpei Dong retweeted
Recursive self improvement
92
159
55
2,042
119,418
Runpei Dong retweeted
Can we make dexterous robots compliant without sacrificing precise control? Excited to introduce FOMO-Dex: Force-Modulated Dexterity! 🖐️ FOMO-Dex brings force-aware compliance to dexterous teleoperation, enabling robots to pick up thin cards and chips 🃏, cut paper 🔪, turn nuts 🔩, and use screwdrivers 🪛. These force-aware demonstrations also train better autonomous policies! 🎥 Videos & paper: yingyuan0414.github.io/fomo-… 🧵👇 (1/8)
11
41
4
166
13,516
Runpei Dong retweeted
My new video on ai and design is now live on @Microsoft’s official website! I believe designing intelligence means shaping humanity. In the video, I talk about: - Building @copilot into a thinking partner - Designing beyond interfaces: models, memory, evaluation... And a question I keep coming back to: as ai reshapes our world, how can we live fully, creatively, and meaningfully?
55
53
35
248
491,665
Runpei Dong retweeted
Perceptive Humanoid Parkour (PHP) is now OPEN-SOURCE! 🔥 Try training your own parkour robot! 🤖 🔗 Project: php-parkour.github.io/ 💻 Code: github.com/amazon-far/php_pa… 🎮 Try it in your browser: php-parkour.github.io/demo.h…
Can humanoids perform agile, autonomous, long-horizon parkour—based on what they see in the world? We present 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗛𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗣𝗮𝗿𝗸𝗼𝘂𝗿 (𝗣𝗛𝗣): a framework that chains dynamic human skills using onboard depth perception for long-horizon traversal. 1/6
4
51
5
407
21,474
Runpei Dong retweeted
🤖 For parallel-jaw grippers, high-level plans can already be executed with mature primitives such as grasping. What about dexterous hands? Introducing GOTT (General Object Trajectory Tracking): object trajectories to dexterous execution with a reusable cross-embodiment contact primitive. The same primitive supports diverse real-world tasks including sweeping, pouring, whiteboard erasing, and hammering. With one checkpoint, GOTT reaches 90-96% grasp-and-lift success across multiple objects on three real-world arm-hand platforms. Check videos 👇
11
35
7
140
14,257
Runpei Dong retweeted
From 2D tracks to 3D tracks... what's next for video motion estimation? 🎬 Check out MoSE3, our new framework for dense, world-space SE(3) motion estimation from monocular video! Beyond where points move, MoSE3 predicts how they move: full 6DoF transformations (rotation + translation) plus motion grouping, for rigid, articulated, and deformable objects. The result is a structured, actionable motion representation for dynamic scenes. It could be useful for robot learning and manipulation.
Excited to share that MoSE3 was accepted as #NeurIPS2026 Spotlight! MoSE3 estimates an SE(3) transform at every pixel from monocular video, all in a shared world coordinate frame and in a single forward pass. It captures rigid, articulated, and deformable motion. 1/
3
15
1
113
7,059
Practice makes perfect. A truly agentic robot shouldn’t just know what to do. It should also figure out how to get better. What if a robot could watch a demonstration, figure out what it needs to practice, and learn how to get better through practice? Introducing RPG: Guided Self-Improvement for Embodied Agents, a real-to-sim-to-real pipeline that turns real-world demonstrations into targeted practice. RPG reconstructs practice tasks from demonstrations, uses demo videos to guide skill improvement, and continually updates a shared skill system from failures. On real robots, RPG achieves 30/30 successes across 3 tasks. In simulation, it reaches 95.0% success across 22 manipulation tasks after 15 rounds of self-improvement. Demo → Reconstruct → Guided Practice → Self-Improve → Go Real
9
53
15
227
166,270
Okay, now mathematical research scales.
We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results. github.com/openai/math
381
Runpei Dong retweeted
Meet Workhorse 🐴: we study how to learn whole-body loco-manipulation skills from human demos without teleop. Here is my favourite video: we tried everything to make it fail🏒. One take, real-time, fully autonomous.
Can a humanoid learn robust whole-body loco-manipulation directly from a human? No robot, no teleop, no retargeting. Meet Workhorse 🐴: our G1 topples and climbs a suitcase, catches a thrown box, and sorts boxes with hands and a kick. Fully autonomous. More videos at hybridrobotics.github.io/wor…
6
25
10
173
16,688
Runpei Dong retweeted
A robot learning a new task from 25 human video demonstrations, with its model weights frozen: that’s one result from Cornell’s Unified Motion-Action (UMA) model. Adaptation updates only a small set of task tokens that represent what to do. UMA connects human videos to robot control through 3D object motion: how points on objects move over time. It learns motion from videos without robot action labels, and learns the connection between motion and actions from robot data. Pretraining combines human videos, real robot demonstrations, and simulation. In video-only adaptation, UMA’s success rate was approximately 25 percentage points higher than UVA’s on each task: utensil insertion, sweeping, and folding jeans. Each task had 20 evaluation trials. The human demonstrations came from the evaluation scene and viewpoint. The practical takeaway is that a new task can be taught through human demonstrations without collecting robot action labels for that task. Prior robot training still supplies the action knowledge; deploying on a new robot embodiment still requires compatible robot data. How far does video-only adaptation transfer when the demonstrations come from a different scene or viewpoint? That’s the next test I’d want to see.
1
7
81
4,771
Runpei Dong retweeted
Dear algorithm, I'm moving to Europe. Find me a full-time job in XR / Robotics. Shipped my AR game Sardo, featured by Apple. Product-minded; quick learner; independent. Unity, ECS, ROS 2, IK, Pathfinding, Behavior Trees, NVIDIA Issac, OpenUSD. Co-authored at ACM IUI 2026.
17
21
264
9,863
Runpei Dong retweeted
ℑ 𝔴𝔦𝔩𝔩 𝔰𝔲𝔯𝔳𝔦𝔳𝔢
363
13,680
365
125,917
4,485,528
Runpei Dong retweeted
🔥HERO live demo and code is available now! Play it at at hero-humanoid.github.io/#int… 🔥Check out the code at github.com/RunpeiDong/HERO
4
17
10,748
🔥HERO live demo and code is available now! Play it at at hero-humanoid.github.io/#int… 🔥Check out the code at github.com/RunpeiDong/HERO
4
17
10,748
Runpei Dong retweeted
Wtf is happening to my response. You guys spend too much time on twitter.
2
3
50
4,440
Runpei Dong retweeted
Can a humanoid be your badminton buddy, learning from just 30 min of human motion data? Humanoid Badminton 🏸, accepted at #CoRL2026, learns diverse racket skills with a single policy, enabling natural whole-body motion and real-world multi-skill rallies with forehand, backhand, and highly dynamic jump returns. 🌐 Project: sunlight02.github.io/humanoi…
1
13
3
88
6,021
Can't agree more. Every paper that can bring new thoughts and results is a contribution to the whole area, no matter it is significant, subtle, or "useless" (negative results). As more humanity intelligetnce is involved, technique evolves. "Chaos is ladder" is somehow correct.
NeurIPS decisions brought joy to some and disappointment to others. My first NeurIPS paper was in 2020, but given how much AI and robotics have changed, it feels like a lifetime ago. I’ve shared a few thoughts with my students on the role of publications and so-called “A+B” research, and I’d like to share them more broadly here.
1
4
1,030
Runpei Dong retweeted
We trained a robot to play football. How? Self-play for 140 years in a virtual World Cup. Meet the #Messinator:
216
409
258
2,029
1,774,911