CS PhD @uchicago | Prev. @waseda_univ

Joined October 2022
Force sensing for low-cost robot arms — without adding force sensors. 🚀 Excited to share FACTR 2! 🚀 FACTR 2 enables external torque sensing on low cost arms and uses it to improve policy learning. w/ @JasonJZLiu  @_tonytao_ 🧵(1/6)
8
23
8
150
40,432
Introducing PACT: we co-designed a 17-servo anthropomorphic hand and its retargeter together, so teleop demos for imitation learning track the human hand much more closely. 3.9 mm keypoint RMSE on human grasps (vs. 12.8 mm for LEAP Hand). 📄 arxiv.org/abs/2610.18443
1
3
3
18
3,092
Coding agents are emerging as general robot policies, but they are still inefficient (e.g., requiring long real-world exploration), especially in mobile manipulation. Humans become efficient by watching others. Can coding agents learn from passive human videos, just like we do? Introducing MoCaP: Mobile Code-as-Policies for Zero-Shot Mobile Manipulation from Passive Human Videos ↓🧵
8
18
4
105
9,778
Steven Oh retweeted
Can we make dexterous robots compliant without sacrificing precise control? Excited to introduce FOMO-Dex: Force-Modulated Dexterity! 🖐️ FOMO-Dex brings force-aware compliance to dexterous teleoperation, enabling robots to pick up thin cards and chips 🃏, cut paper 🔪, turn nuts 🔩, and use screwdrivers 🪛. These force-aware demonstrations also train better autonomous policies! 🎥 Videos & paper: yingyuan0414.github.io/fomo-… 🧵👇 (1/8)
11
40
4
160
12,665
Steven Oh retweeted
🤖 For parallel-jaw grippers, high-level plans can already be executed with mature primitives such as grasping. What about dexterous hands? Introducing GOTT (General Object Trajectory Tracking): object trajectories to dexterous execution with a reusable cross-embodiment contact primitive. The same primitive supports diverse real-world tasks including sweeping, pouring, whiteboard erasing, and hammering. With one checkpoint, GOTT reaches 90-96% grasp-and-lift success across multiple objects on three real-world arm-hand platforms. Check videos 👇
10
34
7
137
14,068
Steven Oh retweeted
We’ve all seen the cool “Astra for robotics” demos. But behind the scenes? Slow rollouts, hand-engineered APIs, and/or simulated behaviors that struggle on real robots. How do we make coding agent actually work in the physical world? 🦾 Introducing SimEX: Simulation-Integrated Robotics AutoResearch 🧵 arxiv.org/abs/2609.38982
11
60
8
293
31,730
Can robots remember what contact felt like, and reuse it in new situations? 🧠 ✋ In our new RA-L paper, we present TaMeSo-bot: a robotic system with a soft wrist that retrieves similar past tactile experiences to guide robust object insertion. ↓🧵
3
14
1
81
13,972
How can we leverage pre-trained VLMs for long-horizon, bimanual tasks like cooking? 👨‍🍳🤖 Our #IROS2026 paper, ViLaIn-TAMP, combines VLM reasoning with structured task & motion planning. With all the Astra hype, this feels even more relevant, especially since we did it w/ 4o!
4
19
1
90
5,068
How can robots learn contact-rich tasks, like insertion, with very little data? Introducing PHASE, a retrieval-based imitation learning framework that efficiently learns insertion skills by reusing prior experience. We’ll be presenting PHASE at #IROS2026 in Pittsburgh! 🇺🇸
2
16
1
57
4,883
Steven Oh retweeted
We trained a robot to play football. How? Self-play for 140 years in a virtual World Cup. Meet the #Messinator:
217
410
259
2,033
1,771,938
Accepted to #CoRL2026!🥳 Special shoutout to all my co-authors🐐 who worked hard on this one, see you in Austin!
Can pretrained VLMs become capable VLAs without losing their semantic capabilities?🤖 Excited to introduce CLAP👏 (Causal Language-Action Prediction), a lightweight VLM-to-VLA fine-tuning recipe that requires no action expert, no vocabulary modification, and no architectural change! 🧵👇🏻
4
13
53
3,765
Steven Oh retweeted
We’ve released code for NEXT, our neural external torque estimation method from FACTR 2! NEXT enables commodity robot arms to estimate external joint torques without dedicated force sensors. We hope this release will be useful to the robotics community. Code: github.com/philiphan0109/fac… Website: jasonjzliu.com/factr2
Force sensing for low-cost robot arms — without adding force sensors. 🚀 Excited to share FACTR 2! 🚀 FACTR 2 enables external torque sensing on low cost arms and uses it to improve policy learning. w/ @JasonJZLiu  @_tonytao_ 🧵(1/6)
13
98
10,092
Steven Oh retweeted
NEXTを使った各関節の外力推定を観測情報としてACTにそのまま突っ込んで重さの違う物体を分類する。つまりモーターの外力を特徴として動作を生成している。関節に負荷かかりやすいように姿勢を工夫してます。 #lerobot #omx #robotis #bilateral
1
2
20
1,273
VLAモデルを用いたロボット動作デモの映像を公開しました。 同一構造のモデルを各タスク向けに学習することで、食器のような剛体から布のような柔軟物まで幅広く扱えるようになります。 各タスク向けの学習結果を適宜切り替えることで、動画のような複数タスクの連続実行も可能になります!
1
39
3
119
7,865
Steven Oh retweeted
重力保証+フォースフィードバック フォロワー側を手で押さえるとリーダー側が動かなくなる。
4
7
71
5,564
Steven Oh retweeted
[1/N] Large flow policies (e.g., VLAs) have become extremely popular, unlocking scalability. However, they come at a cost: the robot cannot react quickly. π𝐑² enables them to achieve reactive, real-time control.
10
38
3
139
26,311
Steven Oh retweeted
First demo of reproducing force feedback from the FACTR 2 paper It relies on torque and current sensing which the feetechs don’t handle as well due to large gear ratios so it’s not perfect. Not sure if limits are in hw or we can do better with core changes & more data
Impedance control makes robots safer to interact with (sorry for camera I only have 2 hands) Also note leader hand floating in free space, not possible without gravity comp
5
11
3
153
19,235
Can pretrained VLMs become capable VLAs without losing their semantic capabilities?🤖 Excited to introduce CLAP👏 (Causal Language-Action Prediction), a lightweight VLM-to-VLA fine-tuning recipe that requires no action expert, no vocabulary modification, and no architectural change! 🧵👇🏻
7
25
7
165
29,705
Steven Oh retweeted
Thrilled to share CLAP from my internship at @omron_sinicx ! We turn a pretrained VLM into a VLA with zero architectural changes: describe actions in natural language, prepend them to action tokens. 90.8% on LIBERO with a 2B model, <6 hrs of post-training. omron-sinicx.github.io/clap/
4
10
2
58
27,218
How can we scale perception-based humanoid learning without collecting massive humanoid teleoperation data? 🚀 Excited to finally share VLK! What excites me most about VLK is that it reframes data collection as a data generation problem. Instead of relying on expensive humanoid teleoperation, we automatically generate synchronized vision, language, and whole-body kinematics from reconstructed real-world scenes. Making this vision a reality required bridging three fundamental challenges: 👀 Perception: Bridging the RGB sim → real gap through visual domain randomization and motion blur mitigation during both training and deployment. 🤖 Embodiment: Bridging the kinematics → dynamics gap with real-time VLA deployment, test-time RTC, and SceneBot, enabling seamless deployment on a real humanoid. 🌍 Environment: Bridging the real-world → synthetic gap to enable scalable Vision-Language-Kinematics data generation through scene reconstruction and interaction synthesis. It has been an amazing journey working with such an incredible team. For a complete walkthrough of the project, check out @jiaman01's thread below 👇 🌐 Project: vision-language-kinematics.g… 📄 Paper: arxiv.org/abs/2606.30645 🎦 Video: youtu.be/ZB6k_iMJP7M Huge thanks to my amazing collaborators @jiaman01 @eric_srchen @TakaraTruong @ Pei Xu, and to our advisors @pabbeel @rocky_duan @KoushilSreenath @akanazawa @carlo_sferrazza @GuanyaShi @ckarenliu.
🤖 How can we scale up humanoid robot learning? Introducing 🌟VLK🌟: generating large-scale synthetic data with paired egocentric observations, text, and full-body G1 kinematics for learning humanoid loco-manipulation. No teleoperation needed! Website: vision-language-kinematics.g…
13
43
10
307
879,700