runtime @runtimesi / physics @penn

San Francisco, CA
Joined February 2023
Chen Peng retweeted
Introducing SimInfinity -- the world's largest open-source robotic simulation benchmark Coming soon: 10K open-source simulations with well-defined robotic tasks inspired by real-world activities across factories, logistics, homes, and beyond.
3
17
1
108
29,590
RSI for robotics
Verbal reinforcement learning is very effective for improving agentic performance on benchmarks, but we wanted to test how this transfers to robotics. We ran GPT 6 Astra, Gemini Argon 4, and Fable 5.1 on the LIBERO benchmark, and then iteratively developed skills which significantly improved performance. Full results linked in the comments
2
3
126
Chen Peng retweeted
Verbal reinforcement learning is very effective for improving agentic performance on benchmarks, but we wanted to test how this transfers to robotics. We ran GPT 6 Astra, Gemini Argon 4, and Fable 5.1 on the LIBERO benchmark, and then iteratively developed skills which significantly improved performance. Full results linked in the comments
2
4
1
12
15,939
Same model. Same task. Different skills. 0/20 → 8/20 successes through skill refinement. The model isn't the whole story. How you use it matters.
Frontier models like GPT-6 Astra and Gemini Argon 4 can control robots, but how much does the harness around them impact performance? We used our Nomad-Harness to test Gemini Argon on LIBERO tasks to test if text based skills could help with harder tasks that require more manipulation. We iteratively refined the skill with a critic agent reviewing failed attempts. Across 20 initial states per task, Argon’s native success rate improved: • Bowl into drawer: 0/20 → 17/20 • Mug into microwave: 0/20 → 8/20 These results show that reusable text instructions can improve robot control, and they also transfer between models, with improvements also shown for GPT-6-Astra and Fable 5.1 with the same skill.
3
101
Chen Peng retweeted
Frontier models like GPT-6 Astra and Gemini Argon 4 can control robots, but how much does the harness around them impact performance? We used our Nomad-Harness to test Gemini Argon on LIBERO tasks to test if text based skills could help with harder tasks that require more manipulation. We iteratively refined the skill with a critic agent reviewing failed attempts. Across 20 initial states per task, Argon’s native success rate improved: • Bowl into drawer: 0/20 → 17/20 • Mug into microwave: 0/20 → 8/20 These results show that reusable text instructions can improve robot control, and they also transfer between models, with improvements also shown for GPT-6-Astra and Fable 5.1 with the same skill.
1
8
2
35
32,049
Chen Peng retweeted
🚀 Runtime (YC F26): The simulation and evaluation platform for physical AI Build realistic environments, stress-test robot policies across thousands of variations, and identify failures before real-world deployment. Building robots? We'd love to chat!
17
8
1
61
90,954
Chen Peng retweeted
🚀 Runtime (YC F26): The simulation and evaluation platform for physical AI Build realistic environments, stress-test robot policies across thousands of variations, and identify failures before real-world deployment. Building robots? We'd love to chat! Reach out to us at runtime.si
18
9
2
69
1,881