Co-founder & CTO at @OpenGraph_Labs, building multimodal data infrastructure

Seoul
Joined September 2025
Physical AI needs human data, but human data capture is still way too hard. Not because pressing record is hard. Because the moment you add cameras + sensors, everything gets messy: Every device has its own clock. Streams can silently fail. Recording health has to be checked. Start / stop has to line up. Synchronization has to be solved after. SyncField Desktop turns it into one workflow. Auto-discover cameras + sensors. Connect streams with aliases. Drag, arrange, and monitor panels. Record everything in one click. Review synchronized playback. Get frame-aligned data on disk. No handclaps. No LED flashes. No sync scripts. No file wrangling. Just humans doing real tasks, captured cleanly. If you're working on human data for Physical AI, reach out: opengraphlabs.com/#contact
2
3
5
1,485
Jerry Han retweeted
Introducing EgoVerse: an ecosystem for robot learning from egocentric human data. Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling 1300+ hrs, 240 scenes, 2000+ tasks, and growing Dataset design, findings, and ecosystem 🧵
38
160
45
906
311,461
Excited to share that @OpenGraph_Labs has been accepted into @NVIDIA’s Inception Program 🚀 Our mission is to build reliable infrastructure for multimodal data capture, powering the next generation of robotics & world models 🌎
3
23
2,437
World models can predict the next frame. They can't predict the next touch. That's the gap visuo-tactile world models will close. Is the robot gripping hard enough? Is the surface rigid or soft? When exactly does contact begin and end? Vision doesn't know. Tactile does. We built @OpenGraph_Labs to capture what cameras miss. Egocentric RGB × 5-finger multi-taxel tactile gloves. Frame-synced. Calibrated. In-the-wild. No lab setups. No scripted pick-and-place. Just humans doing real tasks in real stores. Watch the exact moment contact happens. The pressure map lights up in sync. Every touch. Every frame. 👇
4
14
1
116
13,504
Owning your data infra is the new moat in robotics.
Data can’t just be outsourced🤯 To iterate fast, robotics teams must own their data infrastructure Introducing SyncField: turnkey data infrastructure for in-the-wild data collection (Best for UMI-style & Embodied human) #Robotics #UMI #DataCollection
4
978
Are you sure your training data is actually synced? Egocentric camera sees a hand grasping an orange, but the wrist cam shows nothing and tactile reads zero contact. Your policy is learning from broken data and doesn't even know it. In Physical AI, multi-modal sync is everything. → Egocentric: 30fps → Wrist: 30fps → Tactile: 100Hz Different devices, different clocks, slightly different rates. The drift starts small. Barely noticeable frame by frame. But over a 4-minute episode, that tiny difference compounds into seconds of misalignment. And you had no way to even check. Until now. We built the Sync Quality Dashboard. One score tells you if your data is clean. Then go deeper. Clock offset, drift rate, jitter, frame drops, per-stream correction. All visible, all measurable. In a 4-min episode, accumulated clock drift reached 7.5 seconds by the end of the recording. After correction: 9.0ms. That's the difference between "roughly aligned" and "actually aligned." Visually confirm vision-to-vision, vision-to-tactile alignment frame by frame. No more "trust me, the data is fine." We don't just collect multi-modal demos. We ship a quality assurance layer so you can verify every episode before it touches your model. All data in @LeRobotHF format. Ready to train. Verified in sync. Stop guessing. Start verifying.
5
11
1
104
10,485
VLMs see everything. Feel nothing. VLMs annotate what looks like contact. Tactile sensors verify what actually is contact. We ran VLM annotation on real manipulation demos. It labeled a grasp as "approach." Skipped release phases entirely. Hallucinated state transitions that never occurred. 6 out of 36 action phases wrong. 17% of your training data, corrupted. Why? Pixels don't know when a fingertip touches a surface. Pixels don't know when grip pressure hits zero. Tactile sensors do. So we built a pipeline that catches every error automatically. Tactile evidence validates every contact transition. Wrong labels corrected, missing phases inserted. Not faster labeling. Truthful labeling. This is one of the core problems we're solving at @OpenGraph_Labs
2
5
43
5,022
Robots can't learn if their eyes and hands are out of sync. A 30fps camera and a 1kHz tactile sensor don't speak the same language. Multiple cameras, multiple sensors, all on different clocks at different rates. Jitter from USB and OS scheduling. Drift that compounds every second of recording. We built a multi-modal sync pipeline that aligns all of it to ±2.5ms. Automatically. Every frame matched. Zero sensor samples lost. No hardware triggers needed. Sensor-agnostic. Hardware-agnostic. Just plug in and record. Physical AI needs real hand-eye coordination. Not approximate, precise. We're building this at OpenGraph. opengraphlabs.com/
5
36
286
21,337
Jerry Han retweeted
🏆 Won 1st Place at the AGI Hackathon at @agihouse_org with @juliakeem @JerryHan_og and @OpenGraph_Labs! We built a "Temporal Action Segmentation Pipeline" for Physical AI. The Problem: Robotics data today = short clips, RGB-only, lab settings. We need long-horizon, multi-modal, in-the-wild data. Our Solution: 🎬 Input: Long manipulation video (5+ mins) 🤖 Gemini VLM → Action & Phase segmentation 🎯 SAM3 → Object tracking with text prompts 🌐 Pi3 → 3D reconstruction & camera poses 📚 Skill clustering → Reusable skill library → Output: Structured robot training data with timestamps, masks & 3D Humans ARE the ultimate robots 🦾 #PhysicalAI #Robotics #Hackathon #Gemini #SegmentAnything Huge thanks to @henry_yu_01 @NomadicML @zoox @DynaRobotics
9
8
2
97
7,811
Now robots can feel the world.
Introducing the Tactile Data Engine - the frontier dataset for robotics. Tactile sensing is the new modality for next-generation robot training and world models, enabling robots to understand physical interaction beyond vision. Let’s get to the next level 🚀
6
1,000
Identifying what truly constitutes high-quality data for robot learning requires iterative experimentation 🛠️ We’re currently experimenting with more than 10 types of smart gloves, and @ManusMeta is one of the highest-fidelity sources of hand data
Physical AI needs data. But here's the problem: Full facilities are expensive and limited. Vision-only systems fail during hand occlusion. And robotics teams can't wait months to access specialized infrastructure. @OpenGraph_Labs built a deployable solution: iPhone for egocentric vision + MANUS data gloves for precise hand tracking. The result: Synchronized multimodal data collection that eliminates infrastructure dependency. This configuration enables robotics teams to capture production-grade manipulation data in any environment, accelerating the development of datasets critical for teaching Physical AI systems dexterous human movement. Full use case: manus-meta.com/use-cases/man… #PhysicalAI #Robotics #DataCollection #EgocentricData #Teleoperation
3
4
16
2,248
We’re now controlling a Unitree robot inside a Gaussian Splat world. In the browser. Workflow: - Generate world via @theworldlabs Marble - Load Gaussian Splat into @threejs - Run MuJoCo physics (decoupled from renderer) Zero setup. Just open a tab and teleoperate a robot. This changes robotics sim. It's so easy. Thanks @drfeifei @mrdoob to build this elegant tool! #robotics #simulation #GaussianSplat #web3D
4
7
34
2,790
I pulled a random street from Google Maps. Turned it into a 3D world with World Labs. Dropped a Unitree G1 into it and hooked it up to our server-side MuJoCo setup. Now the G1 is actually walking around a real Malaysian street in my browser. Three.js is doing the rendering. MuJoCo is running the physics and walking policy on the server. A simple WebSocket keeps everything locked together. Real street → 3D world → server physics + policy → live control in the browser. Kinda feels like cheating. Physical AI is getting wild. Huge shoutout to @theworldlabs. Marble makes this stuff way too fun. #robotics #MuJoCo #WorldLabs #simulation #unitree #sim2real
60
310
31
2,738
167,033
And here’s the original street next to the Marble world it turned into.
1
42
5,827
Cloned my real LeRobot SO-ARM101 setup straight into a MuJoCo scene. same robot, same red/blue/yellow cubes, same place zone. Open the browser, hit the keyboard, and you can move the arm instantly. Real robot → MuJoCo twin → web teleop. Anyone can try it. app.opengraphlabs.com #robotics #MuJoCo #LeRobot #teleop
3
21
161
9,551
+ 266.8K trajectories were generated in the last 12 hours 🔥 all episodes collected from the playground are fully replayable like this
2
2
14
1,506
Making a humanoid stand is harder than it looks. It needs a brain, a policy constantly deciding where every joint should be. And fast reflexes, a PD controller correcting errors in real-time. We run both on the server: - Pre-trained RL policy from Unitree's framework → target positions - PD controller → smooth, stable torques User presses arrow keys in browser → WebSocket → server runs inference → robot walks. Why server-side? The policy is a PyTorch model. Running ML in a browser means conversion headaches. On the server, just load and run. Swap policies by replacing one file. No client redeploy. Same code that would run on real hardware. Browser stays lightweight, send commands, receive poses, render frames. Your browser doesn't need to understand robot control. It sends intent, receives results, draws pixels. Server-authoritative = real robotics code in the loop. #Robotics #mujoco #unitree #reinforcementlearning
3
6
1
77
4,863
MuJoCo on server. Three.js in browser. WebSocket in between. Physics and rendering don't need to live together. Streaming robot state, rendering in Three.js, physics running server-side. Benefits: - Swap physics engines without touching renderer - No browser memory ceiling - Precise MuJoCo physics + beautiful Three.js visuals PoC done. Unitree G1 walking demo synced across both views. Working toward a clean streaming protocol. #robotics #mujoco #threejs
5
22
201
12,567