Scaling Real-World Experience for Physical AI
Joined May 2026
- Tweets88
- Following11
- Followers185
- Likes22
Pinned Tweet
We have 2M hours of egocentric human experience recorded. While it's the largest library of datasets, it is also the least informative number we publish.
Two recordings of equal duration can differ by an order of magnitude in what a model can learn from them. One holds repetitive tabletop pick-and-place under fixed conditions. The other holds bimanual manipulation across changing states, with precise geometry, varied contacts, coordinated fingers and tool use. Equal duration, nowhere near equal learning value.
Effective experience ≈ hours × information per hour.
At ten thousand hours that difference is noise. At ten million it is the difference between a corpus and an archive.
glad they left the stuck pucks in. how does it figure out one's misplaced in the first place?
How many pucks could a puck bot pack if a puck bot could pack pucks? 🏒
12 hockey pucks is a tight fit in this container – watch the model fix the arrangement when pucks get stuck or misplaced.
GEN-1.5 doesn't just puck around. Read more about it in our blog post and sign up for the waitlist in the comments below.
Say hello to our new logo. The US office celebrated Friday with fruit cake and boba. Sample size: one cake. We're calling it statistically significant.
Four levels of language annotation on one clip:
Environment: Cafe
Task: Prepare a latte with latte art
Subtask: Pick up the coffee sifter
Instruction: Right hand picks up the coffee sifter
Plus 3D hand pose (<4 mm), body joints (<2 cm) and hand contact heatmaps.
we'd put the data pipeline right up there with the gpus as a bottleneck. feed it clean egocentric data and you need a lot less compute to get to a policy you can actually use
Our own models now label 90% of our egocentric data. At the start of 2025 they labelled none of it, and people did every hour by hand.
Our quality score was 8.5 back then, 8.6 in late 2025 when we were at 40% automatic, and it's 8.9 now.
MaxEgo does the hands and body labels and MaxVLM does video, and both are trained on our own data. People still check what they produce, and 98% of what we ship also goes through a four stage automated QA system.
For 2026 Q4 we're aiming for 99% automatic at 9.2, and we haven't measured that one yet.
Physical AI learns from people.
So did our logo.
Meet the new Maxinsights mark: { ]
It's open on purpose, a space for what matters. So we handed our team a marker and let them fill it in.
The story behind the mark comes later this week.
What goes in yours?
Four orders of magnitude of human egocentric video, and the only pseudo-actions Dyna-2 got were 3D hand tracks. It kept improving the whole way.
The ablation's the bit we've reread more times than we'd like to admit. Train on actions only and cross-embodiment scaling doesn't show up the same way. Throw in video prediction plus co-training on more human video and it shows up.
Next order of magnitude, how good do the hand tracks actually need to be? maxinsights.ai/blog/Beyond_1…
Maxinsights retweeted
"Make them do dexterous things" is nice.
The test I'd use: what does the hand do in the half second after a grasp slips. That recovery is most of what a human hand is doing all day.
Richer data. Stronger supervision. More reliable robot actions.
This video showcases our precise full-body pose tracking—including individual hand joints—alongside tracking of objects being manipulated and heatmaps of contact regions.
Together, these complementary data modalities provide richer supervision for training humanoid robot models, helping them learn more effective action plans and execute tasks with higher success rates.
Full releaseon on Monday.
This is the input. One head-mounted camera, one person, one ordinary task.
Hand joints, the object, and where the hand is making contact, all from that single view.
Our CTO shows you what comes out of it on Monday.
Maxinsights is heading to #IROS2026!
Meet our team at Booth 630 to talk real-world data for robotics and explore ways to collaborate.
Real-world data. Real conversations. See you there!
Every element of the physical state left unrecorded is one the model has to infer from pixels.
Sometimes that is fine. Sometimes it is the difference between learning contact physics and memorising appearance. Either way the decision is made at capture, by whoever writes the protocol, and no downstream process reverses it.
🧵
Single-view head-mounted collection is enough for a large share of tasks.
A lot of it is everyday manipulation, like the outdoor food work here, captured single-view. Running the full rig everywhere would be slower without making most of that footage better, so we reserve it for the contact-critical subset. Matching capture to the task, not maximising sensors, is part of what makes 450K hours a month possible.
The overlays are the output the rig choice produces. Fine assembly in an electronics lab, box folding, gloved hands at a sink. Each one is a case where a single head-mounted view loses the contact, and nothing downstream reports it missing.
This is the depth axis: not more kitchens, but more of each interaction preserved inside the kitchens already recorded.
A pipeline is a series of gates, and yield multiplies.
Four gates at 95% give 81% end to end. Push every gate to 99% and yield reaches 96%. At ten-million-hour volumes those fifteen points separate a corpus that gets delivered from one that gets reprocessed.
🧵
Underneath it, the ingestion layer: 84 batches, 799,946 files, 12.84 TB across 100 nodes on a 100 Gbps cluster.
This is the least interesting thing to publish and the part that decides how many recorded hours become training hours.
End-to-end yield is the one metric of the three that is well defined today.
Of the hours recorded, how many survived to training. Not per stage, end to end. It is answerable now, which is what separates it from marginal information gain and state coverage, and it is the number worth asking any supplier for.
