I’m rethinking what we should optimize for in robot generalization. The central question is: what data and pretraining would make a model fast at learning almost any new task? The tasks it already performs, and the demonstrations it was trained on, tell only part of the story. Consider learning tennis. Coding knowledge is not directly needed to hit a ball. Yet humans bring a broadly developed efficient learning system that supports acquiring unfamiliar skills compared to monkeys. My intuition is that diverse experience improves learning efficiency beyond the specific skills it teaches. That makes labels like “spatial intelligence” or “cat-level intelligence” less central to me than understanding how to build a more capable learner. It also changes how I think about System 1/System 2. A sufficiently general world model might eventually handle manipulation efficiently on its own. A world model as System 1, paired with an LLM agent as System 2, could still offer practical advantages in latency, compute, and deployment. But today’s binding constraint in robotics is generalization. Efficiency becomes much more valuable once the system can reliably adapt. My current bet is that multimodal foundation models offer broader pretrained knowledge and visual understanding than the video backbones we currently use for WAMs. I suspect a key advantage lies upstream: their training pipelines can draw on a much larger pool of high-quality visual data and egocentric manipulation data, while highly diverse pretraining—whether in language or vision—equips them to learn brand-new tasks efficiently. Robotics agents offer a way to turn that foundation into learning through interaction, feedback, and experimentation. That is why prioritizing Agent for Robotics now could be a good idea. I expect rapid progress over the next 6–12 months. The question is: How broad pretraining can make robots substantially faster learners? Apprecite the insights shared by @RunhanH!

That’s exactly what we do in VTAM and DexTacWAM at @berkeley_ai: without any tactile pretraining, we transfer world-modeling capabilities learned from vision to the tactile domain—supporting not only grippers, but also distributed tactile sensing on @SharpaRobotics dexterous hands. Take a look! VTAM: plan-lab.github.io/projects/… DexTacWAM: dextacwam.github.io/