Necro

This live demo of four Microducks with Jensen Huang went completely viral, and we're ecstatic about it. But there's also a lot of confusion, so I want to clarify what Microduck can and cannot do, and what it will realistically be able to do in the future. I count 60M+ views on this demo alone, based on the few big posts I see on social media, and everything has been buzzing. Even my own simple emotional design post on a newly created Instagram account got 1M+ views. We are selling Microducks at an incredible rate, even faster than during its historic launch. I've been trying to answer all the questions I see on our socials: people are asking if X or Y is possible on the robots, and this post aims to answer most of them. First of all, the demo you've seen was not fully autonomous. We had one engineer piloting each Microduck. The motions on the robots were trained with reinforcement learning, using a sim-to-real pipeline: walking, standing up after a fall, sitting, doing a somersault, roller skating. All of those are policies trained this way. What the pilots did during the live event was send commands: directions (move forward, rotate), head orientation (look here), quacks, and special actions (do the somersault, sit down). If you buy a Microduck, will you get an autonomous companion right out of the box? The current answer is no. But there is no impossible technical barrier to get there. I'm not sure how much time we'll have to work on this before shipping, but since everything is open source, I fully expect the community to build it, and to build it fast. Now let's understand what's already done, what will be easy to add, and what's an open problem. What you'll be able to do out of the box, and you can already do it in simulation right now, is train new policies with reinforcement learning. You could teach your duck to walk differently from the default gait we provide, to do a weird dance, or to walk on stilts. The community has been crazy creative with this, even though they only have the simulator. Note that you don't need reinforcement learning when the movements are simple. For example, the early emotional design I did is mostly coded movements plus pre-generated sounds. A simple mental model: use reinforcement learning when balance is part of the movement. If you try to code a cinematic walk by naively putting one foot in front of the other, your robot will realistically never walk, because you need to account for the dynamics of the movement, the inaccuracies of the sensors, the friction with the ground, and potential perturbations from the outside, like being pushed. So training new policies is definitely doable, but you can't train a policy to get a genuine long-term companion in your home (well, technically you could, but that's a whole other range of policy: a research-grade problem, and the resulting neural network would probably be huge and never able to run on such a small compute). Instead, you need a few more building blocks: voice recognition, facial detection/recognition, localization, some simple navigation, and a behavior layer, which is less technical and harder to describe. As a rule of thumb, a prototype of any of these is within reach of anyone if they have a modern coding agent and if the computation doesn't run on the robot (meaning the robot can reach the internet or an external computer and use external APIs). A rough prototype for each of the above can be a chill weekend project. As always, going from the prototype to a high-quality implementation is another story, but still doable for most if you put in the time and test a lot. However, having all of this run locally, on the onboard chip, I would call a very hard task. You'd need some very clever engineering to get a good enough version of everything working together. But I can't say it's impossible, because I've seen some crazy efficient implementations of all of the above. And here we don't need a super high success rate: it's okay if the duck sometimes does something a bit random. So I wouldn't be surprised if folks managed a fully local, cool, simple version of this autonomous companion. For example, you don't need Whisper-grade open-vocabulary transcription: being able to recognize a few keywords would already be great for interaction. One of the fastest ways to turn this robot into an autonomous companion would be to stream Microduck's sensors (the camera, the depth sensor, the IMU and the joint positions) to a computer, and use a software stack like the one we use for the conversation app on Reachy Mini: a local VAD, then a cloud speech-to-text model, then a call to a frontier LLM that decides on actions. Actions could be quacking, looking at things/faces/a direction, walking to a given position, sitting down, playing an emotion, kicking or doing a somersault. A SLAM algorithm would run in parallel so the robot can localize itself in your home, and you would prompt the LLM for a lifelike behavior: moving to random places, sometimes following people, and having emotional responses that fit what you're saying and the duck's environment. Of course someone is going to build a Tamagotchi-like layer on top of this, and it will be insane. How much demand is there for a software suite that would do all of this? I for one think it would be super cool. A word about the humans. We had five engineers backstage, and they are pretty much the core team behind Microduck. Their impact per head is absolutely off the charts. Antoine Pirrone piloted Graphite, Augustin Crampette Lavender (the rebel one), Matthieu Lapeyre Sky, Coralie Deplane Cream, and Pierre Rouanet was stressing over the Wi-Fi. Give them a follow! (Antoine @antoinepirrone, Augustin on LinkedIn, Matthieu @matth_lapeyre, Coralie @Drenaya_, Pierre on LinkedIn). I get visibility because I communicate a lot, but I mainly work on Reachy Mini (you might want to check it out: it's a very cool robot, and it ships much faster than Microduck :) ). My contribution to Microduck has been minimal: I'm mostly a first power user. My main added value at GTC was having hands big enough to carry four Microducks below the stage. Where I take pride is that Antoine and Coralie are former students and former interns of mine. I'm therefore entitled to 3% credit on anything (good) they do, for life. If they do something bad, however, it was clearly the fault of their parents. About the super smart engineering that can be done on board: Antoine just released two very cool blog posts (links below). The first is on odometry, the robot estimating its own movement from its own internal sensors. The second is about SLAM, which allows long-term localization: it adds the depth sensor to the odometry to get a stable position over time. Think of it as recognizing known landmarks in your home to know where the robot is. The crazy part is that all of this runs on the robot. So this gives me hope that we'll be able to do a lot of cool stuff on board.