Once it worked well enough, I used it to collect 99 demonstrations of picking up a cube and placing it inside a box. The “dataset” contains ~28 minutes of robot motion. Collecting them took around an hour once you include resetting the cube and failed attempts. Teleoperating also got noticeably easier as I continued. During the first quarter of one session, 73.5% of the control ticks were clean (the controller didn’t freeze the arm) and the median IK error was 93mm. By the last quarter, those numbers were 99.3% and 1.3mm. I was getting much better at knowing where its limits were and when to use the clutch. I had some teammates who’d never done teleoperation before try it and they even managed to contribute to the robot data. I later used it to collect another 122 robot demonstrations, bringing the total to 221 unique demonstrations. By then teleoperating felt much more natural, and I was getting surprisingly good at performing tasks with the robot. Remember though, this still isn’t the original goal of having people perform work naturally by wearing the gloves. I’m still directly controlling a robot here, with a Coil sitting beside it and a calibration specific to that setup. But it proved that the mapping worked well enough on physical hardware to collect useful robot data. I’ve since trained a policy on some of the data and got the robot to perform the task on its own, but that part deserves another post.

sharing a post soon on this teleoperation stack