3. Take over from anywhere 🌏 In this demo, an operator in Singapore helps a robot in San Francisco pick up and place a battery into a slot. When the robot struggles to grasp the battery, the remote human operator takes over to help complete the task, then hands control back once the step is complete.

2. Guide the robot with language 📝 Give prompts as the robot executes a task to steer its actions. In one cup-stacking episode, the model lacked a good view of the task. The user prompted it to move its idle wrist camera for a better view.

1. Choose a model. Give the robot a task. 🤖 Select from a list of models, then configure reasoning effort and inference speed. Describe the task, then watch the robot reason and act.

Introducing Cortex Harness, a platform for running AI models on real robots with remote human guidance. Prompt a robot to act. Steer it with language. Let a human take over remotely when needed to keep tasks moving. Choose from GPT-6 Astra (Ultrafast), Claude, Gemini, open-weight models, and VLAs. Here’s how it works 🧵

In one microwave run, Astra used its empty gripper to push a protruding microwave tray further inside. In another run, it proposed an extra 1.5 cm push to engage the microwave door latch. Then after closing the microwave door with its arm, it even pulled its arm back to verify that the microwave door stayed shut. 🤯

The VLA model only understands what it can see from its three cameras on the robot's "head", "left wrist", and "right wrist". In the same pipette fitting example, the cameras on the "head" and the "left wrist" did not have a clear view of the pipette holder. Astra cleverly moved its idle right gripper so that it could leverage the "right wrist" camera to get a better view of the task. 🎥

Attaching a pipette to a 5 mm tip requires precise alignment. Simply lowering a pipette onto a tip doesn't guarantee that it will dock. Most of the VLAs in our MolmoAct2 evals made the mistake of assuming that the pipette tip attached and continued the task without correcting the failure. ❌ Meanwhile, Astra noticed when the pipette tip did not attach and tried again until it succeeded. ✅

We tested GPT-6 Astra, Gemini 3.8 Flash, and Fable 5.1 on two real-world robot tasks from our MolmoAct2 real-world evaluations: 1. Fitting a 5 mm pipette tip 2. Loading a microwave Astra scored highest on both. 🥇 Here are some few surprising things that Astra did to come out on top. 🧵

Cortex AI in Times Square. Accelerating physical AI with large-scale, real-world robotics data. Thanks to @brexHQ for making it happen.