So I found this paper from 2005 that might help solve one of the biggest problems in computer-use agents: Accurate Pointing! The paper is called The Bubble Cursor and the idea is really simple, but kind of genius. Instead of treating the cursor as one tiny pixel, dynamically expand its activation area so it locks onto the nearest valid target. Originally, this made pointing faster for humans. But for AI agents, it solves something bigger. When I say “click Add” or “change this,” the hard part is not reasoning. Its the unnecessary loop of sending your whole screenshot to the LLM and asking it the coordinates for where to click. So I’m combining Bubble Cursor with: - accessibility trees - browser DOM coordinates - on-device OCR - local UI object detection - CUA for actions The goal is simple: - LLM understands intent. - Local OCR and YOLO understands the screen. - Bubble Cursor resolves the target. - Pointer clicks Less screenshot guessing. Less latency. More privacy. I can be super vague about it like "can you click on the big button on the bottom left" and it just knows and I didnt even have to send a screenshot! I want to make the FASTEST and the most Accurate computer use agent which can keep up with the users intent. Shipping out a connector for @OpenAI and @claudeai so that even they can point and click quickly and take actions faster. The paper is by @ToviGrossman and Ravin Balakrishnan.