Richard Lee retweeted
People long believed in human-like computer use. But software (even terminals) is built for humans, not AI agents. Why not let agents build their own?
Meet ๐๐ด๐ฒ๐ป๐ ๐ฆ๐ฐ, an agent scientist evolving its own interfaces, letting an open 27B model beats GPT-5.5 & Opus 4.8.
With its evolved interfaces, the same model improves ๐ฏ.๐ดร and costs <๐ฑ% as much per task as either frontier model.
Agents today read screenshots and act through click, scroll, type, maybe a shell. So an agent fills a spreadsheet cell by cell when one line of code would do, or prints two thousand lines of a file to find one function.
Every new app or model release means a human reads trajectories, diagnoses failures, and hand-writes new primitives. Skill-distillation methods can reorder what the interface allows, but they canโt extend it. Agents compensate with more test-time compute instead of a better interface.
๐ ๐ผ๐ฑ๐ฒ๐น๐ ๐ถ๐บ๐ฝ๐ฟ๐ผ๐๐ฒ; ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ๐ ๐๐๐ฎ๐ ๐ณ๐ฟ๐ผ๐๐ฒ๐ป.
โWe want AI agents that can discover like we can, not which contain what we have discovered.โ
โ Rich Sutton, The Bitter Lesson
Hand-designed interfaces are what we have discovered. Agents can read their own trajectories, spot whatโs missing, and write and test code. ๐๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ ๐ฑ๐ฒ๐๐ถ๐ด๐ป ๐๐ต๐ผ๐๐น๐ฑ ๐ฏ๐ฒ ๐ฎ๐ป ๐ฎ๐ด๐ฒ๐ป๐ ๐ฐ๐ฎ๐ฝ๐ฎ๐ฏ๐ถ๐น๐ถ๐๐.
Thatโs what Agent S4 does.
Agent S4 makes a ๐ฟ๐ฒ๐ฝ๐ผ๐๐ถ๐๐ผ๐ฟ๐ ๐ผ๐ณ ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ๐ ๐ฎ๐ป๐ฑ ๐๐๐ฎ๐ด๐ฒ ๐ธ๐ป๐ผ๐๐น๐ฒ๐ฑ๐ด๐ฒ the unit of evolution. A task agent attempts synthetic tasks with no success labels or verifiers. A meta-agent studies the trajectories, forms hypotheses about which actions, observations, or knowledge would help, and builds descendants through mutation or crossover. Validation tasks the meta-agent never sees decide which repositories survive.
The model and harness stay fixed, and the meta-agent ๐ป๐ฒ๐๐ฒ๐ฟ ๐๐ฒ๐ฒ๐ ๐ฎ ๐ฏ๐ฒ๐ป๐ฐ๐ต๐บ๐ฎ๐ฟ๐ธ ๐๐ฎ๐๐ธ. ๐ข๐ป๐น๐ ๐๐ต๐ฒ ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ ๐ฒ๐๐ผ๐น๐๐ฒ๐.
With the evolved repository, Qwen3.8-27B goes from ๐ฑ.๐ฒ% to ๐ฎ๐ญ.๐ฏ% on OSWorld V2, ๐ฏ.๐ดร the same model with fixed interfaces and a new state of the art for open-source computer-use agents.
It beats GPT-5.5 (13.0%, $25.50/task) and edges past Opus 4.8 (20.6%, $72.40/task) at $๐ญ.๐ญ๐ฌ per task.
On ScienceBoard, the same model goes from 41.4% to 58.6%.
The analysis is the more interesting part. Evolution mostly deepens interfaces the agent already has: 4 new interfaces and 127 extensions. Give the meta-agent a scratch VM to test its code, and it builds 12 new interfaces and 253 extensions, reaching music scores, audio projects, and 3D scenes. The best repository came from crossing two earlier branches.
In one case study, the seed agent hand-builds a browser client mid-task and debugs through failure after failure; the evolved agent calls page_text and reaches the same message at tool call ๐ญ๐ญ instead of ๐ฏ๐ณ.
Human-like computer use was a starting point, not the destination. The interface an agent acts through is a second axis of improvement, beyond better models and harnesses, and agents can now design it themselves. ๐๐๐ผ๐น๐๐ถ๐ป๐ด ๐ฎ๐ด๐ฒ๐ป๐ ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ๐ ๐ถ๐ ๐ฎ ๐ป๐ฒ๐ ๐ฑ๐ถ๐บ๐ฒ๐ป๐๐ถ๐ผ๐ป ๐ผ๐ณ ๐ฎ๐ด๐ฒ๐ป๐ ๐๐ฒ๐น๐ณ-๐ถ๐บ๐ฝ๐ฟ๐ผ๐๐ฒ๐บ๐ฒ๐ป๐.
Richard Lee retweeted
Enterprises and individuals increasingly find computer-use agents powerful. But the cost of scaling usage weighs on them.
That's because these agents call on LLMs to reason every step and repeat. Our CTO Jiachen Yang told @miyadavid at @ISMG_News how we use neuro-symbolic to bring costs down:
The agent learns a workflow once, then repeats it with code, so it doesn't run a frontier model on every step.
On OSWorld, our @sai_borg was 217x cheaper than Opus alone, with a 72% success rate. Savings start after the third run, and Sai checks before each action.
As Jiachen points out, a single LinkedIn recruiting task costs at least 50 cents with today's best models. "If a recruiter wants to do it for thousands of people, I'm pretty sure at some point it will be unjustifiable in terms of the cost."
The full story here ๐f.mtr.cool/xyrmkelccm
Richard Lee retweeted
Ever wondered what goes into making @sai_borg more reliable?
@Richard_simular from @SimularAIโs research team was invited to BayLearn 2026. His presentation explored how computer agents can reuse successful steps and handle repetitive tasks more consistently.
For more on how agents learn reusable workflows, read the paper Richard coauthored: arxiv.org/abs/2609.36927
Thanks to Santa Clara University for hosting, and to San Josรฉ State University, UC Berkeley, and UC Santa Cruz for co-organizing the event ๐
#BayLearn2026 #AIResearch
Richard Lee retweeted
๐๐๐ญ๐ญ๐๐ซ ๐ซ๐๐ฌ๐ฎ๐ฅ๐ญ๐ฌ & ๐ฅ๐จ๐ฐ๐๐ซ ๐๐จ๐ฌ๐ญ, ๐๐ฌ ๐ฌ๐จ๐จ๐ง ๐๐ฌ ๐ฒ๐จ๐ฎ ๐ฉ๐ฎ๐ญ ๐๐ซ๐จ๐ง๐ญ๐ข๐๐ซ ๐๐ ๐๐ง๐ญ๐ฌ ๐จ๐ง ๐๐๐๐๐! ๐ฆฎ
Why use a frontier model at every step when a lightweight agent can handle lots of the work?
Introducing ๐๐๐๐๐: ๐๐ง๐๐๐ซ๐ญ๐๐ข๐ง๐ญ๐ฒ-๐๐ฐ๐๐ซ๐ ๐๐๐๐ค-๐ญ๐จ-๐๐ญ๐ซ๐จ๐ง๐ ๐๐ฌ๐๐๐ฅ๐๐ญ๐ข๐จ๐ง ๐๐ง๐ ๐๐ฏ๐๐ซ๐ฌ๐ข๐ ๐ก๐ญ.
LEASH unleashes the strong agent only when the weak agent's latent uncertainty signals the need, and keeps it on a leash afterward.
Two key challenges:
1๏ธโฃ ๐๐ก๐๐ง ๐ญ๐จ ๐๐ฌ๐๐๐ฅ๐๐ญ๐? Use the weak agent's latent uncertainty as an inexpensive signal that can be monitored at every step.
2๏ธโฃ ๐๐จ๐ฐ ๐ญ๐จ ๐ฆ๐๐ข๐ง๐ญ๐๐ข๐ง ๐จ๐ฏ๐๐ซ๐ฌ๐ข๐ ๐ก๐ญ ๐๐๐ญ๐๐ซ ๐๐ฌ๐๐๐ฅ๐๐ญ๐ข๐จ๐ง? Even frontier agents can go astray. And when served through black-box APIs, their internal states are inaccessible. LEASH enables continued oversight without accessing the strong agent's internals.
๐๐ก๐ ๐ค๐๐ฒ ๐ข๐ง๐ฌ๐ข๐ ๐ก๐ญ: Knowing when to escalateโand when to interveneโis just as important as knowing how to act.
The future of agentic AI isn't about using the strongest model everywhere. ๐๐ญ'๐ฌ ๐๐๐จ๐ฎ๐ญ ๐๐๐ฉ๐ฅ๐จ๐ฒ๐ข๐ง๐ ๐ข๐ง๐ญ๐๐ฅ๐ฅ๐ข๐ ๐๐ง๐๐ ๐ฐ๐ก๐๐ซ๐ ๐ข๐ญ ๐ฆ๐๐ญ๐ญ๐๐ซ๐ฌ ๐ฆ๐จ๐ฌ๐ญ.
๐ธ For computer-use agents, using a frontier model at every step is expensiveโbut relying on a small model alone is risky.
LEASH offers a middle ground: let a lightweight agent handle what it can, bring in a stronger model only when needed, and keep monitoring afterward. Put your frontier model on a LEASH !! ๐ฆฎ
๐ arxiv.org/pdf/2610.22280
Replying to @Richard_simular
@Richard_simular presenting neuro-symbolic computer use agents at @BayLearnOrg today!
Build your own personal agent within 10 minutes, using the Autonomous Computer API by @SimularAI
It also self optimizes token usage for repetitive tasks btw
youtu.be/doddTcFZMw8?si=YQWPโฆ
Richard Lee retweeted
Our MTS @Richard_simular, part of the team that built @sai_borg, answered 4 critical questions on personal agents on a panel at #SFTechWeek:
1. When should an agent think, when should it just act, and when must it stop and ask โ whatโs your decision architecture?
Start super strict then slowly relax the guard. User & personal agent have a continual "bonding" process, and people have different preferences so it's never going to a one size fits all
2. Do agents overthink into paralysis?
Usually it's not an agent, but a weak model overthinking.
Some models do this to benchmaxx so you will see an agent thinks for 4 minutes for a simple task like open google.com.
3. How do we know how much autonomy to give to an agent?
Running on sandbox is (almost) always the best approach. Factors: 1. how safe is sandbox 2. how reachable is the request to you 3. how quickly it learns and relax permission
4. With Muse Gadgets and OpenAI Dots launching last week, what does personal-agent decision architecture look like 12 months from now
How to give more information to agents. If we can find out a way to send explicit and exact request to agent then we don't need any context.
Thanks for having us on!
Insights on building reliable agents (they also happen to be very cost effective!)
Three years into making computer use agents, I kept asking myself:
Why hasn't CUA reached super intelligence yet?
@sai_borg (Sai), which happens to sound like SI, has invented a new way to build computer use agents: not just more reliable, but also 99% cheaper
The answer lies in how humans function: build muscle memory to optimize for efficient output over time
We released a paper today on this new paradigm, called neuro-symbolic
Give it a read here: simular.ai/articles/real-worโฆ
Our @Richard_simular will speak here and tell why your agents shouldnโt overthink too much
Weโre hosting an event about โNew era of Personal Agentโ at SF @Techweek_
partner with SEAMATE X @CoreSpeedHQ
And we invited๏ผ
โข @Richard_simular โ @SimularAI
โข @ZongZiWang โ @harvey
โข @haosu_cs โ @CoreSpeedHQ
โข @samuel_spitz โ @Replit
๐ Downtown SF
๐ Oct 8 ยท 5โ8 PM PT
See you there.
luma.com/seamate-tdd8
Day 1 of assembling my gtm team with Sai agent ๐
#SaiFleet #Robosecretary
Richard Lee retweeted
There are plenty of โhuman-pretending-to-be-an-agentโ game demos out there.
This one is real.
Sai + Jev played Minecraft, fought mobs, and spawned mining sub-agentsโall in real time.
1 hour. $0.01.
And Sai can spin up 100ร agents in parallel for under $1 total.
#RoboSecretary #SaiFleet
Agent demos fooled you ๐
It trips, it asks for help, and somehow you're the one copying, pasting and running the commands.
Meet your #Robosecretary. Meet your #SaiFleet.
Sai brings state-of-the-art computer use to your own computer - your apps, your OS, in a floating window that leaves your mouse alone.
Need more than one? Spin up to 100 cloud machines, each on the OS you pick, each on a different job. 100x the work, done without you.
Sai. The autonomous computer fleet at your command.
Free to start โ sai.simular.ai
Agent demos fooled you ๐
It trips, it asks for help, and somehow you're the one copying, pasting and running the commands.
Meet your #Robosecretary. Meet your #SaiFleet.
Sai brings state-of-the-art computer use to your own computer - your apps, your OS, in a floating window that leaves your mouse alone.
Need more than one? Spin up to 100 cloud machines, each on the OS you pick, each on a different job. 100x the work, done without you.
Sai. The autonomous computer fleet at your command.
Free to start โ sai.simular.ai
Pretty Serene
#robosecretary
Richard Lee retweeted
Our computer-use agent @sai_borg just beat Opus 5 and GPT-5.6 Sol on OSWorld 2.0.
Sai scored 73% on the CUA benchmark where each complex task takes a skilled human over an hour to execute.
And Sai did it at about 2/3 the cost per task of Opus and GPT ๐ธ
The unlock is our neurosymbolic framework that pairs the exploratory power of neural networks with the logic of symbolic code, resulting in reliability and cost saving.
Read the full report: simular.ai/articles/sai-topsโฆ
Richard Lee retweeted
Not @Richard_simular telling me @sai_borg's deterministic code replay drops token usage by 90%
For a repetitive task our accountant hates ๐ค
Agent S3, the first computer-use agent that surpasses human-level performance in OSWorld v1 benchmark, has now been accepted to @TmlrOrg (Transactions on Machine Learning Research).
Congratulations to the @SimularAI team!
@chalo2000, vincent, @Richard_simular, jiachen, @xwang_lk
Stay tuned for what's next!
๐ Introducing ๐๐ ๐๐ง๐ญ ๐3, the most advanced computer-use agent, now ๐๐ฉ๐ฉ๐ซ๐จ๐๐๐ก๐ข๐ง๐ ๐ก๐ฎ๐ฆ๐๐ง-๐ฅ๐๐ฏ๐๐ฅ ๐ฉ๐๐ซ๐๐จ๐ซ๐ฆ๐๐ง๐๐๐ง ๐ป
Just one year ago, Agent S scored ~20% on OSWorld: SOTA then, but far from human 72%.
Today, Agent S3 reaches 6ฬณ9ฬณ.ฬณ9ฬณ%ฬณ (โฌ10% over prior SOTA), nearly matching humans on computer use.
Few imagined this gap could close so fast, but with steady progress and ๐๐ข๐ซ๐ฌ๐ญ-๐ฉ๐ซ๐ข๐ง๐๐ข๐ฉ๐ฅ๐ simplicity, @SimularAI did it in just one year.
๐งช The sฬตeฬตcฬตrฬตeฬตtฬต ๐จ๐ฉ๐๐ง ๐ซ๐๐๐ข๐ฉ๐:
โ ๐๐๐ก๐๐ฏ๐ข๐จ๐ซ ๐๐๐ฌ๐ญ-๐จ๐-๐ โ scaling agents ๐ต๐ฉ๐ฆ ๐ณ๐ช๐จ๐ฉ๐ต ๐ธ๐ข๐บ
โ Simpler design, stronger baseline
โ 100% ๐จ๐ฉ๐๐ง ๐ฌ๐จ๐ฎ๐ซ๐๐ (as always)
Learn more at simular.ai/articles/agent-s3 & ๐
























