Research Lead @yutori_ai. Prev, post-training lead of Llama 3 multimodal @AIatMeta. Earlier, @FAIR, @CarnegieMellon
Joined September 2016
- Tweets137
- Following283
- Followers291
- Likes684
Pinned Tweet
Excited to share that Navigator n2 is officially out!
The core idea behind n2 is simple: models should use computers the computer way, not the human way.
There’s no single best interface. GUI, CLI, tools, code — pick the right one at every step. The hard part is making those choices well over long horizons: knowing when to switch, when not to, and how to keep making progress over hundreds of steps.
The training loop ended up following the same idea: training computer-use models is itself a computer-use task.
We use computer-use agents to explore applications, create tasks, test verifiers, find reward hacks, edge cases, and where the latest model fails. Those failures shape the next round of training data.
It creates a pretty natural recursive loop: better models make better data, which makes better models.
The result is a frontier-level model at a fraction of the cost— 65.2% on OSWorld 2.0 at $1.46 per task.
Very proud of the team and where n2 landed!
(life) probably not many sim racers have tried this: Codex Computer Use helped me configure AC on a fresh Windows install, with me handling the physical part. we got this list done in about an hour:
✓ content manager, CSP & Pure LCS installation
✓ weather controller and PP filter configs
✓ wheel, pedal and shifter mapping
✓ steering and pedal calibration
✓ display settings
✓ driving assist configs
✓ force feedback tuning
✓ perf benchmarks and config backups
turned a notoriously tedious setup into an efficient & enjoyable process. highly recommended
Rui Wang retweeted
Visiting SF for COLM and curious what SF is all about?
The bad news: COLM is in the dodgiest neighborhood and if you hang around there, you’ll hate SF just like I used to.
The good news: your twitter buddy Jason Wei wrote a guide on experiencing SF like a local: jasonwei.net/blog/the-best-t…
Rui Wang retweeted
Today, Carnegie Mellon University and Ken Griffin, founder and CEO of Citadel and founder of Griffin Catalyst, announced a historic $3 billion gift to elevate Carnegie Mellon in Pittsburgh and launch Carnegie Mellon University Miami. #CMUMiami
Learn more: cmu.edu/news/stories/archive…
Rui Wang retweeted
Computer-use agents are extremely good!
But they need to improve on a metric which most benchmarks don't track. How much time do CUAs take to complete a task?
Introducing cua-speedrun: a benchmark for accuracy, speed, and cost under matched infra.
cuaspeedrun.com
👇🧵
Rui Wang retweeted
Today we're excited to announce @HalluminateAI's $30M Series A led by @oakhcft with participation from @YCombinator, @orangecollectv, @FTPartners, @heavybit, and more.
7/ Computer use will advance on two frontiers:
The first frontier is open-ended knowledge work. The bottleneck is quality, not compute: valuable tasks require domain expertise from task design through verification. Coding environments already sell for thousands of dollars; realistic end-to-end knowledge work environments could soon be worth hundreds of thousands.
The second is digital automation at scale. This one is getting less attention than it should. Much valuable digital work is made of small, repetitive tasks too variable for scripting, like validating promo codes across O(10k) websites. Each task may be worth only cents, but at scale the value is enormous. The challenge is frontier-level performance at a cost that justifies the business.
Typesafe AI is a great attempt at the latter. Excitement aside, I’m sure some folks have already realized the limitations when it comes to computer use—the problems aren’t new. I might expand on this separately.
8/ To sum up:
Two years ago, “computer use” named a research problem: can a model use a computer like us?
Now it can.
That was a milestone, not the destination.
A computer offers much more than its screen.
Using a computer should mean using ALL of it.
More in:
xitter.cf/theruiwang/status/2099…
Rui Wang retweeted
As a PhD in mathematics who now works in AI/ML, the recent Navier–Stokes news may bring both excitement and unease to the math community. My PhD advisor, Yizhao T. Hou, has spent more than two decades studying blowup in the Euler equations with his students and collaborators.
Rui Wang retweeted
A bit late on this but congratulations to the @yutori_ai team for taking #1 on the MyPCBench leaderboard 🙂 Happy to see more work on pushing forward personalization + computer-use!
The evidence lies at mypcbench.com/leaderboard
Rui Wang retweeted
This was wonderful :) Thank you all for coming! 🩵
An evening in SF with food, drinks, and conversation about all things CUA (computer-use agents).
Research, models, data, RL environments, inference, sandboxes.
With researchers and builders from frontier labs, big tech, fast growing startups.
Thursday.
Come join us!
@yutori_ai is co-hosting with @togethercompute and @CrusoeAI 💙
luma.com/aqmwya3d
Excited to see this
Introducing CUA-Lite 🧵 — an open platform for computer-use agents.
Training and benchmarking CUAs (Computer-Use Agents) requires four core pieces:
1️⃣ Agents — the models and the scaffolding that drives them
2️⃣ Environments — runtime/sandboxes for agents to interact with, tasks & verifiers/graders
3️⃣ Traces — records of agent trajectories
4️⃣ Frameworks — to evaluate, SFT & RL-train agents
Today, all four are fragmented.
Every agent ships with its own implementation, often in a separate repo — there is no unified way to run them all. Every environment exposes its own interface and action space, often requiring an expensive VM sandbox for each verifiable task. Traces come in incompatible formats. And without common standards across the stack, every project ends up rebuilding its own tooling/framework for eval, SFT, and RL.
CUA-Lite unifies the stack:
→ One standardized interface & action space for agents and environments
→ One standardized format for agent traces
→ One framework for evaluation, SFT & RL
→ Across desktop, browser & mobile
And open resources plug straight in, creating the largest open collection of CUA agents, environments and traces, all in a unified format:
🤖 10+ CUAs, including GPT, Claude, Gemini, Qwen, Muse-Glimmer, UI-TARS
🌐 15+ benchmarks, including OSWorld, WebArena & AndroidWorld
⚡ Optional VM-free sandboxes with 30K+ verifiable tasks for training
📚 10+ trace datasets, freely available on Hugging Face, including public datasets converted into the standardized format and fresh rollouts from frontier open-weight CUAs
Led by @BerkeleyRDI , our goal is for CUA-Lite to become a community-driven, open-source ecosystem for computer-use agents.
Join the community and contribute today: bring an environment (runtime/sandbox + tasks + verifier), traces, or an agent, and plug it into CUA-Lite!