WebAgentLab is building an open-source community focused on Web Agent and the broader GUI Agent field.

join to contribute ๐Ÿ‘‰
Joined November 2024
Include

Only show posts containing:

Exclude

Hide posts containing:

Time range
-
Minimum likes
CUA-Lite: An Open Platform for Computer Use Agents Zhanhui Zhou, Weichen Zhang, Haoran Liu, Lingjie Chen, Tianneng Shi, Kevin Lin, Zhengyuan Yang, Lijuan Wang, Dawn Song arxiv.org/abs/2610.26071 [๐šŒ๐šœ.๐™ฐ๐™ธ]
1
1
157
WebAgentlab retweeted
People long believed in human-like computer use. But software (even terminals) is built for humans, not AI agents. Why not let agents build their own? Meet ๐—”๐—ด๐—ฒ๐—ป๐˜ ๐—ฆ๐Ÿฐ, an agent scientist evolving its own interfaces, letting an open 27B model beats GPT-5.5 & Opus 4.8. With its evolved interfaces, the same model improves ๐Ÿฏ.๐Ÿดร— and costs <๐Ÿฑ% as much per task as either frontier model. Agents today read screenshots and act through click, scroll, type, maybe a shell. So an agent fills a spreadsheet cell by cell when one line of code would do, or prints two thousand lines of a file to find one function. Every new app or model release means a human reads trajectories, diagnoses failures, and hand-writes new primitives. Skill-distillation methods can reorder what the interface allows, but they canโ€™t extend it. Agents compensate with more test-time compute instead of a better interface. ๐— ๐—ผ๐—ฑ๐—ฒ๐—น๐˜€ ๐—ถ๐—บ๐—ฝ๐—ฟ๐—ผ๐˜ƒ๐—ฒ; ๐—ถ๐—ป๐˜๐—ฒ๐—ฟ๐—ณ๐—ฎ๐—ฐ๐—ฒ๐˜€ ๐˜€๐˜๐—ฎ๐˜† ๐—ณ๐—ฟ๐—ผ๐˜‡๐—ฒ๐—ป. โ€œWe want AI agents that can discover like we can, not which contain what we have discovered.โ€ โ€” Rich Sutton, The Bitter Lesson Hand-designed interfaces are what we have discovered. Agents can read their own trajectories, spot whatโ€™s missing, and write and test code. ๐—œ๐—ป๐˜๐—ฒ๐—ฟ๐—ณ๐—ฎ๐—ฐ๐—ฒ ๐—ฑ๐—ฒ๐˜€๐—ถ๐—ด๐—ป ๐˜€๐—ต๐—ผ๐˜‚๐—น๐—ฑ ๐—ฏ๐—ฒ ๐—ฎ๐—ป ๐—ฎ๐—ด๐—ฒ๐—ป๐˜ ๐—ฐ๐—ฎ๐—ฝ๐—ฎ๐—ฏ๐—ถ๐—น๐—ถ๐˜๐˜†. Thatโ€™s what Agent S4 does. Agent S4 makes a ๐—ฟ๐—ฒ๐—ฝ๐—ผ๐˜€๐—ถ๐˜๐—ผ๐—ฟ๐˜† ๐—ผ๐—ณ ๐—ถ๐—ป๐˜๐—ฒ๐—ฟ๐—ณ๐—ฎ๐—ฐ๐—ฒ๐˜€ ๐—ฎ๐—ป๐—ฑ ๐˜‚๐˜€๐—ฎ๐—ด๐—ฒ ๐—ธ๐—ป๐—ผ๐˜„๐—น๐—ฒ๐—ฑ๐—ด๐—ฒ the unit of evolution. A task agent attempts synthetic tasks with no success labels or verifiers. A meta-agent studies the trajectories, forms hypotheses about which actions, observations, or knowledge would help, and builds descendants through mutation or crossover. Validation tasks the meta-agent never sees decide which repositories survive. The model and harness stay fixed, and the meta-agent ๐—ป๐—ฒ๐˜ƒ๐—ฒ๐—ฟ ๐˜€๐—ฒ๐—ฒ๐˜€ ๐—ฎ ๐—ฏ๐—ฒ๐—ป๐—ฐ๐—ต๐—บ๐—ฎ๐—ฟ๐—ธ ๐˜๐—ฎ๐˜€๐—ธ. ๐—ข๐—ป๐—น๐˜† ๐˜๐—ต๐—ฒ ๐—ถ๐—ป๐˜๐—ฒ๐—ฟ๐—ณ๐—ฎ๐—ฐ๐—ฒ ๐—ฒ๐˜ƒ๐—ผ๐—น๐˜ƒ๐—ฒ๐˜€. With the evolved repository, Qwen3.8-27B goes from ๐Ÿฑ.๐Ÿฒ% to ๐Ÿฎ๐Ÿญ.๐Ÿฏ% on OSWorld V2, ๐Ÿฏ.๐Ÿดร— the same model with fixed interfaces and a new state of the art for open-source computer-use agents. It beats GPT-5.5 (13.0%, $25.50/task) and edges past Opus 4.8 (20.6%, $72.40/task) at $๐Ÿญ.๐Ÿญ๐Ÿฌ per task. On ScienceBoard, the same model goes from 41.4% to 58.6%. The analysis is the more interesting part. Evolution mostly deepens interfaces the agent already has: 4 new interfaces and 127 extensions. Give the meta-agent a scratch VM to test its code, and it builds 12 new interfaces and 253 extensions, reaching music scores, audio projects, and 3D scenes. The best repository came from crossing two earlier branches. In one case study, the seed agent hand-builds a browser client mid-task and debugs through failure after failure; the evolved agent calls page_text and reaches the same message at tool call ๐Ÿญ๐Ÿญ instead of ๐Ÿฏ๐Ÿณ. Human-like computer use was a starting point, not the destination. The interface an agent acts through is a second axis of improvement, beyond better models and harnesses, and agents can now design it themselves. ๐—˜๐˜ƒ๐—ผ๐—น๐˜ƒ๐—ถ๐—ป๐—ด ๐—ฎ๐—ด๐—ฒ๐—ป๐˜ ๐—ถ๐—ป๐˜๐—ฒ๐—ฟ๐—ณ๐—ฎ๐—ฐ๐—ฒ๐˜€ ๐—ถ๐˜€ ๐—ฎ ๐—ป๐—ฒ๐˜„ ๐—ฑ๐—ถ๐—บ๐—ฒ๐—ป๐˜€๐—ถ๐—ผ๐—ป ๐—ผ๐—ณ ๐—ฎ๐—ด๐—ฒ๐—ป๐˜ ๐˜€๐—ฒ๐—น๐—ณ-๐—ถ๐—บ๐—ฝ๐—ฟ๐—ผ๐˜ƒ๐—ฒ๐—บ๐—ฒ๐—ป๐˜.
15
38
8
121
6,842
WebAgentlab retweeted
ClaudeDevs
@ClaudeDevsOct 7
Computer use and browser use toolsets are now built into the Python and TypeScript SDKs for Claude. The API tells you what Claude wants to click or type. Previously, you had to write your own loop and map clicks and keystrokes to commands, but the SDKs now run the loop and send actions to drivers.
134
300
61
5,326
442,301
incredibly based that Xiaomi MiMo released their RL dataset link: huggingface.co/datasets/Xiaoโ€ฆ
15
104
7
1,241
72,965
WebAgentlab retweeted
1/ Today we're announcing the stable Cua Driver release for Omarchy - a new foundation for computer use, built into the OS from the ground up. Over the last month, we worked directly with @dhh, @SpencerGBull and @vaxryy to bring a native synthetic cursor to Omarchy's Hyprland compositor, enabling true multi-cursor computer use at the OS level. Cua Driver is open source and available at github.com/trycua/cua
118
167
57
1,961
341,214
WebAgentlab retweeted
1/ You can now run OSWorld tasks on Cua Cloud Fleets. Our new guide shows how to turn the official OSWorld Ubuntu image into a Fleet desktop with Cua Driver, then run your agent and evaluate the tasks. Get started: run.cua.ai/?utm_source=x&utmโ€ฆ
1
3
1
20
6,381
WebAgentlab retweeted
There is no benchmark tells you if your agent could resolve your mechanical CAD design in accurate, editable, reliable, efficiently. We designed the benchmark CADWORLD from professional prospective that really focused on the engineers workflow. arxiv.org/abs/2609.16251
1
1
5
1,523
WebAgentlab retweeted
Check out my first music video ๐Ÿ–๏ธ Plus, accepting applications for hot (robo)secretaries Want yours?
99
51
39
344
142,614
WebAgentlab retweeted
We just raised $5.7M for @PolarBrowser, the AI browser that beats Anthropic and OpenAI on every major web agent benchmark. - 4,500,000+ actions taken for users, automating sales, recruiting, and ops - One company cancelled Clay and saves 25+ hrs/wk per person - Team is from MIT, YC, Prod, Citadel, Jane Street, Perplexity, Modal, and Apple 100 hours of work. From 30 seconds of typing. Download the world's most powerful AI browser: polarbrowser.com
1,112
1,062
167
14,625
16,145,845
WebAgentlab retweeted
New blog post: What's the Point of Computer Use Agents? There's been a lot of hype about CUAs recently. But where do CUAs shine? When should we use CUAs over API- or text-based agents? And what are the remaining research problems? jykoh.com/blog/whats-the-poiโ€ฆ
13
39
10
195
26,915
WebAgentlab retweeted
Introducing ApprenticeBench: computer use + continual learning on a real job. We show Fable 5.1 and GPT-6 Astra can now continually learn on a job and surpass human professionals. A decisive step change in AI's job readiness. No FDEs. Agents deploy themselves into the job. ๐Ÿงต
34
169
51
1,226
231,051
WebAgentlab retweeted
Watch Fable 5.1 agent onboards itself in the accounts payable department at a simulated California-based construction company. Learn the ERP system, conventions from company history, and lifelong learning from feedback. A glimpse into what the future of work may look like.
1
9
4
109
24,075
WebAgentlab retweeted
OpenAI
@OpenAISep 3
1,978
3,448
2,450
38,858
11,407,635
35 of the 88 RSI-Exam tasks are now public! ๐ŸŽ‰ Weโ€™re continuously expanding RSI-Exam and always welcome new contributors across fields. Bring a challenging task from your own research area: ๐Ÿ”— rsi-exam.ai/contribute.html
Can frontier AI agents achieve Recursive Self-Improvement by turning a weak method into one that performs better on hidden data? Introducing RSI-Exam ๐Ÿงต: 88 executable research tasks across 6 domains, now open for task contributions from every field๐Ÿง‘โ€๐Ÿ”ฌ. Its task bank spans virtual cells, TPU kernels, chip design, quantitative finance, agent harnesses, model distillation, and much more. Across these tasks, every rollout follows the same core protocol: ๐Ÿ” An agent inherits a working method and iterates using visible data. ๐Ÿ”’ Only its final artifact enters a fresh-container evaluation on the hidden set. ๐Ÿ“Š The result so far: Opus 5 leads the 88-task leaderboard with a mean hidden-set score of 0.464. ๐Ÿ“ข Want to help build the next release? Author an executable task, review one, or audit a rollout. Substantial contributions qualify for paper authorship. Special thanks to @XinyeYee for the tremendous effort in leading this project. Explore: rsi-exam.ai/ Blog: rsi-exam.ai/blog.html Contribute: rsi-exam.ai/contribute.html GitHub: github.com/aiming-lab/RSI-Exโ€ฆ
5
23
3
163
26,049
WebAgentlab retweeted
We have two very exciting things cooking right now. One of them we can tell you about: weโ€™re teaming up with Together AI and Crusoe to bring together researchers and builders across the computer-use agent stack for an evening in San Francisco this Thursday, August 27. Models, data, RL environments, inference, sandboxes etc. Plus drinks, bites, and folks from Meta, DeepMind, OpenAI, Mistral, Thinking Machines, Microsoft AI, and fast-growing startups. Come hang out with us โ†’ luma.com/aqmwya3d The other thing? More soon ๐Ÿ‘€
2
6
1
17
4,349
WebAgentlab retweeted
Dev
@DevvMandalAug 21
Today, we're launching the world's largest open-source CAD dataset for computer-use. 1,000+ hours across SolidWorks, Siemens-NX, AutoCAD & more. Screen recordings, mouse/keyboard actions, input diagrams, output files, rubrics and narrations. Link in the comments :)
42
149
29
1,420
159,789
Wuying-Browser-Agent: Real-World Centric Fundamental Long-Horizon Browser Agents AIMAE Team, Tianxiang Chen, Yan Cheng, Zhangye Han, Xiaowei Li, Chang Liu, Cheng Liu, Zhongqiang Ma, Long Peng, Xiaobing Tu, Yinggui Wang, Hongliang Wei, โ€ฆ arxiv.org/abs/2608.17319 [๐šŒ๐šœ.๐™ฐ๐™ธ]
1
2
281
WebAgentlab retweeted
Dev
@DevvMandalAug 18
Today, our computer-use datasets crossed 50,000+ hours. Screen recordings + synced mouse/keyboard actions + narrations across CAD, design, browser-use & more. Sourced from humans and available off the shelf today :) Samples below.
15
15
3
238
83,629
Today we release my favorite episode of Training Data yet: the great Rich Sutton. @RichardSSutton wrote the textbook, wrote The Bitter Lesson (and many other on-point essays like "Self-Verification, The Key to AI"), and trained a mafia of talented students who went on to change the AI landscape forever including David Silver, inventor of built AlphaGo. @kjaved_ was Rich's PhD student at Alberta and wrote The Big World Hypothesis. They just left academia to start @oaklab Their core argument: (1) The Bitter Lesson: the world is massively more complex than any model of it, so anything trained on human-curated data has a ceiling (2) Continual Learning: intelligence is continual by definition, and today's models stop learning the moment they ship. The conversation covers: โ€” what The Bitter Lesson actually says, and what people get wrong โ€” why synthetic data is "just a big mistake," and the Big World Hypothesis behind it โ€” how LLMs are both a positive and a negative example of his own essay โ€” why no animal learns by supervised learning, and what squirrels can do that we can't โ€” the cure for catastrophic forgetting: per-weight step sizes and continual backprop โ€” why the biggest labs can't take a path where performance gets worse before it gets better โ€” a trillion parameters on 20 watts, and the Moore's Law math that makes it plausible โ€” why the endpoint isn't one mind but one design, running as many minds It was both a fun generative idea- and debate-filled conversation, and a surprisingly human one too. Rich, thank you for beating cancer and changing the trajectory of AI. ๐Ÿ’™ 00:00 Introduction 02:10 An AI winter, a cancer diagnosis, and the move to Alberta 07:07 Writing "The Bitter Lesson," and what people get wrong 09:53 Are LLMs a positive or a negative example of it? 11:03 Synthetic data is "just a big mistake," and the Big World Hypothesis 18:01 AlphaGo, human priors, and why prior knowledge and learning should be friends 22:37 "Their weights never change": do LLM assistants actually learn? 26:09 Babies, squirrels, and why no animal learns by supervised learning 32:02 Rockets, imagination, and where paradigm shifts come from 36:42 The Alberta Plan and its 12 steps 38:53 Catastrophic forgetting and the cure 43:43 Oak's biggest ambition: a self-maintaining mind 47:56 Why the big labs are stuck in a local minimum 49:13 If everything goes right: LLMs, many minds, and hiring The man who pioneered reinforcement learning thinks the rest of the field is weird, and lays it all out in today's episode. Together w/ @Alfred_Lin @sequoia
23
77
14
582
120,530
WebAgentlab retweeted
1/ Today, we're releasing the first open-source Computer History - now in early preview for Cua Driver on macOS, Windows, and Linux. It gives agents an encrypted, local record of actions they took through Cua Driver, so new sessions can recover useful context from earlier work.
41
150
41
1,701
249,549