Pinned Tweet
From OSWorld 1.0 to 2.0, we went from minutes (~30 steps) to hours (~318), from single apps to real workflows, from high scores (83%) to hard problems (21%).
1+ year, 20+ people, every task rigorously verified. This is what real cua evaluation takes.๐
๐osworld-v2.xlang.ai
Two years ago, we built OSWorld 1.0 โ the benchmark that became the standard for computer-use agents. Agents now score 83.5% on it. Problem solved?
Not even close.
๐Today we introduce OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks.
What's new:
๐ฏ 108 real-world workflows, each ~1.6 hours โฑ๏ธ for a skilled human
โ๏ธ ~318 tool calls/task vs. ~30 in OSWorld 1.0
๐ Grounded in authentic artifacts & stateful user profiles
โก Captures real phenomena: dynamic environments, streaming interaction, cross-source reasoning, implicit-state inference & more
๐ Best results: Claude Opus 4.8 reaches the highest accuracy at 20.6%, while GPT-5.5 is far more token-efficient but plateaus near 13%. No one is close to solving real computer use.
๐ Homepage: osworld-v2.xlang.ai
๐ Paper: github.com/xlang-ai/OSWorld-โฆ
๐ป Code: github.com/xlang-ai/OSWorld-โฆ
๐ค Dataset: huggingface.co/datasets/xlanโฆ
๐งต [1/8]
Tao Yu retweeted
We launched ๐ (Muse Wonder) and ๐ Ultra (Muse Wonder Full) today!
What excites me most is how much closer weโre getting to agents you can actually delegate work to. Personal agents that understand you and your life, and professional agents that can take on complex, long-running projects. ๐ is designed for personal and professional agent use
Pretty Wonder Full to see this come together just a year after starting the agent team at TBD Labs. Huge credit to the incredible team that made it happen!
Tao Yu retweeted
MUSE FEATURE ALERT: muse for mac now has computer use! queue up your jobs, walk away, and it keeps going.
muse loves laptop โฅ๏ธ
Tao Yu retweeted
Introducing ApprenticeBench: computer use + continual learning on a real job.
We show Fable 5.1 and GPT-6 Astra can now continually learn on a job and surpass human professionals. A decisive step change in AI's job readiness.
No FDEs. Agents deploy themselves into the job. ๐งต
Tao Yu retweeted
1/ we just publicly released Muse Spark 1.3 max!
we see significantly stronger coding and agentic performance on muse spark 1.3 max, so would strongly recommend trying it out even if you've already tried muse spark 1.3 high or muse spark 1.3 xhigh.
Tao Yu retweeted
Muse Spark 1.3 is rolling out today with frontier performance almost too cheap to meter. This is the biggest jump we've made so far on coding and agentic work. Try it in Muse Code and our API.
Next up ๐ and Muse Spark open weights releases coming soon.
Tao Yu retweeted
1/ today weโre releasing muse spark 1.3โavailable in muse code & the meta model api.
this is our most capable model yetโfrontier performance almost too cheap to meter. much stronger at agentic and coding with better usability. we think users will really notice the jump.
๐thrilled to introduce muse spark 1.3 ๐ฅ โ a major leap in coding and agentic capabilities, including computer use. give it a try!
โnext up ๐ and muse spark open weights releases coming soon.โ ๐
Tao Yu retweeted
Three ๐ฅ in a row from Muse Spark 1.1 to 1.3 within two months. Huge momentum of the team pushing on the model capability especially on coding, agent, instruction following, and long context capability. Try it out at developer.meta.com/ai/modelsโฆ and let us know what you think.
Tao Yu retweeted
Weโre excited to release Muse Spark 1.3 with improved performance on agentic and coding tasks, and a focus on real-world usability.
Key capabilities:
โ Sustains longer-horizon work across multiple workflows in a single thread
โ More actively collaborates with users: it asks clarifying questions, flags when it's stuck, confirms before consequential actions
โ Better calibrated on its own limits instead of hallucinating outcomes
โ ~20% fewer tool calls and ~25% fewer tokens vs. Muse Spark 1.2 in internal comparisons
Tao Yu retweeted
๐๐๐๐ญ ๐ญ๐ก๐ ๐ฐ๐จ๐ซ๐ฅ๐'๐ฌ ๐๐๐ฌ๐ญ ๐๐จ๐ฆ๐ฉ๐ฎ๐ญ๐๐ซ-๐ฎ๐ฌ๐ ๐๐ ๐๐ง๐ญ, ๐๐๐ข. ๐ป
Sai now ๐ญ๐จ๐ฉ๐ฌ ๐๐๐๐จ๐ซ๐ฅ๐ 2.0 ๐ฐ๐ข๐ญ๐ก ๐ 73% ๐ฌ๐ฎ๐๐๐๐ฌ๐ฌ ๐ซ๐๐ญ๐, outperforming GPT-5.6 Sol and Claude Opus 5 ๐๐ญ ๐ฌ๐ข๐ ๐ง๐ข๐๐ข๐๐๐ง๐ญ๐ฅ๐ฒ ๐ฅ๐จ๐ฐ๐๐ซ ๐๐จ๐ฌ๐ญ.
What excites me most is that OSWorld 2.0 moves beyond short benchmark tasks toward longer, more realistic computer workflows, where reliability, planning, memory, and efficiency really matter.
This is exactly the direction we have been pursuing at Simular: ๐๐ฎ๐ข๐ฅ๐๐ข๐ง๐ ๐๐ฎ๐ญ๐จ๐ง๐จ๐ฆ๐จ๐ฎ๐ฌ ๐๐จ๐ฆ๐ฉ๐ฎ๐ญ๐๐ซ๐ฌ ๐ญ๐ก๐๐ญ ๐๐๐ง ๐ซ๐๐ฅ๐ข๐๐๐ฅ๐ฒ ๐ก๐๐ง๐๐ฅ๐ ๐ซ๐๐๐ฅ ๐ฐ๐จ๐ซ๐ค, ๐ง๐จ๐ญ ๐ฃ๐ฎ๐ฌ๐ญ ๐๐๐ฆ๐จ๐ฌ.
Proud of the team! There is still a long way to go toward reliable, efficient, superhuman computer use, but it is closer and closer!
Our computer-use agent @sai_borg just beat Opus 5 and GPT-5.6 Sol on OSWorld 2.0.
Sai scored 73% on the CUA benchmark where each complex task takes a skilled human over an hour to execute.
And Sai did it at about 2/3 the cost per task of Opus and GPT ๐ธ
The unlock is our neurosymbolic framework that pairs the exploratory power of neural networks with the logic of symbolic code, resulting in reliability and cost saving.
Read the full report: simular.ai/articles/sai-topsโฆ
Tao Yu retweeted
Excited to share Muse Spark 1.2 and Muse Code (beta). Muse Spark 1.2 is an upgrade to 1.1 with overall improvements in coding and general agents. Muse Code is our harness designed for efficiently and robustly solving complex software engineering tasks. Try the bundle out!
research.meta.ai/blog/introdโฆ
Tao Yu retweeted
We trained a new model and released our first coding agent harness!
Try it: curl -fsS dev.meta.ai/install.sh | bash
muse code in beta is live. first coding agent from msl, built on muse spark 1.2.
install: curl -fsS dev.meta.ai/install.sh | bash
here's what you should know:
Tao Yu retweeted
Today we're releasing Muse Code (beta) and Muse Spark 1.2. Muse Code is our first coding agent โ it coordinates multiple persistent subagents to complete complex engineering tasks faster, more accurately, and with less intervention. Muse Spark 1.2 is our newest model, co-trained with Muse Code for best performance when paired together. Available now through Meta Model API.
Try it: curl -fsS dev.meta.ai/install.sh | bash
Tao Yu retweeted
๐ป Meet Qwen-CUA โ our native computer-use agent for (almost) everything.
Code, APIs, and computer use are three of the most important interfaces for agents. Todayโs models are already highly capable with the first two. Qwen-CUA is built to unlock the third: graphical interfaces designed for people.
๐ Native Perception โ screenshots only. No DOM, accessibility tree, or other hidden machine-readable state.
๐ฑ๏ธ Native Interaction โ keyboard and mouse events across browsers, desktop apps, and professional software. No task-specific APIs.
๐ง Native Intelligence โ maintains long-horizon visual context, verifies progress, and learns from large-scale interactive experience with verifiable outcomes.
To make native computer use trainable at scale, we built approximately 40K verifiable tasks and rollout infrastructure with nearly 100K vCPUs, supporting tens of thousands of concurrent environments.
Across eight computer-use benchmarks spanning everyday desktop use, long-horizon workflows, personalized computing, scientific research, web interaction, macOS, and adversarial robustness, Qwen-CUA demonstrates strong and broadly competitive capabilities. Scaling the same recipe to Qwen-CUA-Max pushes this frontier further.
Not just โclicking the screenโ โ native computer use unlocks software and workflows that previously required a human at the keyboard. Together with code and APIs, it completes the interface stack for more general agents.
Joint work by Qwen Team ร XLang Lab.
๐ Technical Report:
github.com/xlang-ai/Qwen-CUAโฆ
๐ป Code:
github.com/xlang-ai/Qwen-CUA
Tao Yu retweeted
Congrats the Claude team on the big jump on OSWorld!
Worth flagging this 70.6% is partial-credit scoring โ under strict binary task-level success (OSWorld 1.0-style), Opus 5 is still only ~30%.
We're working on improving task quality so task-level binary success becomes more robust and reliable even on hard tasks too.
Tao Yu retweeted
Really impressive to see Opus 5 hit 70.6% on OSWorld 2.0! ๐
Every time we build a harder eval, models catch up sooner than expected. Keeping evals ahead is getting harder and harder๐คฏ
Tao Yu retweeted
Exciting to see progress in long-horizon computer use with the GPT-5.6 release.
Glad that OSWorld 2.0 is being used to measure end-to-end computer-use workflows in realistic, complex environments.
Computer use is moving fast. Looking forward to more results from the community๐
Tao Yu retweeted
Excited to share what weโve been building at Meta Superintelligence Labs!
Today weโre launching Muse Spark 1.1, our strongest model yet for complex agentic workflows โ delivering massive gains in agents, computer use, coding, multimodal reasoning, and multi-agent orchestration.
This is also our first API release! Weโd love to hear your feedback.
And weโre just getting started, super proud of the team behind it, and the larger models are training right now. ๐
ai.meta.com/blog/introducingโฆ
Tao Yu retweeted
Muse Spark 1.1 is industry competitive on real world long-horizon computer use across desktop, browser, and mobile
