To clarify things: I may have been unfair to the model.
Usually, when a new model drops, everyone tests it to its full potential. My previous prompt was also weak, so I improved it and tested Muse Wonder Max again.
I know I might get some hate for admitting this, but I was unfair to the model. It’s still not on Opus’s level, in my opinion, but I didn’t test it properly the first time.
Shikai Li retweeted
Introducing Muse Wonder, our new family of frontier models with state of the art performance for personal and professional agentic work.
Shikai Li retweeted
Latest @SemiAnalysis_ article comparing coding agent subscription plans show that @Muse Code's $50 plan delivers 51.3x the value per dollar, topping the API value leaderboard.
The $50 plan gives a comparable amount of token value as both the Codex $200 and Claude $200 plans. For good coding performance (give max reasoning effort a spin!) + affordable and fast inference, it's hard to argue anything beats Muse Code right now.
To users hitting limits on this tier, we hear you on wanting a higher subscription option and will be sharing more on this soon!
Download Muse Code at dev.meta.ai
Full SemiAnalysis post: newsletter.semianalysis.com/…
Replying to @SemiAnalysis_
We did the same analysis on subscription plans offered by Meta, SpaceX, Cursor, Cognition, MiniMax, Zdotai, and Moonshot as well. For more details, see our newsletter! (6/6) newsletter.semianalysis.com/…
Shikai Li retweeted
Which model is benchmaxxing?
If a model is good across different measurements, maybe it’s just good
🤖 Made with AI
Announcing Artificial Analysis Intelligence Index v4.3, upgrading Terminal-Bench to 4.0 and adding AutomationBench-AA, an agentic workflow automation benchmark with a private test set. This is a continuation of our rollout of Intelligence Index v5
Changelog (Index v4.2 → Index v4.3):
➤ Terminal-Bench: 2.1 → 4.0, completing our upgrade to the latest version of Terminal-Bench
➤ Replacing 𝜏³-Banking with AutomationBench-AA, our implementation of Zapier's business workflow automation benchmark
We are continuing to prioritize keeping Intelligence Index as useful as possible by bringing forward a subset of the changes we had planned for Index v5. Each change in v4.2 and v4.3 stands on its own merits and brings the Index closer to real-world problem solving, adds more private test sets to prevent gaming, and reduces saturation
Intelligence Index v4.3 raises the difficulty of agentic coding tasks and broadens the types of agentic workflows tested. Because we use a held-out test set for AutomationBench-AA, in collaboration with @zapier, the weight assigned to evaluations with private tasks or answers increases from 40% to 45%. Category weights are unchanged from v4.2: Agents 30%, Coding 20%, General 30%, Scientific Reasoning 20%
Detailed changes:
➤ Upgraded Terminal-Bench 2.1 to 4.0: 66 multi-step tasks testing agents on tasks run in agent sandboxes driven via the terminal, including tasks involving software engineering, machine learning, science, and operations. The 4.0 update recalibrates compute and time allowances, and improves task instructions and verification. We have changed from the Terminus 2 harness to mini-SWE-agent, a minimal, model-agnostic harness. We will also be updating our Coding Agent Index, where we test model and harness pairs, to include Terminal-Bench 4.0 soon
➤ Replaced 𝜏³-Banking with AutomationBench-AA: Our implementation of Zapier’s AutomationBench tests agents on 657 business workflows across simulated applications such as Gmail, Slack, Salesforce, and Jira. Agents must complete task objectives while following business rules. AutomationBench-AA uses Zapier’s private set of 657 tasks, and is built on v1.0.6
Key results:
➤ Claude Fable 5.1 and GPT-6 Astra lead the Intelligence Index: Both Claude Fable 5.1 (max with fallback) and GPT-6 Astra (max) score 53 on Intelligence Index v4.3, followed by Claude Opus 5 (max, 51), Claude Fable 5 (with fallback, 50), Muse Spark 1.3 (max, 48) and GPT-5.6 Sol (max, 47)
➤ GLM-5.3 and Kimi K3 continue to lead open weights models (both at 44): GLM-5.3-Flash (42) is the third strongest open weights model, followed by Qwen3.8 2.4T A95B (40) and DeepSeek V4 Pro 0813 (max, 36)
➤ 4 labs occupy the Intelligence vs. Cost per Task Pareto frontier: OpenAI occupies the majority of the cost-efficiency frontier, with all five reasoning efforts of the recently released GPT-6 Astra offering the lowest Cost per Task at their respective levels of intelligence. Claude Fable 5.1 (xhigh, max, 53), GLM-5.3-Flash (42) and MiMo-V2.5-Pro (26) round out the rest of the frontier
Shikai Li retweeted
1/ we just publicly released Muse Spark 1.3 max!
we see significantly stronger coding and agentic performance on muse spark 1.3 max, so would strongly recommend trying it out even if you've already tried muse spark 1.3 high or muse spark 1.3 xhigh.
Shikai Li retweeted
Muse Spark 1.3 is rolling out today with frontier performance almost too cheap to meter. This is the biggest jump we've made so far on coding and agentic work. Try it in Muse Code and our API.
Next up 🍉 and Muse Spark open weights releases coming soon.
Shikai Li retweeted
good iteration for muse spark 1.2
onto bigger and better things 🍉
Meta has released Muse Spark 1.2. It's their third release in four months and scores 54 on the Artificial Analysis Intelligence Index, significantly improving agentic knowledge work capabilities over prior releases and putting Meta next to SpaceXAI in a tie for third place amongst US labs
Muse Spark 1.2 (xhigh) lands at 54, up 3 points from Muse Spark 1.1 (51) and 11 points from Muse Spark 1.0 (43, April). It enters effectively tied with GPT-5.5 (xhigh, 55) and Grok 4.5 (high, 54), narrowly behind current frontier models Claude Opus 5 (max, 61), Claude Fable 5 (max w/ fallback, 60), GPT-5.6 Sol (max, 59), and Kimi K3 (max, 57)
Congratulations to @AIatMeta, @finkd, and @alexandr_wang on the release!
Key Takeaways:
➤ Muse Spark 1.2 gets closer to the frontier on agentic knowledge work. At Muse Spark 1.1's launch, we noted agentic knowledge work as its clearest gap; Muse Spark 1.2's gains help to close this. Its GDPval-AA v2 Elo rose 260 points to 1631, #5 among all models we have benchmarked and ahead of Claude Opus 4.8 (max, 1588). Terminal-Bench 2.1 gained 2 points (78% to 80%), and Tau3-Bench Banking rose 2 points (25% to 27%)
➤ Among the most cost-efficient models at its intelligence level. Muse Spark 1.2 costs $0.40 per Intelligence Index task at Meta's unchanged $1.25/$4.25 per 1M token pricing, with only Grok 4.5 (high, $0.37) and GPT-5.6 Sol (medium, $0.39) cheaper in its intelligence cluster - GPT-5.6 Terra (max, $0.51), Kimi K3 (max, $0.86), and GPT-5.5 (xhigh, $1.18) all cost more per task. The cost increase over Muse Spark 1.1 ($0.29 per task) is driven by increased token usage per Intelligence Index task
➤ AA-Omniscience abstention rate increases. The score rose from 18 to 22 as the hallucination rate fell 10 points (38% to 28%) and the attempt rate dropped from 82% to 67%. This heavy abstention (not answering questions when unsure) now drives both the low hallucination rate and a lower accuracy (41% to 38%)
➤ Scientific Reasoning results remain largely unchanged. CritPt notably gained 3 points (15% to 18%), while SciCode fell 2 points (58% to 56%), and Humanity's Last Exam fell 1 point (45% to 44%)
Other model details:
➤ Context window: 1M tokens, unchanged from Muse Spark 1.1
➤ Pricing: unchanged from Muse Spark 1.1: $1.25/$4.25 per 1M input/output tokens, with cache hits discounted to $0.15 per 1M
➤ Availability: Meta's first-party API at launch
Shikai Li retweeted
muse code in beta is live. first coding agent from msl, built on muse spark 1.2.
install: curl -fsS dev.meta.ai/install.sh | bash
here's what you should know:
Shikai Li retweeted
1/ today we're releasing muse spark, the first model from MSL. nine months ago we rebuilt our ai stack from scratch. new infrastructure, new architecture, new data pipelines. muse spark is the result of that work, and now it powers meta ai. 🧵
Shikai Li retweeted
Mixture-of-Experts (MoE) is a popular #LLM architecture that reduces computation by activating fewer parameters per token. But it brings memory, communication, and control challenges.
💡We introduce MetaShuffling, enabling efficient Llama 4 model inference in production.🔗 Read our latest blog to learn more: hubs.la/Q03m7XDP0
#PyTorch #OpenSourceAI
Shikai Li retweeted
what happened yesterday in Atlanta was brutal, tragic & is certainly not an isolated incident by any means. AAPI hate has been rampantly perpetuated & it’s disgusting! I’m heartbroken for the Asian community & my heart is with the loved ones of those we lost. The hate must stop.
Shikai Li retweeted
A man followed a Chinese-American doctor “spewing a profanity-laced racist tirade as she walked to the subway. ‘Why are you Chinese people killing everyone?’”
There is no place for hatred in our country and these racist attacks must be condemned.
washingtonpost.com/business/…
Shikai Li retweeted
Watch this. It shows why we should all do the right thing and stay home to the fullest extent possible. All of us can help slow the spread of the virus, protecting the elderly, the vulnerable, and each other. washingtonpost.com/graphics/…
Shikai Li retweeted
“We are fully aligned and continue to work with the US Government to contain the spread of COVID-19, inform citizens, and protect the health of our communities. (1/6)




























