Independent AI evaluations lab. We measure performance on real world tasks.
Joined February 2025
- Tweets406
- Following117
- Followers2K
- Likes559
NEW: Muse Wonder Full from @AIatMeta is live on LLM Stats.
> #13 on the LLM Stats Index at 51.7
llm-stats.com/models/muse-wo…
ALT Muse Wonder Full ranks #13 on the LLM Stats Index with a score of 51.7
LATEST: President Trump says the White House considers anyone who uses the term "Artificial Intelligence" instead of "Super Intelligence" as THE ENEMY.
ALT Screenshot of Donald J. Trump's Truth Social post: The White House considers anyone that uses the term "Artificial Intelligence," as opposed to the highly accepted new and more accurate term, "Super Intelligence," THE ENEMY!
JUST IN: OpenAI's AI found a slightly faster way to compute the Fourier transform than the classic FFT.
New bound: O(n (log n)^0.9999999999999), down from O(n log n).
ALT Title and abstract of OpenAI's paper 'An explicit power saving for the exact discrete Fourier transform' (Sept 25, 2026), with the bound O(n(log n)^(1-10^-13)) highlighted.
NEW: Mistral Large 4 (@MistralAI) is live on LLM Stats.
> #27 on the LLM Stats Coding Index at 35.8
> Early rating: 6 coding benchmarks so far
ALT LLM Stats Coding Index ranking: top 8 models led by Claude Opus 5.5 at 51.0, with Mistral Large 4 highlighted at #27 with a score of 35.8.
LLM Stats retweeted
Meet Mistral Large 4, aka Le Chonk.
• 1T parameters, natively multimodal. 49B active.
It is the best open weights model from US or Europe on aggregated benchmarks.
• State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding.
• Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure.
• Available to all via API today. Working with cybersecurity partners privately.
Open weights release end of October.
🤖 Made with AI
LLM Stats retweeted
best site to research the best models to use?
• llm-stats.com
@LlmStats
Qwen 3.8 27b. It's a beast.
For the first time, a model that you can run locally is challenging the frontier on the AI Leaderboard.
On the LLM Stats Index, it scores 46.9, which is right above Claude Opus 4.6 at 45.7.
Congratulations @Alibaba_Qwen. This is awesome.
LLM Stats retweeted
🚨Kimi K3 BEATS Opus 4.8!
K3 is in 3rd place on the ArtificialWatch Index🥉
Here's how the Index is computed:
💠 Best Variant Wins: Models are scored using only their top-performing version, ensuring a fair, identical comparison for everyone.
💠 Percentiles, Not Averages: Since different scoring systems can't be accurately averaged, each score is converted into a percentile rank from 0 to 100.
💠 Calculated as a Median: A model's final rank is the middle value (median) of its percentiles across different leaderboards, preventing any single board from skewing the results.
💠 No Estimations or Guesses: Every single data point is pulled directly from the source leaderboard, carries a timestamp, and links back to the original data.
When we take K3's scores across all major benchmarks:
- @arena = 1,678.5
- LiveBench = 78.5
- @LlmStats = 48.4
- @ValsAI = 74.7
- SimpleBench = 60.7
- @BridgeBench = 1,021.9
- @ArtificialAnlys = 57.1
🤯Final Artificial Watch Index Score = 87.5/100
Putting Kimi just ahead of Opus 4.8 and only behind GPT 5.6 and Fable 5.
Have you tried Kimi K3? What did you think about it?
Is the Artificial Watch Index an accurate representation of its capabilities?
Check it out yourself at: artificialwatch.com/benchmar…
LLM Stats retweeted
Everyone wants to know what the best AI model is.
So I combined all the top AI benchmarks in one place.
- LiveBench
- @arena
- SimpleBench
- @LlmStats
- @bridgebench
Creating what I call the ArtificialWatch Index.
GPT-5.6 is taking the lead, though I still personally prefer Fable 5.
Check it out on ArtificialWatch.com/benchmar…
LLM Stats retweeted
Great list! Here are some more fantastic AI eval labs:
@ValsAI (publishes one of the most comprehensive and high quality set of benchmarks)
@EpochAIResearch (publishes FrontierMath, the highest quality set of math evals)
@Designarena (publishes the highest quality set of design evals)
@scale_AI (publishes SWE-Bench Pro and a bunch of other fantastic benchmarks)
@LlmStats (publishes a very comprehensive set of benchmarks)
@cognition (publishes FrontierCode)
@LatchBio @SecureBio @FutureHouseSF (each publish some of the highest quality biotech benchmarks out there)
xitter.cf/lillian_ma_/status/207…
some AI evals labs + builders worth following:
@METR_Evals (@BethMayBarnes)
Independent evals for the uncomfortable question: can frontier models actually do dangerous autonomous work, or are we mostly measuring test-taking?
@arcprize (@fchollet, @mikeknoop)
The benchmark people cite when they want to separate real generalization from “the model has seen a lot of internet.”
@apolloresearch (@MariusHobbhahn)
Testing for scheming, strategic deception, and whether models behave differently once they realize they’re being evaluated.
@haizelabs (@leonardtang_, @josephsemrai)
Automated red-teaming for models and agents. Basically fuzzing, except the bugs can negotiate with you.
@arena (@istoica05, @ml_angelopoulos)
Human preference evals at internet scale. The place where every new model briefly becomes the world’s best conversationalist.
@ArtificialAnlys (@_micah_h, @grmcameron)
The scoreboard people actually use before deploying: quality, latency, price, and increasingly, agent capability.
@PatronusAI (@anandnk24)
Simulated users and adversarial evals for enterprise agents — where one confident hallucination can become a compliance ticket.
@giskard_ai (@alex_combessie, @jeanmarie_johnm)
Continuous agent testing for prompt injection, data leaks, hallucinations, and all the other ways a demo turns into an incident.
@confident_ai (@jeffr_yyy)
The DeepEval crew. Open-source-first tooling for treating LLM regressions more like normal CI failures.
@braintrust (@ankrgyl)
The strongest case for “evals are product specs”: turn real production traces into datasets, then stop shipping based on vibes.
not exhaustive - add the teams I’m missing in the comment
@Unisound_AI has launched U2, and we've independently benchmarked it on LLM Stats.
U2 is a 266B-parameter sparse model with just 10B active parameters (3.8% of total), priced at $0.15 / $0.30 per 1M input/output tokens ($0.003/M cached input). Unisound positions it around three claims: high intelligence density, high token efficiency, and a native agent architecture.
We evaluated these claims independently across coding, reasoning, long context, agents, math, and healthcare.
Key benchmarking takeaways:
➤ Coding: LiveCodeBench 0.85, #3 of 74, behind only DeepSeek V4-Pro-Max (0.94) and V4-Flash-Max (0.92), and ahead of DeepSeek-V3.2 Thinking (0.83)
➤ Long-context retrieval: MRCR v2 0.77, #1
➤ Science & knowledge: GPQA 0.87 (#34), MATH-500 0.86 (#30)
➤ Math: AIME 2025 0.73 (#81)
➤ Agentic SWE: SWE-Bench Verified 0.734
➤ All on ~10B active parameters at $0.15/$0.30 per 1M tokens
Coding is U2's standout. On LiveCodeBench (pass@1, release_v6) it scores 0.85, third of 74 models, trailing only two DeepSeek V4 variants. For a model activating just 10B parameters per forward pass, that's an exceptional result.
Reasoning and knowledge are strong on absolute scores: GPQA 0.87 and MATH-500 0.86. The middling ranks here (GPQA #34, MATH-500 #30, AIME 2025 #81) reflect a saturated frontier where the top models sit at ~0.93+ and competition-math scores cluster near 1.00, not weak absolute performance.
Where U2 still has clear room to grow:
➤ Extreme long context: despite #1 on MRCR v2 (0.77 aggregate), per-length splits show sharp degradation, from 77% at 4-8k tokens to 18% at 32–65k; LongBench v2's long split is ~3%
➤ Agentic terminal use: Terminal-Bench 2.0 0.44, #41 of 47, a notable gap vs U2's "native agent architecture" positioning
➤ Instruction following & grounding: Multi-Challenge 0.38 (#25); FACTS Grounding 0.44 (#12)
The efficiency story is the most interesting part: #3 coding, #1 MRCR v2, and 0.87 GPQA, all on 10B active parameters at $0.15/$0.30 per 1M tokens. Among sparse models at this price point, U2 is one of the strongest picks we've seen for code generation and core reasoning workloads.
TL;DR U2 is a credible flagship from @Unisound_AI, with top-tier coding and retrieval, strong science/math absolutes, and aggressive pricing on a heavily sparse 266B/10B architecture. The clearest opportunities to build on are agentic terminal performance and holding up at extreme context lengths.
Congratulations to Unisound on the release!
Detailed model breakdown: llm-stats.com/blog/research/…
NEW: Zhipu AI released GLM-5.2 (753B parameters) on June 16, 2026, and it lands #3 on the LLM Stats Index.
The standout feature: a 1M-token context window (5x jump from GLM-5.1's 200K) with 131K output tokens, plus two thinking-effort levels (High/Max recommended for complex coding).
Performance from verified benchmarks:
- 91.2% on GPQA
- 74% on FrontierSWE
- 82.7% on Terminal-Bench 2.1
- 62.1% on SWE-Bench Pro
- 46.2% on DeepSWE.
The 1M-token window changes how coding agents work in practice. You can hold an entire mid-sized repository in working memory (source files, tests, configuration, conversation history) without constant summarization.
Timeline context: GLM-5 (Feb 11) > GLM-5-Turbo (Mar 15) > GLM-5.1 (Apr 7) > GLM-5.2 (Jun 16). Four major releases in ~4 months. MIT license weights coming next week per their announcement.
Congrats to the Zhipu team on shipping this fast.
PROGRESSION: Claude Opus 4.8 (@AnthropicAI) sets a new high on LLM Stats Index after 3 years of progress.
> 68 on LLM Stats Index
> Frontier gained 85 index points over the 3.3-year window
This shifts the predictions towards a more optimistic scenario of complete benchmarking saturation.
NEW #1: Claude Opus 4.8 (@AnthropicAI) takes the top spot on LLM Stats Index.
> 68 on LLM Stats Index
> +5 over previous SOTA (GPT-5.5)
NEW: Gemini 3.5 Flash (@GoogleDeepMind) lands at #5 on LLM Stats Index.
>4x faster than other frontier models
>$1.50 / $9 per 1M tokens
It's significantly faster than other frontier models and the quality has increased significantly.
It's also the best tool calling model we've tested lately.
