@slime_framework
Account based inJapan!
About this account
- Account based in
- Japan
- Connected via
- Web
X says this location may be affected by a proxy or VPN.
Account-level information from X, not a live location or the device used for a specific post.
The LLM post-training framework for RL Scaling. https://nitter.cf/t.co/4ILpx8hfKN
Joined September 2025
- Tweets115
- Following11
- Followers2.3K
- Likes61
Pinned Tweet
Ahead of the upcoming open-source release of GLM-5.3, we’re releasing slime v0.3.2 🚀
Highlights:
- Fully aligned GLM-5 training in Megatron with SGLang rollout
- Tons of community-contributed bug fixes and optimizations
- Continued refactoring and codebase cleanup, including the new slime/observability module
- CUDA 13 support
Getting slime ready for GLM-5.3 👀
github.com/THUDM/slime/relea…
slime retweeted
We wanted Score Centering to be a plug-and-play mismatch fix for off-policy RL setups. That's why it is very rewarding to see our method in slime in less than a week from release
Try it out for stabilizing your next run!
We've just added score centering to slime!
You can enable sc with --use-score-centering
PR: github.com/THUDM/slime/pull/…
We've just added score centering to slime!
You can enable sc with --use-score-centering
PR: github.com/THUDM/slime/pull/…
RL with LLMs is very unstable when training and sampling policies differ. Standard fixes (matching numerics, importance sampling) work around the problem.
We find the root cause of this instability from first principles and propose a way to directly cancel it. Score Centering is competitive and compatible with existing approaches — while simple to implement!
🧵 [1/6]
Huge congrats to the @radixark team on Miles v0.1! 🚀
Fully async agentic RL on a 744B MoE across 64 GB300s, with @sgl_project powering rollouts. Love the focus on getting rollout/training fidelity right alongside throughput.
huggingface.co/papers/2609.0…
The real world is multimodal. For AI to understand and recreate it, models need to learn across modalities.
In our latest blog, we show how Miles supports that learning with a shared post-training design for VLMs and diffusion models.
Link in the comments. 🔗
GLM-5.3 is now open-weight.
Our most capable model for agentic coding and cyber defense is now available to download, run, and customize.
Weights: huggingface.co/zai-org/GLM-5…
Tech blog: z.ai/blog/glm-5.3
slime retweeted
GLM-5.3-Flash scores 57 on the Artificial Analysis Intelligence Index. At $0.09 Cost per Task, it sits comfortably on the Intelligence vs. Cost per Task Pareto frontier
@Zai_org has released GLM-5.3-Flash, a smaller and cheaper sibling to GLM-5.3 at 320B total parameters and just 18B active parameters. GLM-5.3-Flash supports low/high/max reasoning efforts, and scores 57 evaluation on the Artificial Analysis Intelligence Index with max reasoning effort. This places the model only 3 points behind GLM-5.3 at 60 and in line with GPT-5.6 Terra and Muse Spark 1.2.
On Z AI's first-party API, GLM-5.3-Flash is priced at $0.15 / 1M input tokens and $0.50 / 1M output tokens, just over 10% of the price of GLM-5.3. Cached input tokens are priced at $0.026 / 1M tokens, an 80% discount. Its Cost per Task on the Intelligence Index is $0.09, compared to $0.68 for GLM-5.3 (max), and it sits on the Pareto frontier for Intelligence vs. Cost per Task.
Key results:
➤ GLM-5.3-Flash is 3 points behind GLM-5.3 (max) on the Artificial Analysis Intelligence Index, at ~7.5x lower Cost per Task. At $0.09 per Intelligence Index task against $0.68 for GLM-5.3, it sits on the Pareto frontier for Intelligence vs. Cost per Task. It ties GPT-5.6 Terra ($0.51) and Muse Spark 1.2 ($0.40) at 57 while costing ~5.7x and ~4.4x less per task.
➤ GLM-5.3-Flash is less token efficient, but its low per-token pricing means this does not translate into a high Cost per Task. The model used 149M output tokens to run the Intelligence Index, ~11% fewer than GLM-5.3 at 168M, but more than Kimi K3 (133M) and Qwen3.8 2.4T A95B (136M) which score the same on the Intelligence Index. Reasoning tokens account for 134M of the 149M total (~90%).
➤ GLM-5.3-Flash matches GLM-5.3 on real-world agentic work on GDPval-AA v2. With an Elo of 1770, the model is tied within the margin of error for GLM-5.3 and Grok 4.6. This places it behind only Claude Opus 5 (xhigh and max). On Terminal-Bench v2.1 it also matches GLM-5.3 (84.3% vs 83.9%), and on τ³-Banking it trails by 3.1 p.p. at 47.2%.
➤ GLM-5.3-Flash demonstrates good real-world knowledge and hallucination rate, scoring +7 on AA-Omniscience. Its AA-Omniscience Accuracy is 28%, 6 p.p. below GLM-5.3 (max) at 34% and well below GPT-5.6 Terra at 47%. However, with a Hallucination Rate of 28%, it is an improvement over GLM-5.3 at 30%. In real-world knowledge, GLM-5.3-Flash knows less than the bigger models and frontier proprietary models in its Intelligence Index tier with an accuracy of 28%.
Additional model details:
➤ Pricing: On Z AI's first-party API, $0.15 / 1M input tokens and $0.50 / 1M output tokens . Cached input tokens are priced at $0.03/ 1M tokens, an 80% discount.
➤ Accessibility: Accessible through Z AI's first-party API at launch.
➤ Size: 320B total parameters with 18B active parameters
➤ License: MIT
➤ Context Window: 400k
Unbelievable effort from the team!
Glad to see slime finally helps ship a multimodal model. We’ll gradually open-source the training implementation as the corresponding support in upstream frameworks stabilizes.
Introducing GLM-5.3-Flash
- Leading capabilities at a highly competitive price
- Natively multimodal with a 1M-token context window
- A 320B-A18B model released under the MIT License
- Previously previewed as Ox Alpha, running entirely on Chinese AI chips
Blog: z.ai/blog/glm-5.3-flash
Available now across all official platforms:
Weights: huggingface.co/zai-org/GLM-5…
API: docs.z.ai/guides/llm/glm-5.3…
Coding Plan: z.ai/subscribe
ZCode: zcode.z.ai/en
Chat: chat.z.ai
AutoClaw: autoclaw.z.ai
slime retweeted
It's my last day as an intern on @modal's training team.
With the rise of RL, it's becoming increasingly critical to have robust, reliable observability. I wanted to share some of my work building an observability system for the Modal Training Gym (gym.modal.dev), built on top of @slime_framework & @radixark miles.
Thank you @qjoyliu, @peywalt, training team, and Modal for an awesome summer!
slime retweeted
Continuous self-improvement needs an ever-expanding supply of training environments (goals).
SPADE: one model self-plays the Environment Designer and the Reasoning Agent, writing executable, agentic environments that get harder as it improves. Environment scaling on its own. ♠️
Artificial Analysis Index = 60
GLM-5.3 API is now live.
- Built for coding, defensive cybersecurity, and long-horizon agentic tasks
- Priced the same as GLM-5.2
- Available via the official API and partner model gateways
Get started: docs.z.ai/guides/llm/glm-5.3
Congrats 🎉🎉🎉 Amazing work by the great @radixark team!
Today we're launching Miles v0.1, an open-source RL framework for LLMs and multimodal models.
RL training is easy to start and hard to debug. Miles helps you ensure your run is correct, use hardware efficiently, and keep RL running at scale.
Over the past 9 months, 72 contributors have landed 1,326 commits, 85 GPU E2E CI tests, battle-testing Miles on frontier open models like Kimi K3, DeepSeek V4, Qwen 3.8, GLM 5.2, Inkling, MiniMax H3, etc.
Miles powers frontier-model development and production RL workloads at @humansand, @periodiclabs, @modal, @DecagonAI, @Eigent_AI, @nebiusai, @IBM and more, on both @NVIDIAAI and @AIatAMD hardware.
Here is what we built, and why teams picked Miles🧵
Proudly presented by the team ❤️
Introducing GLM-5.3: Built to Code. Ready for Cyber Defense.
- Top-tier coding and agentic capabilities, achieved through post-training on the 743B base model
- A major leap in cybersecurity, setting a new standard among open models
Tech Blog: z.ai/blog/glm-5.3
slime ❤️ SGLang
Our ecosystem project and the RL framework behind GLM series training, Slime, just open-sourced its deterministic train–rollout alignment path for GLM-5.2.
Megatron training and SGLang rollout matched down to a 4096-token logprob MAE of 1.9e-7, with exact zero hidden-state diff across decoder layers 0–5.
If you're dealing with train–inference mismatches in your RL runs, start here!
Ever since RL became a core part of LLM training, train–rollout numerical mismatch has been one of the recurring problems in RL infra.
There have been several great open-source efforts toward alignment, but production-scale support still often comes with tradeoffs in model scale, performance, or feature coverage.
Today, we're open-sourcing the deterministic train–rollout alignment stack we use for GLM-5.2-scale training — covering FP8 weights + FP8 KV rollout, DeepEP, DeepGEMM, and sparse attention.
github.com/THUDM/slime/pull/…
1/3
In one of our in-house runs with deliberately extreme train–rollout mismatch, the aligned implementation reduced the difference to <5e-7, with ~25% time overhead over the default path.
Importantly, the remaining diff comes from the sampler, rather than the model forward itself. This residual is therefore model-size independent — it does not grow as the model scales.
2/3
We also added layer-by-layer exact-matching tests to make sure the agreement is not just happening at the final logprobs.
In our validation, decoder layers 0–5 achieve exactly zero hidden-state diff over 4,593 matched tokens per layer.
We hope this provides a production-scale reference for studying train–rollout mismatch — both as an infra problem and as a way to better understand its impact on RL training dynamics.
Would love to see what the community finds with it.
3/3
slime v0.3.1 is out! 🚀
Unlike previous releases, this one is not about major new features. As AI agents improve our engineering productivity, we can revisit and remove many compromises across the stack.
Highlights:
• Lower Megatron offload overhead for colocated training
• Faster and more flexible weight updates
• Top-p masking and better rollout–training alignment
• PD + R3 routed-expert support
• Lower-memory PPO logprob/entropy computation
• PYTORCH_ALLOC_CONF support and --release-train
We also removed Megatron Bridge support to simplify slime. For broader model support, check out @radixark’s Miles.
github.com/THUDM/slime/relea…
SpecForge v0.3.0 is here! It's a major update to how draft models get trained. The new runtime separates target-model inference from draft-model training; unifies online, offline, and disaggregated workflows behind one entry point, and supports a broader family of speculative decoding algorithms.
SpecBundle also grows to 11 open draft models, spanning GLM-5.1, Kimi K2.5/K2.6/K2.7-Code, Qwen3, Step-3.5-Flash, Inkling-Small, and Kimi K3.
Full release note 👇
