Today we're launching Miles v0.1, an open-source RL framework for LLMs and multimodal models. RL training is easy to start and hard to debug. Miles helps you ensure your run is correct, use hardware efficiently, and keep RL running at scale. Over the past 9 months, 72 contributors have landed 1,326 commits, 85 GPU E2E CI tests, battle-testing Miles on frontier open models like Kimi K3, DeepSeek V4, Qwen 3.8, GLM 5.2, Inkling, MiniMax H3, etc. Miles powers frontier-model development and production RL workloads at @humansand, @periodiclabs, @modal, @DecagonAI, @Eigent_AI, @nebiusai, @IBM and more, on both @NVIDIAAI and @AIatAMD hardware. Here is what we built, and why teams picked Miles🧵

🚀 New blog: Advanced CUDA Graph Techniques in Inference and the design story behind them. CUDA Graph removes kernel-launch overhead by recording GPU work once and replaying it. But bringing it into a real inference engine introduces challenges in ops compatibility, graph management, and memory overhead. Together with the @AIatMeta team, we refactored SGLang's CUDA Graph support around a common runner/backend interface to tackle these problems systematically. On top of this architecture, we highlight several techniques first developed in SGLang: 1️⃣Breakable CUDA Graph (BCG), our new default prefill CUDA Graph backend, removes the dependency on torch.compile for faster setup and broader compatibility. 2️⃣Experimental Full CUDA Graph for prefill, first introduced in SGLang for FA4 and FlashInfer, using request padding to enable full graph capture for dynamic prefill workloads. 3️⃣Graph memory reuse across shapes and segments, keeping the CUDA Graph memory footprint fixed and making memory management more predictable as graph coverage expands.

The king of small models is back! Qwen3.8-27B from @Alibaba_Qwen is open source, and Day-0 support is live in SGLang: - 206.1 tok/s decode on a single RTX 5090, with our NVFP4 plus DSpark - 38.28 tok/s decode on DGX Spark Qwen3.8-27B raises the bar again for what a small model can do on agentic planning and long-horizon tasks. Long live the (small model) king! Run it locally with SGLang 👇

🚀 New blog: Unified Radix Cache: One Tree for Hybrid Model Prefix Caching Hybrid models complicate prefix caching: each attention type has its own cache reuse semantics, and specialized cache classes multiply combinatorially, duplicating tree logic as caching features grow. Unified Radix Cache brings everything into one shared tree: 1️⃣ One token-keyed radix topology for canonical prefix identity, with composable components for architecture-specific reuse semantics 2️⃣ HiCache and session-aware eviction are natively supported in the shared tree, built for agentic workloads 3️⃣ A clean separation between the controller and tree core makes the core replaceable, including an experimental Rust tree core implementation One tree, composable cache semantics, and a unified foundation for hybrid model prefix caching.

DSpark for Kimi K3 just got an upgrade: we further improved the model for long context and agentic use cases. - Accept length 4.2 in RULER v2 1M context length - Accept length 4.66 on SWE-rebench - Improvements across most of the benchmarks we tested (one small regression on GSM8K) And a milestone worth celebrating: DSpark crossed 140k downloads in the first few days after launch. Seeing a model we trained get picked up this quickly means a lot to us 🌟 Thank you to everyone who tried it, filed issues, and pushed it into real workloads.

🚀 SGLang now officially supports @thinkymachines Inkling-Small running on 2× @NVIDIAAI DGX Spark systems connected via ConnectX-7. Huge thanks to @realJerryzhou @ispobaoke for implementing the support, with valuable community contributions from @u1tra_instinct and @TechMDAI. This enables strong agentic capabilities to run locally on a compact platform. The current setup achieves 24 tok/s (concurrency=1) without MTP. DSpark support is coming soon. Check out the cookbook below to get started 👇

Inkling-small is out today! With SGLang, you can get 648 tok/s decode with DSpark (simulated acc len=4) and 288 tok/s w/o DSpark, under the same setup (8x @NVIDIAAI B200, TP 8, NVFP4, bs=1). What makes this model different is the size. 276B total with 12B active is a sweet spot for RL, and both LoRA and full-parameter training become well within reach. Miles is ready and verified for multimodal RL on Inkling-small, so you can turn your multimodal data into real capability gains. At ~1/4 the size, Inkling-small matches the bigger version in capability and even wins on some benchmarks. Run Inkling-small with SGLang, and customize it with Miles.

Inkling, @thinkymachines' first open model, dropped today: 975B total / 41B active MoE, up to 1M context, reasoning natively over text, images, and audio. Serving and RL support are already live: you can run and shape it on an open stack, starting now. Day 0 support on SGLang @sgl_project and Miles @radixark👇 - Inkling's new architecture (ShortConv, attention with relative positional embedding, shared expert sink MoE) is natively implemented and deeply optimized, with prefill full CUDA graph and MXFP8 KV cache - Full parameter and LoRA RL in a customized Megatron backend, train inference consistency via customized kernels, routing replay, and cross-runtime parameter synchronization - DFlash speculative decoding from @modal for low-latency serving Launching now, blog and cookbook in the comments ⬇️

🎉 SGLang v0.5.13 is out! First, new model support! Nemotron 3 Ultra, Step-3.7-Flash, Command A+, plus new diffusion models: Cosmos3, FLUX.2-Klein, Ideogram 4, LingBot-World, SANA-WM, and Ernie-Image. Here are the highlights for this release: - Speculative Decoding V2 is now the default! Tree drafting (topk>1) for faster generation - Breakable CUDA Graphs now make prefill faster - Qwen 3.5 runs faster on NVIDIA Blackwell with new GDN kernels - HiCache with UnifiedTree on by default for hybrid SWA/Mamba models - SGLang-Diffusion now supports realtime generation! Plus progressive resolution - Multiple performance and feature updates for DeepSeek V4 Thanks to our amazing partners and model makers: @NVIDIAAI @AMD @intel @awscloud @boson_ai @cohere @bfl_ml @ideogram_ai @deepseek_ai @Kimi_Moonshot @Alibaba_Qwen @StepFun_ai @Baidu_Inc @robbyant_brain