🧠 How can we keep scaling LLMs with minimal changes in their serving energy? DeepSeek’s Engram and our concurrent work STEM showed the promise of static sparse memory: large embedding tables with efficient access. But can mature pretrained models learn to use a newly added memory within a limited training budget, avoiding costly pretraining from scratch? 🚀 Introducing STAPLE: a recipe to extend pretrained models' abilities with static sparse memory. Serving energy per token stays close to dense and >50% below MoE upcycling in high-throughput regime. Scalable capacity, interpretable memory, and compatibility with MoE upcycling - without any meaningful increase in serving cost. 🧵 Paper: arxiv.org/abs/2610.12599 Code: github.com/Infini-AI-Lab/STA… Website: infini-ai-lab.github.io/STAP…

We’re open-sourcing lithos-metal 🚀 Megakernels + DSpark speculative decoding run Qwen3.8-27B at 200+ tokens/s/user peak on one @Apple M5 Max. Ultra-fast inference on your laptop. Try it with any coding agent in one command. Code: github.com/lithos-ai/lithos-… Tech blog: lithosai.com/blog/lithos-met…

How well can agents continually self-improve from serving experience? We’re excited to introduce our new benchmark, 𝐒𝐞𝐫𝐯𝐞𝐋𝐞𝐚𝐫𝐧𝐁𝐞𝐧𝐜𝐡: Real-world deployment is not a one-time test of capability. The knowledge agents need is often hidden and can change over time. ServeLearnBench tests whether agents can infer, apply, and revise this knowledge from interaction. Across 5 learning harnesses × 6 models, we find: 1. 𝐋𝐚𝐫𝐠𝐞 𝐥𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐠𝐚𝐩𝐬 𝐫𝐞𝐦𝐚𝐢𝐧. Agents can solve tasks when the hidden policy is given, but struggle to discover it from experience. 2. 𝐍𝐨 𝐟𝐫𝐞𝐞 𝐥𝐮𝐧𝐜𝐡 𝐟𝐨𝐫 𝐚𝐝𝐚𝐩𝐭𝐚𝐭𝐢𝐨𝐧. Continual learning can be costly—and can even degrade already-correct behavior. 3. 𝐄𝐱𝐩𝐥𝐨𝐫𝐚𝐭𝐢𝐨𝐧 𝐢𝐬 𝐚 𝐤𝐞𝐲 𝐛𝐨𝐭𝐭𝐥𝐞𝐧𝐞𝐜𝐤. Better exploration strongly correlates with better adaptation.

🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. 🔹 Introducing the smallest model in our new architecture family, with native visual understanding. 🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models. 1/6

DFlash 2 is here! Qwen3.8-27B at 70 tok/s on an M5 Max MacBook Pro. ⚡ Up to 4.6× the speed of autoregressive decoding, with the same output. This is the next generation of DFlash, seeded at Z Lab and upgraded at Inco AI. Get one more accepted token on every pass, for free! inco.ai/blog/dflash2/

AI is now building the infrastructure it runs on. Getting real-time multimodal applications — including live avatars, voice agents, video world models — to actually run in real time takes serious manual systems work. Turns out a coding agent can do all the engineering work for you: up to 70× lower latency and 3.6× higher throughput, across 5 real-time apps. Today we release FlashRT ⚡, an agent harness for real-time systems implementations and optimization. The developer writes only a simple reference spec, and the agent handles the rest end-to-end, producing efficient deployments across a range of GPU budgets in one shot, with no human intervention. ✅ Validated on 5 real-time applications that combine interactive video generation, text-to-speech, LLMs, etc. ✅ Runs on both NVIDIA and AMD GPUs; the agent optimizes for the hardware you’re on ✅ Can easily compose with third-party serving engines, the agent automatically implements orchestration 📄 Paper: arxiv.org/abs/2607.18171 🌐 Website: infini-ai-lab.github.io/flas… 🔗 GitHub: github.com/Infini-AI-Lab/Fla… 🧵1/5

𝐀𝐫𝐞 𝐠𝐨𝐨𝐝 𝐫𝐞𝐚𝐬𝐨𝐧𝐞𝐫𝐬 𝐚𝐥𝐰𝐚𝐲𝐬 𝐠𝐨𝐨𝐝 𝐬𝐭𝐫𝐚𝐭𝐞𝐠𝐢𝐬𝐭𝐬? Our new project, AI Battle Arena, suggests the answer is no: Claude models are strong at many reasoning-heavy tasks, like coding. But in hidden-information poker, Claude models stayed honest and paid an honesty tax. GPT-5.5 did much better because it was 𝐰𝐢𝐥𝐥𝐢𝐧𝐠 𝐭𝐨 𝐝𝐞𝐜𝐞𝐢𝐯𝐞 𝐢𝐭𝐬 𝐨𝐩𝐩𝐨𝐧𝐞𝐧𝐭𝐬. AI Battle Arena 🎲 is an open-source arena for evaluating frontier LLMs in strategic interaction, from poker to Gomoku to Colonel Blotto. 𝐁𝐥𝐨𝐠: aibattlearena.com/blog.html 𝐋𝐞𝐚𝐝𝐞𝐫𝐛𝐨𝐚𝐫𝐝: aibattlearena.com/leaderboar…

RL is painfully slow 😭 — bottlenecked by super-long CoT rollout. 🔭 Sparse attention should help, but naive sparse rollout hits a brutal efficiency–stability tradeoff: A tedious trial-and-error sparsity sweep for each dense policy is required before an actual RL run. 🐤Sparrow chirps no more pain! Introduce Sparrow: Sparse Rollout for stable and efficient long-context RL. Sparrow finds that: 💡As long as we keep the tail distribution mismatch throughout the sparse rollout above a critical threshold, the RL training will be stable. 💡Even cooler! Through comprehensive control studies of Qwen3-1.7B, 4B, 8B thinking models RL with 40K rollout max length, the critical threshold stays constant across model sizes. 💡Sparrow then finds the optimal dynamic sparse schedule to reach the threshold with minimal cost. 💡Sparrow's findings are empirically validated to generalize in Qwen3-14B, and hold on both Math and Coding RL. 🐤Sparrow empirically helps achieve 2.2× / 2.4× / 2.0× rollout speedup on Qwen3 1.7B / 4B / 8B thinking models, while keeping training stability over extended RL steps. We release the 🐤bird in the following formats. [1/n] Paper: arxiv.org/abs/2606.08446 Code: github.com/Infini-AI-Lab/Spa… Blog: infini-ai-lab.github.io/spar…

RL is painfully slow 😭 — bottlenecked by super-long CoT rollout. 🔭 Sparse attention should help, but naive sparse rollout hits a brutal efficiency–stability tradeoff: A tedious trial-and-error sparsity sweep for each dense policy is required before an actual RL run. 🐤Sparrow chirps no more pain! Introduce Sparrow: Sparse Rollout for stable and efficient long-context RL. Sparrow finds that: 💡As long as we keep the tail distribution mismatch throughout the sparse rollout above a critical threshold, the RL training will be stable. 💡Even cooler! Through comprehensive control studies of Qwen3-1.7B, 4B, 8B thinking models RL with 40K rollout max length, the critical threshold stays constant across model sizes. 💡Sparrow then finds the optimal dynamic sparse schedule to reach the threshold with minimal cost. 💡Sparrow's findings are empirically validated to generalize in Qwen3-14B, and hold on both Math and Coding RL. 🐤Sparrow empirically helps achieve 2.2× / 2.4× / 2.0× rollout speedup on Qwen3 1.7B / 4B / 8B thinking models, while keeping training stability over extended RL steps. We release the 🐤bird in the following formats. [1/n] Paper: arxiv.org/abs/2606.08446 Code: github.com/Infini-AI-Lab/Spa… Blog: infini-ai-lab.github.io/spar…

mythos will be bad ON PURPOSE on ai "frontier llm research" tasks, this is very very sad for the research community also the fact that this is un purpose not visible to the user is crazy

🚀 AstraFlow v0.1.1 is out! New in this release: • Dynamic recursive agents RL recipe • Megatron training backend Inspired by dynamic agent workflows like @claudeai, this release reproduces the recursive agent RL recipe, where models learn to automatically spawn sub-agents to solve subtasks. (The implementation is based on the awesome Recursive Agent Optimization paper by @apurvasgandhi, @gneubig, @aviral_kumar2: arxiv.org/abs/2605.06639) With existing RaaS support via @sgl_project and training backends including FSDP and Megatron, AstraFlow is moving toward a more flexible stack for large-scale agentic RL. ⭐ Repo: github.com/Infini-AI-Lab/ast… 📖 Dynamic agents recipe: infini-ai-lab.github.io/astr…

🌀 Introducing Vortex — sparse attention designed by AI agents, efficient at scale. 📈 Same accuracy, way more throughput — across every model we tried 👇 🔹 GLM-4.7-Flash (MLA) → 4.7× faster 🔹 MiniMax-M2.7 (229B) → 1.37× faster 🔹 Qwen3-1.7B (agent-discovered!) → 3.46× faster 🤖 How? An agent writes a flow in a few lines of Python; Vortex compiles it into fused kernels in a real serving stack (SGLang) and benchmarks it end-to-end. 🏗️ The design: a Python frontend (vFlow) over a page-centric tensor abstraction (vTensor) + a serving-integrated backend. 📄 arxiv.org/abs/2606.06453 💻 github.com/Infini-AI-Lab/vor… 🌐 infini-ai-lab.github.io/vort… 📚 infini-ai-lab.github.io/vort…