Open RL library for LLMs. https://xitter.cf/t.co/Xpaq0thhgi Join us on https://xitter.cf/t.co/uWI5Zbd6IH

Joined January 2025
Mind Lab, together with our community partners NVIDIA @Nvidia @NVIDIAAIDev, VeRL @verl_project, and vLLM @vllm_project, is making LoRA RL practical at trillion-parameter scale. We mapped the failure modes of LoRA RL on trillion-parameter MoE reasoning models and built a hybrid-parallel engine—VeRL for RL orchestration + NVIDIA NeMo’s Megatron-Bridge for MoE parallelism + vLLM for efficient rollouts/inference—that enables stable, cost-efficient RL at ~10% GPU usage. With comparable RL compute, trillion-parameter LoRA RL beats full-parameter RL on smaller models—underscoring that strong priors are crucial for reasoning-level RL. Read the full post: macaron.im/mindlab/research/…
2
12
1
29
3,790
And because we want this capability to scale beyond any one lab, we’re contributing the system back to the ecosystem through major open-source collaborations with @nvidia Megatron-Bridge and Volcengine’s verl. Why RL on trillion-parameter models? Our experiments show a consistent pattern: RL is prior-limited. Under matched RL FLOPs, “large prior + small LoRA” outperforms full-parameter RL on small models (1.5B) on AIME and GPQA. A strong prior generates higher-quality trajectories; RL amplifies signal, not noise. This is why trillion-scale LoRA RL is not indulgence; it is efficiency.
1
2
5
675
verl highlighted as one of the top 10 open source AI infra projects. Thank you @github for supporting the oss projects and communities #octoverse2025
GitHub #Octoverse2025: A New Era for Developers & AI! : must-read for anyone in tech, recruitment, or product strategy GitHub’s latest Octoverse report is a goldmine of insights into how software development is evolving—and it’s happening fast! github.blog/news-insights/oc…
9
934
Reinforcement learning is becoming essential for training coding agents beyond what supervised fine-tuning can do. This template uses @verl_project from the @BytedanceTalk Seed team to train LLMs with RL, the flexible architecture that scales from single GPU to distributed setups. Wrap Python sandboxes as tools, run multi-turn ReAct loops and train LLMs end-to-end with PPO. Train your own coding agent using this tutorial + notebook to build your first RL-powered coding agent → go.lightning.ai/4qsCdZQ
2
3
25
2,470
verl project retweeted
𝗣𝗵𝘆𝘀𝗶𝗰𝘀 is one of the sharpest testbeds when reasoning models move from puzzle-solving to realistic scientific reasoning. We have trained 𝗣𝟭 – a serie of models that achieved 𝗴𝗼𝗹𝗱 𝗺𝗲𝗱𝗮𝗹-𝗹𝗲𝘃𝗲𝗹 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 in the Physics Olympiad.🥇
12
44
5
486
55,809
verl project retweeted
Open. Source. SOTA. Deep. Research. 🚀 Today, we’re releasing PokeeResearch-7B, a SOTA open-source deep research agent that outperforms all other 7B deep research agents. And, we are open-sourcing both the weights and inference code on @huggingface! We're additionally excited to have partnered with vLLM @vllm_project, SGLang @sgl_project, and verl @verl_project on training and inference pipelines. We can’t wait to see what you all build with PokeeResearch! 🤗• Hugging Face Model: huggingface.co/PokeeAI/pokee… 🌐• Webpage: pokee.ai/deepresearch-previe… 📋• ArXiv: arxiv.org/pdf/2510.15862 🔗• Github Repo: github.com/Pokee-AI/PokeeRes…
60
184
131
1,108
584,659
Get started agentic RL with verl, checkout tutorial @LightningAI lightning.ai/lightning-ai/en…
5
826
verl project retweeted
We pushed Reinforcement Learning to new speeds on @AMD's MI300X 🔥 By tuning 3D parallelism with Verl, we hit: ⚡ 4.3× faster rollout 🧠 72% shorter training 💾 192GB HBM3 = zero bottlenecks Smarter orchestration > more GPUs. Read the full study here: yottalabs.ai/post/performanc…
3
2
8
1,544
verl project retweeted
🔥 Announcing our new paper: "SR-Scientist: Scientific Equation Discovery With Agentic AI" Most current work using LLMs for scientific discovery, like AlphaEvolve, follows a rigid "generate → evaluate → refine" loop. We challenge this paradigm for equation discovery. Our work, SR-Scientist, empowers an LLM to act as an autonomous agent, discovering scientific equations through long-horizon, tool-driven data analysis and equation evaluation—much like a human scientist. We further enhance its capabilities with multi-turn RL. 📈 Key Results: 1️⃣Consistently outperforms SOTA methods by a 6% to 35% absolute margin. 2️⃣Achieves significant performance gains after RL training. 3️⃣Demonstrates robustness to noise and generalization to out-of-domain data. 💡 Key Insights: 1️⃣ Long-horizon exploration is vital for performance. 2️⃣ Enabling agents to conduct their own data analysis is crucial. 3️⃣ An experience buffer is key for continuous optimization. 📄 Paper: arxiv.org/abs/2510.11661 💻 Code: github.com/GAIR-NLP/SR-Scien…
6
27
4
97
17,703
Agent training using RL, on any compute infrastructure!
How to train an AI agent -- not just prompt one? 🤖 We just dropped a deep dive on building agents: 🌋Train with RL using @verl_project 🖥️Monitor runs with @wandb 🚀Run & scale training on any AI compute (k8s or clouds) with SkyPilot. blog.skypilot.co/verl-rl-tra…
1
8
1,323
🤔Can we train RL on LLMs with extremely stale data? 🚀Our latest study says YES! Stale data can be as informative as on-policy data, unlocking more scalable, efficient asynchronous RL for LLMs. We introduce M2PO, an off-policy RL algorithm that keeps training stable and performant even when using data stale by 256 model updates. 🔗 Notion Blog: m2po.notion.site/rl-stale-m2… 📄 Paper: arxiv.org/abs/2510.01161 💻 GitHub: github.com/Infini-AI-Lab/M2P… 🧵 1/4
3
39
7
232
64,011
verl project retweeted
With the help from Bytedance @verl_project team, we have integrated collabllm as a recipe to veRL, a to-be-most-popular open RL library for LLMs. Now you are only one step away from making your LLM a great collaborator in multiturn conversations. verl.readthedocs.io/en/lates…
CollabLLM won #ICML2025 ✨Outstanding Paper Award along with 6 other works! icml.cc/virtual/2025/awards_… 🫂 Absolutey honored and grateful for coauthors @MSFTResearch @StanfordAILab and friends who made this happen! 🗣️ Welcome people to our presentations about CollabLLM tomorrow (Tuesday): - Oral 1A (icml.cc/virtual/2025/session…) - Poster Session 1 East (icml.cc/virtual/2025/session…) - Multiagent Social (icml.cc/virtual/2025/49694) Please check out: Website: aka.ms/CollabLLM Github: github.com/Wuyxin/collabllm Paper: arxiv.org/pdf/2502.00640 Blog: wuyxin.github.io/collabllm/#…
12
1
126
17,805
verl deployed in real world RL systems 🚀🚀🚀
At Amazon's AGI Lab, XJ Wang's Reinforcement Learning Pod team trains AI agents using RL gyms for real-world applications. His team handles everything from model training to product integration across Amazon. Learn more:
1
11
1,535
Nice OpenAI gym-like environment interface, compatible with verl
In the era of experience, we're training LLM agents with RL — but something's missing... We miss the good old Gym! So we built 💎GEM: a suite of environments for training LLM 𝚐𝚎𝚗𝚎𝚛𝚊𝚕𝚒𝚜𝚝𝚜. Let’s build the Gym for LLMs, together: axon-rl.notion.site/gem
6
15
1,851
verl project retweeted
VeRL now officially supports launching via SkyPilot! Let SkyPilot deal with infra heavylifting for @verl_project: 🚀Spin up VeRL workers on your k8s or clouds 🔧Set up ray 🤖Ignite your agentic RL training Check out the VeRL doc: verl.readthedocs.io/en/lates…
1
5
4
15
3,231
Proud to introduce Group Sequence Policy Optimization (GSPO), our stable, efficient, and performant RL algorithm that powers the large-scale RL training of the latest Qwen3 models (Instruct, Coder, Thinking) 🚀 📄 huggingface.co/papers/2507.1…
32
242
46
1,663
326,078
GSPO has been integrated into @verl_project (github.com/volcengine/verl/p…) and TRL (github.com/huggingface/trl/r…). Thanks for the prompt support from the community 🚀
Proud to introduce Group Sequence Policy Optimization (GSPO), our stable, efficient, and performant RL algorithm that powers the large-scale RL training of the latest Qwen3 models (Instruct, Coder, Thinking) 🚀 📄 huggingface.co/papers/2507.1…
3
23
1
196
14,476
verl project retweeted
To push the open source frontier for RL + LLMs, we need scalable, modular environments with real-world complexity, beyond math benchmarks. Today, we’re releasing *benchmax*. An open-source framework to build, run, & scale useful RL envs for LLM fine-tuning, with integrations to verl & verifiers (more coming soon!).
4
24
2
85
16,372