Building SkyRL at @BerkeleySky Join the Slack community: https://xitter.cf/t.co/mSO97T61vR
Berkeley, California
Joined January 2025
- Tweets172
- Following18
- Followers2.9K
- Likes278
It was a packed house at the SkyRL Meetup today.
Thanks to all the speakers!
Join us for the first-ever SkyRL Meetup!
It's been a year since we released the SkyRL library. SkyRL now has 2k+ GitHub stars and a growing user base in academia and industry, including researchers from Stanford, CMU, UC Berkeley, Microsoft AI, Datadog, and more.
We're hosting a SkyRL Meetup on August 18th at the Anyscale office in SF. We have a special set of lightning talks:
🔗 Making frontier RL training accessible to everyone, from @trajectorylabs
🐶 Post-training an SRE agent with SkyRL, from @datadoghq
⚡️ Scaling RL with SkyRL on AMD GPUs, from @AMD
🧑🏻💻 Training knowledge work agents with SkyRL, from the SkyRL team (@charlie_ruan)
We’ll also do a deep dive into the library and talk about the road ahead. More details here: luma.com/y4ahouh6
To get started with `skyrl-train` as a library instead of forking, you can follow our instructions here: skyrl.readthedocs.io/en/late… to use the PyPI package!
(6/n)
Recipes: We've compiled a list of end to end recipes for reproducible runs on math (DAPO), Text2SQL, and search tasks!
Recipes page: skyrl.readthedocs.io/en/late…
(3/n)
AsyncRL: We support fully async training (aka "in-flight weight updates" or "multi-turn partial rollout") for any Generator that subclasses our GeneratorInterface for creating trajectories!
Docs: skyrl.readthedocs.io/en/late…
(2/n)
☁️SkyRL now runs seamlessly with SkyPilot!
Let @skypilot_org handle GPU provisioning and cluster setup, so you can focus on RL training with SkyRL.
🎯 Launch distributed RL jobs effortlessly
⚙️ Auto-provision GPUs across clouds
🤖 Train your LLM agents at scale
Get started with the SkyRL × SkyPilot guide:
🔗skyrl.readthedocs.io/en/late…
🔗docs.skypilot.co/en/latest/e…
SkyRL v0.2.0 is here!
With 22 contributors (including 11 new contributors!), this release holds many updates like strong MoE support with Megatron, LoRA support, standardized inference on the OpenAI API, new integrations, and many many more.
The code: github.com/NovaSky-AI/SkyRL
SkyRL just crossed 1000 Github stars! Thank you to all the wonderful contributors and users building this project together 🥳
Check it out: github.com/NovaSky-AI/SkyRL
Scaling agentic simulations is hard, so in collaboration with @anyscalecompute we wrote up our experience using Ray for agent sims—featuring an end-to-end RL example with SkyRL!
Check it out here: anyscale.com/blog/massively-…
3/N
We train three models with different maximum tool-call turns (2, 3, and 4), but a fixed total token generation budget. We observe that the model learns to issue more search queries for more complex questions, showcasing the benefits of multi-turn search.
2/N
In the guide, we provide a step-by-step walkthrough of implementing a multi-turn search environment in the simple SkyRL-Gym interface, with no modifications needed in the training stack.
🔎 SkyRL + Search-R1
Training a multi-turn search agent doesn’t have to be complicated.
With SkyRL, reproducing the SearchR1 recipe at high training throughput is quick and easy! We wrote up a detailed guide to show you how:
novasky-ai.notion.site/skyrl…
1/N 🧵
(4/N) As evidence for SkyRL’s extensibility, we provide several examples modifying the stack, like:
Implement a new environment in <50 LoC.
Update the sync training loop into async one-off pipelining in <100 new LoC.
Disaggregate training and generation on heterogeneous HW with only a config change!
