Building SkyRL at @BerkeleySky Join the Slack community: https://xitter.cf/t.co/mSO97T61vR
Berkeley, California
Joined January 2025
- Tweets172
- Following18
- Followers2.9K
- Likes278
NovaSky retweeted
First day 0 model support on SkyRL. Great work from @sumanthrh !
NovaSky retweeted
🎉 Day-0 RL support for Beam in SkyRL @NovaSkyAI.
LoRA, BF16 full fine-tuning and MXFP8 RL, with vLLM rollouts and Reflection’s Megatron-Bridge implementation.
SkyRL reports +62% LoRA decode throughput from right-sizing expert adapters in vLLM: 276 → 447 tok/s/GPU at 32 concurrent requests, PP2×TP8. This is a decode result, not an end-to-end training speedup.
R3 works with pipeline parallelism, and FP32 router weights are preserved during weight sync. Thanks to Sumanth Hegde and the SkyRL team!
docs.skyrl.ai
NovaSky retweeted
Great to see this out!
Datadog's new SOTA model for investigating production incidents. Their in-house training stack builds on top of SkyRL! 🚀🚀🚀
Meet Scout 1.0, our first AI model built for investigating production incidents.
Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier.
We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence.
Learn more and sign up for the preview: bit.ly/3TJknGO
NovaSky retweeted
datadog ai research used SkyRL to post-train GLM 5.3 Flash to perform at the pareto frontier for their SRE eval
@bthecohen and team were some of the earliest users of SkyRL at scale, really cool to see these results finally out! great tech report as well
Meet Scout 1.0, our first AI model built for investigating production incidents.
Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier.
We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence.
Learn more and sign up for the preview: bit.ly/3TJknGO
NovaSky retweeted
Datadog post-trained GLM-5.3 Flash via RL on real-incident sims. Matches frontier SRE performance at much lower cost.
Their RL stack is built on top of SkyRL! Check out the technical report web-assets.dd-static.net/425…
Meet Scout 1.0, our first AI model built for investigating production incidents.
Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier.
We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence.
Learn more and sign up for the preview: bit.ly/3TJknGO
NovaSky retweeted
Post-trained with SkyRL!
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).
I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.
We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:
"/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started"
I imagine we'll have some backend services that could be replaced entirely by this new architecture.
NovaSky retweeted
Post-trained with SkyRL @NovaSkyAI
Served on Ray Serve LLM @raydistributed
all on Anyscale Platform @anyscalecompute
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).
I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.
We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:
"/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started"
I imagine we'll have some backend services that could be replaced entirely by this new architecture.
NovaSky retweeted
Read more about our experiment using SkyRL here: afterquery.com/blog/improvin…
To learn more about our tasks and reward design, reach out to us at [email protected]
NovaSky retweeted
Kimi K2.5-2.7 can be trained with RL on just 16 B300 GPUs (yes, just 2 nodes!) with full context
Most post-training frameworks out there will tell you that you need 16 full nodes, and might not even give you full context
Thanks to SkyRL maintainers for collab
Excited to announce the release of SkyRL v0.4.0!
Some highlights:
- Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7
- Top-P sampler replay and optimized R3 data transfer
- Full FP8/MXFP8 RL
- Sharded P2P weight transfer
Blog: blog.skyrl.ai/blog/skyrl-v0-…
Details in 🧵
NovaSky retweeted
Excited to announce the release of SkyRL v0.4.0!
Some highlights:
- Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7
- Top-P sampler replay and optimized R3 data transfer
- Full FP8/MXFP8 RL
- Sharded P2P weight transfer
Blog: blog.skyrl.ai/blog/skyrl-v0-…
Details in 🧵
NovaSky retweeted
How does one RL post-train a 397B model for long-horizon knowledge work? 👩💼
We share every step we took to bring Qwen 3.5 397B from 16.1% Pass@1 to 27.3% on APEX-Agents using DPPO, including final models weights and the full training script🚀 This is the first of many works from Mercor Research on open model training research.
Full blog: mercor.com/blog/training-fro…
Source code: github.com/Mercor-Intelligen…
NovaSky retweeted
Worked with Mercor Research and the SkyRL team, training Qwen3.5-397B-A17B on APEX-Agents (long-horizon office work) off-the-shelf data with SkyRL, improving Pass@1 from 16% to 27%.
The post is more of a practical field guide for what to do given an RL dataset, de-risking step by step: environment infra, harness fixes/optimizations, token-in-token-out, an overfitting run, small-model ablations, then the hero run.
We were also delighted to see the gains transfer to OpenCode despite training on an MCP-based harness, and even to Terminal-Bench 2.1.
Code, models, and eval traces are all open source. Hope it helps your next hillclimb!
NovaSky retweeted
In RL training, a vLLM rollout engine and a Megatron trainer can run the same policy yet disagree on a token's logprob due to floating-point non-associativity.
SkyRL's IsoExec combines an execution contract with a unified model, aligning rounding-sensitive execution choices across rollout and training.
Bitwise parity holds across different TP, EP, and SP layouts. For Gated DeltaNet, the chunkwise-parallel recurrent algorithm makes parallel training and prefill bitwise identical to recurrent decode.
Qwen3.5-35B-A3B, DAPO, 8xH100, 50 steps: logprob diff 1.6e-2 to 6.7e-7, full-step overhead 25.3% ✅ vLLM's scheduler and CUDA graphs still apply.
Thanks to @JiangAlexander1 and the SkyRL team at @NovaSkyAI.
🔗 vllm.ai/blog/2026-08-21-isoe…
NovaSky retweeted
The first-ever SkyRL Meetup is happening! A year of hard work from the team, and it's been great to see users pick it up from across industry and academia.
Join us Aug 18 Tuesday 6:00pm in SF for four lightning talks from enterprise users and collaborators (@trajectorylabs , @datadoghq , @AMD , @mercor ), followed by a SkyRL technical deep dive.
Would love to see you there!
NovaSky retweeted
Join us for the first-ever SkyRL Meetup!
It's been a year since we released the SkyRL library. SkyRL now has 2k+ GitHub stars and a growing user base in academia and industry, including researchers from Stanford, CMU, UC Berkeley, Microsoft AI, Datadog, and more.
We're hosting a SkyRL Meetup on August 18th at the Anyscale office in SF. We have a special set of lightning talks:
🔗 Making frontier RL training accessible to everyone, from @trajectorylabs
🐶 Post-training an SRE agent with SkyRL, from @datadoghq
⚡️ Scaling RL with SkyRL on AMD GPUs, from @AMD
🧑🏻💻 Training knowledge work agents with SkyRL, from the SkyRL team (@charlie_ruan)
We’ll also do a deep dive into the library and talk about the road ahead. More details here: luma.com/y4ahouh6
NovaSky retweeted
Excited to announce the release of SkyRL v0.3.0!
SkyRL’s focus in this release revolves around three major themes:
1. Support for large scale, async RL
2. Improving SkyRL’s Tinker API for self-hosting Tinker servers
3. Providing a unified library for SFT and RL
Some highlights 🧵
NovaSky retweeted
Today, we're excited to share that Biomni is published in @ScienceMagazine.
Biomedical research is still fragmented, manual, and difficult to scale. In this work, we introduce Biomni - the first general-purpose biomedical AI agent with an integrated biology environment that can reason, plan, and execute end-to-end scientific workflows.
We show that, with the right environment and harness, AI can automate large-scale omics analyses, orchestrate laboratory robotics, optimize molecular properties, and even train new AI models for biology.
We also introduce a reinforcement learning recipe for continually improving biomedical AI agents, enabling open-source models to achieve frontier-level performance.
It's surreal to look back. We started the Biomni project in early 2024, when agentic AI was still nascent. It is exciting to see tens of thousands of biologists collaborating with agents every day to accelerate science.
Try Biomni: biomni.phylo.bio
Read more: science.org/doi/10.1126/scie…
This work is not possible without this truly inter-disciplinary team: @serena2z @hcwww_ @YuanhaoQ Minta Lu, Ryan Li, @yusufroohani Lin Qiu @shiyi_c98 Gavin Junze Di @rickwierenga @kavi_deniz Sherry @TianweiShe Shruti Jennefer Xin Zhou @MWheelerMD Jon Bernstein @MengdiWang10 @PengHeAtlas @zhou_jingtian @SnyderShot @lecong Aviv Regev @jure
@StanfordAILab @genentech @phylo_bio @arcinstitute @UW @berkeley_ai @RetroBio_ @tamarindbio @Princeton @UCSF
NovaSky retweeted
Biomni is now out in Science @ScienceMagazine ! Huge congrats to the team 🎉
Biomni is a really exciting step toward AI agents that can carry out real biomedical research workflows. Proud that SkyRL supported the RL training for Biomni-R0 early last year, where we explored end-to-end multi-turn RL for biomedical reasoning agents.
It’s really exciting to see how AI agents have evolved the past one year toward carrying out real scientific workflows and accelerating discovery.
Today, we're excited to share that Biomni is published in @ScienceMagazine.
Biomedical research is still fragmented, manual, and difficult to scale. In this work, we introduce Biomni - the first general-purpose biomedical AI agent with an integrated biology environment that can reason, plan, and execute end-to-end scientific workflows.
We show that, with the right environment and harness, AI can automate large-scale omics analyses, orchestrate laboratory robotics, optimize molecular properties, and even train new AI models for biology.
We also introduce a reinforcement learning recipe for continually improving biomedical AI agents, enabling open-source models to achieve frontier-level performance.
It's surreal to look back. We started the Biomni project in early 2024, when agentic AI was still nascent. It is exciting to see tens of thousands of biologists collaborating with agents every day to accelerate science.
Try Biomni: biomni.phylo.bio
Read more: science.org/doi/10.1126/scie…
This work is not possible without this truly inter-disciplinary team: @serena2z @hcwww_ @YuanhaoQ Minta Lu, Ryan Li, @yusufroohani Lin Qiu @shiyi_c98 Gavin Junze Di @rickwierenga @kavi_deniz Sherry @TianweiShe Shruti Jennefer Xin Zhou @MWheelerMD Jon Bernstein @MengdiWang10 @PengHeAtlas @zhou_jingtian @SnyderShot @lecong Aviv Regev @jure
@StanfordAILab @genentech @phylo_bio @arcinstitute @UW @berkeley_ai @RetroBio_ @tamarindbio @Princeton @UCSF
