@winglian
Account based inUnited States
About this account
- Account based in
- United States
- Connected via
- United States App Store
Account-level information from X, not a live location or the device used for a specific post.
@axolotl_ai OSS maintainer. Axolotl AI founder. AI/ML tinkerer. Building tools for everyone.
Annapolis, MD
Joined March 2009
- Tweets3K
- Following1.9K
- Followers11K
- Likes3.4K
Wing Lian (caseus) retweeted
Today, I’m open-sourcing Drost-v1 as Drost Community Edition.
Drost Community is a genuinely powerful offensive cybersecurity tool: It comes with one Docker container, 159 MCP tools, Linux and MacOS support, and direct integration with MCP-compatible AI agents.
In short, you can use your existing coding agents to carry out offensive cybersecurity engagements.
It runs entirely locally, on your hardware, with your models. Drost receives no engagement telemetry.
This is where Drost began.
drost.ai/community
Enjoy! 🚀💥
We can show via experiments on synthetic tasks that there is may be an improvement on some tasks from balancing label counts. While this does show that packing may in fact be worse for this particular synthetic set of tasks, we do see a trend that optimizing for label counts does seem to help.
Axolotl had an issue a while back where we were doing BFD over the entire dataset and saw loss divergence near the end of an epoch where the short tail of multiple documents were concentrated. More modern BFD implementations pack over shorter windows to mitigate this, but I’m not sure structured packing like this is optimal.
But can we do better? For most SFT workloads, we’re coordinating two things, packing to minimize padding tokens, minimizing loss via supervised labels. The thing we’re not doing today is also co-optimizing the distribution of supervised labels. By co-optimizing for label counts we can sharpen the distribution of labels to reduce gradient noise across steps.
Using BFD, you end up with longer samples getting paired up with smaller samples. Depending on the correlation of tasks to sample lengths, this can lead to some pairing off of tasks within a batch (at smaller effective batch sizes).
Hot take for today; BFD packing is suboptimal for actual post-training. Your effective batch sizes are too small and your trading off structured packing for accuracy.
Just found an additional 8% speedup in axolotl just by adapting our existing fused rope+rms norm attention kernels for GLM4.
If you're curious why they only reported against TRL for their single GPU SFT on GLM Flash, it's because at that regime, they're still slower than Axolotl by about 5% (can't find any B300s, so using B200s). But hey, reporting benchmarks these days is just about cherry-picking data, right?
Introducing Halo, the best framework for post-training of open-source models.
Halo delivers up to 2.8x the throughput of stock TRL with less peak memory, while models stay in their native HuggingFace format.
Star us on GitHub: github.com/whitecircle/halo
Wing Lian (caseus) retweeted
After many months of intense work the @Snowflake AI Research team is happy to present to you the new open source project: Arctic RL
snowflake.com/en/blog/engine…
- Arctic RL integrates with VeRL and SkyRL today; enable ZoRRo with one config flag, no code changes required
- ZoRRo delivers up to 6x actor-update acceleration and a 3.5x end-to-end training speedup, reducing Arctic-Text2SQL-R2 training from ~5 days to ~36 hours on 32 H200 GPUs
- Arctic-Text2SQL-R2 achieved higher accuracy scores (48.7) than Gemini 3.1 Pro (47.9) and Claude 4.7 (47.3) on Snowflake's evaluated enterprise SQL benchmark under the tested conditions
- Two open source recipes ship with this release: a text-to-SQL recipe that improved BIRD dev accuracy from 59.92% to 70.35%, and a multi-hop QA recipe that improved average accuracy from 69.6% to 72.3%
I honestly have no words for calling it “megakernel” when in fact it’s just a bag of kernels.
Megakernel for Laguna XS + GRPO training on a single RTX 4090:
~180 tok/s @ batch 1
~499 tok/s @ batch 8
In-place, memory-efficient forward & backward wiith custom kernels.
@poolsideai @eric_alcaide
github.com/KONAKONA666/fastg…
We’re open-sourcing Hatchery Core.
It’s a Tinker-compatible runtime for post-training models on infrastructure you control. Hatchery ships with local/self-hosted gateway, LoRA training, and pluggable backends.
Hatchery is perfect for teams sharing compute for RL workloads with high environment latency. These workloads often keep your trainer idle due to environment spinup and verification/rewards.
Apache 2.0, early alpha.
Wing Lian (caseus) retweeted
Support for @MistralAI's Mistral Medium 3.5 just landed in Axolotl 🎉
128B dense, QLoRA, single GPU — ready to fine-tune today.
docs.axolotl.ai/docs/models/…
Wing Lian (caseus) retweeted
Anthropic just launched the Claude for Open Source Program and they are giving away Claude Max 20x to open-source contributors as a thank you for keeping the ecosystem running.
Here is what you get👇
6 months of Claude Max 20x
Their highest usage tier completely free
You qualify if you maintain or contribute to any public open-source project.
Only 10,000 spots available and they are filling on a rolling basis.
Link to apply is in the comments.
Anthropic's solution to improve uptime is nerfing Opus 4.7 so it's worse at tackling lists of tasks by stopping more often before continuing on. I suppose that's one tradeoff you can make to hit those 9's.
1.8x Faster GRPO! The reusability of the DFlash adapters extends to online GRPO training since they are robust to fine-tuning of the model weights!
The beauty of DFlash is that it reuses the hidden states of the active model, so the you can use DFlash adapters for the base models with post-trained models like Carnice 9B/27B by @kaiostephens and Ornstein by @DJLougen and get these local ~4x speedups for you local @NousResearch Hermes Agent. In testing, the average acceptance rate of Carnice 9B is ~7.5 tokens using the z-lab/Qwen3.5-9B-DFlash adapter. @zhijianliu
The beauty of DFlash is that it reuses the hidden states of the active model, so the you can use DFlash adapters for the base models with post-trained models like Carnice 9B/27B by @kaiostephens and Ornstein by @DJLougen and get these local ~4x speedups for you local @NousResearch Hermes Agent. In testing, the average acceptance rate of Carnice 9B is ~7.5 tokens using the z-lab/Qwen3.5-9B-DFlash adapter. @zhijianliu
Wing Lian (caseus) retweeted
Here is a little torch.Tensor.__repr__ hack that allows you to dump custom tensor attributes in addition or instead the normal tensor's contents
I found it very useful when I need to debug mismatching shapes and I didn't want to add more debug code
github.com/stas00/the-art-of…
