Interested in ML for Systems, Domain-Specific AI Accelerators, and Compilers. Retweets and likes are not endorsements

United States
Joined December 2018
Rasool Sharifi retweeted
We have post-trained GLM 5.3 flash to become as strong as Fable 5.1 on SRE tasks (that is bug finding in complicated software deployments). Checkout our tech paper to see how we did it. web-assets.dd-static.net/425…
18
75
8
755
40,665
Rasool Sharifi retweeted
Spent the past three weeks on this idea. A sampler on a different precision or accelerator is just off-policy, like stale samples in fully async RL, and off-policy correction doesn't care why the sampler differs from the trainer. Meanwhile, inference keeps diversifying across precisions(MXFP8, MXFP4, NVFP4) and accelerators outside RL. So why not use all of that compute for RL rollout? The sampler and the trainer are two systems built for different purposes. Hope this opens up new possibilities for RL algorithms and systems, and leads to cleaner system designs. Enjoy the blog, and stay tuned for the next one on our infra :)
Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive. So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.
23
32
8
383
31,478
Rasool Sharifi retweeted
Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive. So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.
32
128
64
1,660
160,307
Rasool Sharifi retweeted
Real inference workload comparison of NVFP4 and MXFP4 by @realcezarc cezarcocu.com/blog/nvfp4-vs-… TLDR: on B200 GPU, you should probably always prefer NVFP4 over MXFP4, by a meaningful margin (up to ~8% faster decode at small batches)!
11
25
4
234
28,961
Rasool Sharifi retweeted
Reasoning From Scratch: Reinforcement Learning with Verifiable Rewards (RLVR) round 2. Covering clipped policy ratios, KL loss term, format rewards, and other GRPO tips & tricks. 00:00 Introduction and recap 01:52 Interpreting basic GRPO training metrics 06:34 Planned improvements to GRPO 08:58 Running longer training jobs with Python scripts 13:39 Running the baseline GRPO training script 17:29 Loading and plotting training logs 19:29 Diagnosing unstable training 23:55 Evaluating checkpoints on MATH-500 26:26 Downloading existing checkpoints 30:09 Tracking advantage statistics 34:53 Understanding entropy 40:32 Computing entropy in PyTorch 44:17 Interpreting entropy values 48:58 Adding entropy tracking to GRPO 53:36 Analyzing advantage and entropy metrics 56:18 Stabilizing GRPO with clipped policy ratios 1:03:27 Implementing the clipped policy loss 1:09:39 Analyzing clipped policy training results 1:11:25 KL divergence and reward hacking 1:15:12 Adding a KL loss term 1:20:34 Limitations of the simplified KL loss 1:23:04 Format rewards and think tags 1:25:47 Adding special tokens to the tokenizer 1:30:29 Implementing the format reward 1:35:56 Analyzing format reward training 1:38:25 Rewarding format only for correct answers 1:40:48 Further GRPO improvements from research 1:45:43 Next steps and distillation
79
136
4
1,244
56,310
Rasool Sharifi retweeted
What is not covered in the report is how we scaled model training to be efficient on 768 B200 GPUs. I explain it in detail here: aleph-alpha.com/en/blog/scal… A how to guide to scale up model training without wasting your GPU hours. Even if you only have 1 or 2 GPUs and cannot scale, our optimisation philosophy can still make your GPUs go brrr. I couldn't explain ALL the details of our method in the blogpost, since the big launch came after. You can connect the dots now, the 30B MoE architecture was in fact Kolibri Origin all along.
15
78
3
743
176,523
Rasool Sharifi retweeted
It's extremely rare to encounter such a thoughtful blog on pretraining and scaling. I really liked the profiling plots depicting a clear yet sufficient picture.
What is not covered in the report is how we scaled model training to be efficient on 768 B200 GPUs. I explain it in detail here: aleph-alpha.com/en/blog/scal… A how to guide to scale up model training without wasting your GPU hours. Even if you only have 1 or 2 GPUs and cannot scale, our optimisation philosophy can still make your GPUs go brrr. I couldn't explain ALL the details of our method in the blogpost, since the big launch came after. You can connect the dots now, the 30B MoE architecture was in fact Kolibri Origin all along.
1
33
1
496
36,215
Rasool Sharifi retweeted
For years, maybe a decade, I've wanted to build a writing tool that just did a few things I couldn't find anywhere else. A system that mimicked how I think when I write. A system that provided clear affordances, features, and places to do the things I wanted to do. A way to play with my writing as I'm writing it. A way to settle into just the right way to say something. It never made sense to make it a 37signals product because it's not commercially viable, nor would it be worth pulling people off their other work to hack on this. Plus, it would have taken months the old way. So this weekend I just made it myself, with Claude's help. It's called Write_On and the short video walks you through it. Essentially it's "alternative control" at the word, sentence, and paragraph level, plus a way to dim stuff back, and stash stuff near by. Not version control, but alternative control. You'll see what I mean when you watch the video below. So here it is. Write_On.
460
239
174
4,772
531,678
Rasool Sharifi retweeted
Reasoning from scratch, round number 5! This time, talking about log-probability scoring (also a great fundamental concept for loss functions like cross-entropy in pre-training and distillation) and self-refinement. 00:00 Introduction and inference-time scaling recap 05:02 Loading the pretrained LLM 08:00 Comparing and scoring model answers 10:18 Building a rule-based scorer 17:53 Token probabilities and sequence likelihood 26:47 Computing token probabilities in PyTorch 30:12 Token indexing and shifted targets 37:27 Log probabilities and numerical stability 45:57 Scoring answers with average log probabilities 56:24 How self-refinement works 59:07 Generating critiques and revised answers 1:01:00 Implementing the self-refinement loop 1:05:57 MATH-500 evaluation results 1:07:35 Takeaways and next steps
43
130
5
1,030
59,449
Rasool Sharifi retweeted
LLM Compressor v0.14.0 is out, and GPTQ just got its biggest speedup since launch. A new Triton kernel makes quantization ~15x faster end to end. Batching layers that share a shape pushes that to ~30x on some MoE workloads. Even the old eager path is 1.5-2x faster. Also new: expanded MSE/iMatrix observers that beat GPTQ for NVFP4 on internal benchmarks, REAP pruning with distributed DDP, and support for GLM 5.3 and Qwen3.8. Full release notes: github.com/vllm-project/llm-…
3
17
3
118
6,838
Rasool Sharifi retweeted
Our first TPU megakernel for Kimi K3 reaches 709 tokens/s on low-concurrency decode, against 450 tokens/s for our GB200 baseline, both with DSpark speculative decoding. To our knowledge, this is the first TPU inference megakernel. The whole model runs in a single Pallas kernel, and without spec decoding it is roughly 1.4 to 2x the GB200 baseline at batch sizes 1 through 8. We are open sourcing it today. 1/2
26
93
38
865
244,832
Rasool Sharifi retweeted
Delta-rule linear attention appears in LLMs as Gated DeltaNet (GDN; Qwen3.8) and Kimi Delta Attention (KDA; Kimi K3). KDA swaps GDN’s scalar gate for a channel-wise one, but does that improve recurrence expressivity? With 2 tweaks, KDA can do 2D rotations. GDN can’t. How? 🧵👇
5
36
6
221
23,172
Rasool Sharifi retweeted
the most insane part, they will release ~7k RL training data and the framework leading to this top 6 model on AA, they also shipped the model + tech report less than 1 week after starting the final RL run pushing both intelligence and openness level, huge congrats
MiMo-V2.6-Pro debuts as the top open weights model on the Artificial Analysis Intelligence Index (46). At $0.13 per Intelligence Index task, it lands on the Intelligence vs. Cost per Task Pareto frontier @Xiaomi has just released MiMo-V2.6-Pro, an open weights model with major advances in intelligence over its predecessor, MiMo-V2.5-Pro (Intelligence Index: 26). Despite the improvement, it retains the same attractive pricing at $0.435 per 1M input tokens (with a 99% cache-hit discount) and $0.87 per 1M output tokens. This makes MiMo-V2.6-Pro one of the most cost-efficient models to deploy. MiMo-V2.6-Pro is an MoE model with 1.02T total parameters and 42B active parameters. Stay tuned for additional analysis of the model. Check out MiMo-V2.6-Pro full benchmarking breakdown here: artificialanalysis.ai
33
167
11
2,093
120,151
Rasool Sharifi retweeted
MiMo-V2.6: The Hard Road to Scaling Up RL MiMo-V2.6 is very likely one of the largest single RL runs, by compute, that any open-source model team has undertaken to date. In an era when compute is brutally scarce, we still chose to dedicate a team of several dozen people to one goal over an extended period: scaling up RL. That takes more than research conviction. It takes a vision for AGI, respect for the unknown, and the nerve to walk straight into the hardest problems. The result is a model whose potential was built through mid-training and unlocked through heavy RL. Today, it is the number one open-source model. I strongly recommend reading the technical report. I believe it will become one of those papers that Agent RL practitioners keep reopening and discovering something new in each time. In my view, the research innovations and engineering challenges behind it surpass those of DeepSeek R1, which I was partly involved in. Some will ask: why MixRL instead of MOPD? First, they are not competing choices. We ran MixRL on verifiable tasks of moderate difficulty, including code and related agentic tasks, and found that the resulting models generalize remarkably well. Second, tasks that are difficult to verify, extremely long-horizon, or simply too challenging to include in a joint RL run are trained separately. Including them would substantially reduce rollout efficiency or introduce significant rollout staleness. We then merge the resulting capabilities through MOPD. Games, 3D tasks, and tasks with subjective evaluation signals all fall into this category. There is also a third, slightly cheeky answer. Our team is flat enough and free enough of organizational silos that MixRL simply is not difficult for us. More importantly, everyone enjoys working this way. People from different domains come together every day, driven by the pursuit of AGI and intelligence that can continuously improve itself, to confront and resolve the RL bottlenecks in each field. I will always remember the RL daily update meetings from this period. They were intense and dense, with intelligence emerging in real time. To help the open-source community focus on solving real Agentic RL problems, we have released a Qwen model distilled from MiMo RL trajectories as a stronger starting point for RL, along with 7K diverse environments and a complete RL training framework. We hope these resources will help move Agentic RL research forward. MiMo-V2.6 is only the beginning. In an era when intelligence is easy to replicate, we still choose the hard road toward self-improvement and AGI. Much of what lies ahead remains unknown. But we are willing to keep investing the time, compute, and passion required to take on one hard problem after another and work each of them all the way through, until intelligence crosses into a new regime.
237
356
130
4,334
673,399
C'est un honneur pour la cheffe adjointe @MelissaLantsman et moi d'accueillir au Canada le prince héritier @PahlaviReza pour défendre la liberté et dénoncer le terrorisme. Nous avons discuté de la nécessité d'expulser du Canada les agents terroristes du CGRI et de sanctionner le régime terroriste illégitime de Téhéran, qui a tué des dizaines de citoyens canadiens et des milliers de ses propres citoyens. Merci, prince héritier, de lutter pour la liberté de votre peuple.
743
3,656
102
11,882
189,363
Rasool Sharifi retweeted
Why is RL so sensitive to training-inference mismatch (TIM)? With each training step, the trainer is pulled toward the sampler, resulting in accumulating drift. We exploit this intuition to devise a novel correction method to stabilize RL under TIM, called Score Centering.
18
60
11
583
72,451
Rasool Sharifi retweeted
Inference scaling part 1. Starting with a modded text generation function (temperature scaling, top-p filtering, multinomial sampling) to generate diverse outputs for self-consistency and best-of-N (improving answer accuracy by>2x) 00:00 Introduction and recap 00:31 Training-time and inference-time scaling 07:52 What we'll implement 11:47 Notebook setup and model loading 17:43 Building a flexible text generation function 24:40 Chain-of-thought prompting 28:26 Sampling and output diversity 33:43 Next-token logits and greedy decoding 38:20 Temperature scaling step by step 42:46 Softmax and token probabilities 47:42 Multinomial sampling 54:51 Adding temperature sampling to text generation 59:31 Top-p filtering step by step 1:10:23 Adding top-p filtering to text generation 1:13:43 Sampling and LLM watermarking 1:16:01 Self-consistency and majority voting 1:20:36 Implementing self-consistency 1:29:02 MATH-500 results 1:35:01 Accuracy and compute tradeoffs 1:36:50 Next steps and self-refinement
42
127
5
951
45,067
Rasool Sharifi retweeted
the answer is yes and @dogacel0 proved you can do that with his GPU kernel autoresearch project: github.com/Dogacel/auto-gpu-…
Replying to @maharshii
is this the strat to rankmaxx on the @GPU_MODE leaderboard?
1
7
129
7,057
Rasool Sharifi retweeted
Sharing my first of hopefully many research blog posts! jasperlu.com/blog/training-s… This one is the kind of educational blog post I wish I'd had when I started with RL for LLMs. I tried to make it as open as possible. Every rollout is browsable, the code is open source, and I walk through my entire thought process, from learning rate sweeps to reward shaping.
17
99
11
852
115,906