@Swetava

ML/AI Research @Apple. MS/PhD (#ComputationalMath) + MS (#ComputerScience) @Stanford. Formerly @GoldmanSachs. BTech @iitgn

Stanford, CA
Joined December 2009
Swetava Ganguli retweeted
DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair. It even works when the images and the captions come from different datasets. Project page: dominik-schnaus.github.io/un… ⬇️
80
345
83
2,332
228,242
Swetava Ganguli retweeted
Was meant to be a Valentine's gift, but I'm releasing it early ❤️ Wrote a long piece on KV caching in flow models for image generation. Includes intuition, commentary, implementation details, benchmarks, & some quirky experiments. sayak.dev/posts/kv-flow.html
5
45
1
395
26,262
Swetava Ganguli retweeted
So they say "three things we scaled". Let me translate: 1. "compute" - yep, that's compute. 2. "environments and harnesses" - actually, also compute. 3. "and grader compute" - you guessed it, that's also compute. joke aside, pretty cool to see their public live dashboard, including "cost so far" (!) (due to recent events/discussions: no, this QT is not sponsored. I don't do sponsor stuff, I'm here for the fun.)
Nearly half a year of silence. We spent it studying one problem: how far RL can scale. MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks. Streaming the run: mimo.xiaomi.com/rl/
16
17
1
489
46,887
Swetava Ganguli retweeted
Marvelously instructive, for anyone who wants a visual, physical understanding of the recent Navier-Stokes (Millennium) singular solution.
Navier-Stokes Millennium saga continues. Here are my slides and a summary from a recent talk. Disclaimer: this is a physicists view of the problem & solution! 🧵
5
112
708
54,411
Swetava Ganguli retweeted
I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: darioamodei.com/post/we-must…
5,162
7,169
4,384
67,511
17,153,128
Swetava Ganguli retweeted
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: darioamodei.com/post/we-must…
10,665
16,387
11,359
87,804
76,806,920
I just published my complete guide to reinforcement learning for LLMs. It's a single, standalone resource for understanding RL from first principles to frontier research. Read it here: cameronrwolfe.substack.com/p… The topics covered are as follows… RL Fundamentals: - A general framework for RL (policy, states, actions, transition functions, environment, rewards, trajectories, etc.) - Core concepts of RL (returns, discounting, action / state / trajectory probabilities, value / advantage functions). - Formulating and estimating the learning objective for RL. Basic of RL for LLMs: - Token-level (MDP) vs. completion-level (bandit) formulations. - Outcome versus process rewards. - Value estimation via value models / critics. - KL divergence, RL setups (RLHF, RLVR, infra, etc.), importance sampling. Basic Policy Gradients: - Deriving the RL objective / vanilla policy gradient (VPG) from first principles. - Different policy-gradient formulations (trajectory returns, reward-to-go, baselines, value functions, and advantages). - Why policy gradient estimates have high variance (and how baselines help). - Implementation of VPG. REINFORCE and its variants: - REINFORCE as a Monte Carlo implementation of the VPG. - Completion-level vs. token-level implementations of REINFORCE. - Reward baselines and KL-regularized rewards. - RLOO, REINFORCE++ and other critic-free policy gradient algorithms. Actor-Critic Methods: - Training value models / critics alongside the policy to estimate advantages. - Trust Region Policy Optimization (TRPO) and KL-constrained policy updates. - How PPO simplifies TRPO with a clipped surrogate objective. - Full PPO details (clipping, advantage estimations, value loss, KL, policy ratios, multi-epoch optimization, etc.) + implementation. - Generalized Advantage Estimation (GAE) + implementation. - The memory and computational overhead of actor-critic algorithms. GRPO: - Group Relative Policy Optimization (GRPO) as a simpler alternative to PPO. - Group-relative advantage estimates + implementation. - Why removing the critic is helpful. - Limitations of vanilla GRPO algorithm. - Dr. GRPO, DAPO, GSPO, and other recent improvements. Recent research topics: - Online versus offline RL. - Rubric-based RL. - RL scaling laws. - Agentic RL + world modeling. This post is a synthesis of my writing / learning on RL for over a year. I hope it’s helpful!
40
205
14
1,340
59,459
Swetava Ganguli retweeted
my RL for LLMs cheatsheet
4
18
202
12,174
Getting ready to publish my complete guide to RL for LLMs tomorrow morning. Although the post contains many of my own thoughts / learnings, it is also a synthesis of so many great resources that have been published over the years: - The RLHF Book (rlhfbook.com/) by @natolambert - Reinforcement Learning by Richard S. Sutton and Andrew G. Barto - Spinning Up in Deep RL (spinningup.openai.com/en/lat…) from OpenAI - Build an LLM (sebastianraschka.com/llms-fr…) and Reasoning Model (sebastianraschka.com/reasoni…) from Scratch by @rasbt - Various notes (joschu.net/blog/kl-approx.ht…) and papers (TRPO, PPO, etc.) from John Schulman - Policy Gradient Algorithms (lilianweng.github.io/posts/2…) by Lilian Weng - A Vision Researcher’s Guide to RL (yugeten.github.io/posts/2025…) by @YugeTen - From REINFORCE to Dr. GRPO (lancelqf.github.io/note/llm_…) by @qingfeng_lan - Async GRPO in the Wild (yumoxu.notion.site/async-grp…) by @yumo_xu - Open RL infrastructure like TRL (huggingface.co/docs/trl/en/i…) and OpenInstruct (github.com/allenai/open-inst…) I highly recommend reading all of them. They’ve truly helped me to learn so much.
23
189
9
1,400
78,685
Swetava Ganguli retweeted
🚢 Marin 535B-A23B started training this week! As usual, the whole process is open. Voyage plan: pretraining (80%) + midtraining (20%) on 18.75T tokens on 11 x GB200 NVL72 for ~3 months (2.7e24 FLOPs). Post-training will follow. Before kicking off the run, we trained a 4-rung scaling ladder from 1.6B-A61M (48B tokens) to 27.7B-A1.2B (926B tokens) to debug issues, and to make a forecast of our hero run. This is by far our biggest run, so definitely expecting the unexpected.
106
576
99
4,218
986,623
Swetava Ganguli retweeted
It's good to see more real-world uses of RL! I just added three such new cases to my slide-decks, one by Equilibre, one by Netflix and another by Meta. They are pretty cool. Check them out here: tinyurl.com/y2328a7j If you know about applications I missed, let me know!
10
52
4
350
33,728
Recently did some deep reading on continual pretraining (CPT) to better understand best practices for creating domain-specialized open LLMs. Here are the best practical resources / model reports that I found on the topic: - CPT best practices (Databricks): databricks.com/blog/characte… - Composer 2: arxiv.org/abs/2603.24477 - Code Llama: arxiv.org/abs/2308.12950 - DeepSeek-Coder-v2: arxiv.org/abs/2406.11931 - Qwen-2.5-Coder: arxiv.org/abs/2409.12186 - DeepSeekMath: arxiv.org/abs/2402.03300 - Nemotron-MIND: arxiv.org/abs/2410.12881 What am I missing? Will be releasing a brief writeup for this line of work soon!
10
34
1
247
14,673
Swetava Ganguli retweeted
My summer project is done! A 20 video, free course on post-training to accompany my book is all on YouTube with slides open for modification & re-use. ~12 hours of content covers the core foundations and some research areas I think will grow in importance. It was a fun time to review all the fundamentals again, as it is clear in the next 1-3 people the amount of people wanting to learn post training will likely 100X again from today, as we have already 100X'ed from two years ago. As AI agents get increasingly capable at coding and discussing these fundamentals (see the code exercises accompanying the book that I am refining with the community) I think developing clear intuitions for how models work and why is one of the most important skills going forward in AI. Still, learning the post-training math is the best way to battle test them. I personally just in this course am starting to master how forward/reverse KL relates to post-training topics. Thanks to all my viewers, and I'm happy to answer questions in the book discord or understand how to better teach the various reward models, on-policy distillation, new RL algorithms, etc. Plus, the book is 50% off right now with the code PBLambert on Manning to celebrate the launch. I'll share the relevant links below. Who's going to make this course for pretraining?
69
196
17
1,795
80,146
Swetava Ganguli retweeted
I and Azalia Mirhoseini built and taught the course on Self Improving AI Agents (CS329A) twice at Stanford now. Very excited to bring the course to the broader community online. YouTube: lnkd.in/gpeaN4re Research is moving very fast in this field - here we have focused on the key concepts that would allow you to reason from first principles.
I had so much fun creating the Self-Improving AI Agents course with @achowdhery, and teaching it twice in one year at Stanford! We also collaborated with Stanford Online to make the course available online: YouTube: youtu.be/6YnLB0XbTnI?si=MVwR… The field is moving incredibly fast, but we tried to focus on the core concepts that help us build better AI systems. I hope you enjoy it!
19
45
3
389
40,902
Swetava Ganguli retweeted
The final lecture of my course is an intro to character training! This is a topic that I've been quietly very invested in for ~18 months, as it: * Has potential for high real world impact * Clearly used extensively at frontier labs * Almost no empirical literature exists * More accessible on academic compute This lecture covers what character training is, reviews model specs, constitutions, the differences, the motivations in real world events, some example research papers I like, and open questions in how it relates to post-training/model use generally. Hopefully this brings more people into the field (and reach out if you have questions). It is one of the more research-y chapters in my book, but one that I felt needed the reference. There is still so little, educational content on the topic online. 0:00 Intro 6:22 Part 1: Fundamentals — character, constitutions, and model specs 19:21 Part 2: Character training in practice 23:23 Part 3: Character elicitation without gradient steps 28:03 Part 4: Open questions (and the end of the course) 32:27 The course, complete Thanks for watching. No need to like and subscribe now that the course is done, you definitely wouldn't! h/t to @_maiush for leading the technical work I got to do in the space, and @zafstojano for investing a lot of attention at this book chapter.
31
91
17
867
57,594
Swetava Ganguli retweeted
MIT FILMED A PROFESSOR WHOSE EXPLANATIONS ARE SO CLEAR AND INTUITIVE THAT THE MOST COMPLEX CALCULUS ON WALL STREET BECOMES OBVIOUS - AND IT PROVES WHY NEWTON'S VERSION GIVES THE WRONG ANSWER FOR EVERY OPTION PRICE This is an MIT lecture on Itô calculus, course 18.S096. The professor opens with one equation that breaks classical calculus - dB squared equals dt. In normal calculus, dt squared vanishes when you take limits. But the quadratic variation of Brownian motion forces the second derivative term to survive. This one fact changes every formula in calculus. Then Itô's lemma. In classical calculus, if you apply a smooth function to a path, you only need the first derivative term. In Itô calculus you need an extra term - half the second derivative times dt. He derives it in 10 lines of Taylor expansion, showing exactly which terms survive and which vanish. The correction term is not a small adjustment. It is the difference between the right answer and the wrong one. Then geometric Brownian motion. The natural guess for modeling stock prices is to take the exponential of a Brownian motion. It fails - the Itô correction introduces an unwanted drift term that makes the expected value grow over time. The fix is to subtract exactly half of sigma squared from the exponent. This is the formula that runs every Black-Scholes calculation on earth. Then adapted processes and Itô integrals. In classical integration it doesn't matter which point in each interval you use to form Riemann sums - the limit is always the same. In Itô calculus left endpoints and right endpoints give different answers. Itô's choice of left endpoints is not arbitrary - it encodes the fundamental constraint that financial decisions must be made using only past information. Watch the moment he explains the Girsanov theorem - that a Brownian motion with drift and a Brownian motion without drift are equivalent probability measures. Two processes whose paths look completely different in the long run can be converted into each other by multiplication. This is how quants transform non-martingale stock price processes into martingales for pricing. A derivatives trader I know rewatched this lecture before his first day at a quantitative hedge fund. Said it was the first time Itô's lemma felt like a theorem with a reason rather than a formula to memorize. Free on YouTube, MIT OpenCourseWare, Creative Commons license. bookmark this and watch later - after this lecture every option price you see will feel like a solution to a stochastic differential equation waiting to be written down
5
96
1
518
47,262
Swetava Ganguli retweeted
"DiffusionGemma Technical Report" DiffusionGemma turns Gemma 4 26B A4B into a discrete diffusion model that denoises 256 tokens in parallel, reaching about 20 tokens per forward pass and 1,500 tok/s on one H100. By using SFT to teach bidirectional denoising, then use sampler distillation plus RL to reduce denoising steps, they did not have to train this DiffusionLM from scratch while maintaining generation quality. Read more about the technical report here: alphaxiv.org/abs/2608.00146
3
71
2
367
22,401
Swetava Ganguli retweeted
25 years ago today I officially started at a small search engine company, wedged into a tiny office space above what's now a T-Mobile store in downtown Palo Alto. Since then, I have had the incredible pleasure of working with awesome colleagues on software used by billions of people all over the world. To all of my current and former colleagues, thank you for the collaborations, and I delight in what we have helped build together! 🙏 My first office: Current office, my 16th different desk location at Google (more desk moves than 98.88% of Googlers): There's a fun internal site that lets you see what percent of employees are newer than you at the company. I'm almost at five 9s overall, but infinite 9s within our engineering group. 😅 Thank you for indulging me in my moment of reflection!
192
142
35
3,872
322,165
Swetava Ganguli retweeted
My evaluation lecture! I walk you through different evaluation eras I've been a part of, from prompting GPT-3 as elaborate autocomplete to today's complex agentic sandboxes (I expand on agentic more than any other topic, drawing on @xeophon's insights). This lecture is a birds eye view of how evaluation has changed, how it can be gamed, and what it's actually used for. 00:00 Intro: frontier evaluation is harder than ever 03:39 Part 1: The eras of post-training evaluation 17:36 Part 2: An intro to agentic evals 21:19 Part 3: Can you trust the number? 30:41 Takeaways & conclusion Thanks for watching! Just one more lecture after this :)
23
68
3
602
39,572
Swetava Ganguli retweeted
Tomorrow will be my last day at Google after 27 years, and watching it grow from 25 people to 190,000+ has been an amazing journey. Below is a note I shared with many people internally at Google today. An excerpt is: It has been an absolute pleasure to work with you and to help build some of the most widely used and impactful products of all time. As a kid, I dreamed of helping build software that would be used by many people, and Google now has thirteen products used by more than a billion people (amazing!). Our work has had a tremendous impact in the world, and I have been lucky enough to collaborate and form friendships with many colleagues that I deeply admire, respect, and enjoy. It still brings me joy every time I see people out in the world using our products to find information, handle email, translate documents, watch videos, learn new things, navigate and understand the physical world, browse the web, use their phone, run large-scale computations on our infrastructure, ride in an autonomous vehicle, or perform complex tasks with the help of our AI systems. I hope you all share this sense of joy, because it is a shared accomplishment! Thank you to all of my colleagues at Google over many years! Now I'm excited to go start @DiscoLoopAI with my longtime friends and colleagues @Sanjay_Ghemawat, @OriolVinyalsML, and @quocleix. (Updated post: slightly redacted to not have some personal info)
846
2,764
757
33,721
11,896,722