Very excited to announce our new models: Nemotron 3.5 Super and Lightning! This also feels like a good time to share that a few months ago I joined the amazing Nemotron Pretraining team. We're cooking some very large and very strong models for you! Here is a tech summary for 3.5: * Super is a 120B total / 12B active vision language model (VLM) * Lightning is a 30B total / 3B active text-only LLM * Super scores 51.78% on SWE-bench Pro (better than Nemotron 3 Ultra despite ~4.6× fewer active params!) * arch: MoE Hybrid Mamba-Transformer designed for efficient inference, with a 1M token ctx window, and native speculative decoding (via MTP) * training is done across our GB200 and GB300 (and deployment validated across all imaginable SKUs, including consumer (RTX) and DGX Spark) A few details on how we train these: -- Pretraining * pipeline: initialize with Nemotron 3 base -> custom MTP initialization + 100B-token MTP warm-up -> 1T-token distillation from the next larger N3 teacher * Lightning then goes through a 21B token long-context (LC) stage (@ 1M ctx) * Super goes through: - Indic CPT: 100B tokens at 8K (10B Indic + 90B replay, we built a synthetic corpus with 2.83T toks across 22 Indian langs! Indic benchmarks improve by up to 9.82%) - LC extension: 35B tokens at 1M - Agentic SWE mid-training: 65.4B tokens at 256K (uses whole repos, issue-PR pairs, commit histories, etc.); DeepSWE pass@1 improves from 6.4 -> 10.4 - LC recovery: 9B tokens at 1M (as we observed regression after mid-training) -- Post-training * Super: two-stage SFT -> joint RLVR/RLHF -> specialist teachers -> two-stage MOPD -> MTP boosting * Lightning uses a lighter text-only pipeline: single-stage SFT -> RLVR * Super's SFT transforms the text-only base into a VLM using vision, agentic, reasoning, chat, safety, LC data * A few examples of RLVR environments: - Olympiad-level competitive programming - Finance agents using SEC search, document parsing, evidence retrieval, and tool use * Specialist teachers include Agentic, Reasoning, Video reasoning and tracking, Vision tool-use, Vision-reasoning, and an Alignment/RLHF teacher * Two-stage MOPD consolidates these specialist capabilities into one model * Post-training ends with MTP boosting to better align the drafter and reduce recursive train–inference mismatch; this improves Super's SPEED-Bench acceptance length by 21.8% (to 4.612 tokens) -- Inference The model is highly optimized for inference (with plenty of quantized options to choose from)! A few data points: * On a GB200, Super reaches 4,856 output toks/s/GPU with ≥50 toks/s/user on a 20-turn agentic workload * Super provides 2.34x the throughput of Qwen3.5 122B-A10B and 1.90x the throughput of Qwen3.8 Flash-Next * Native MTP is complemented by standalone parallel drafters: LiLiCorr for Super, and DFlash and DSpark for Lightning Looking forward to seeing what you do with 3.5! Phase shift for Nemotron incoming! :)) (with the usual this will be our weakest model ever, but really really so) paper: research.nvidia.com/labs/nem…

a rumor that the VARA AI fund has blown up is making the rounds