💡Excited to be organizing a tutorial at ASPLOS 2026 (lnkd.in/g5auexxg): "Building Efficient Large-Scale Model Systems with DeepSpeed: From Open-Source Foundations to Emerging Research" 🌀 Link: lnkd.in/gixRnAm6 📍 Room: Allegheny 🕘 Time: Monday (Mar 23), 8:30am-12pm 🎤 Speaker: Tunji Ruwase, Masahiro Tanaka, Minjia Zhang, Zhipeng Wang, PhD We will cover how @DeepSpeedAI enables new forms of parallel, distributed, and heterogeneous execution, and how modern systems tackle key challenges in parallelism, offloading, and memory efficiency. If you are working on ML systems, LLM training, or emerging hardware, would love to connect at ASPLOS!

I am thrilled to release our newly re-architected extremely-scale Linear Programming Solver (DuaLip-GPU), which is developed via PyTorch enabling multi-GPU computations and parallelism (github.com/linkedin/DuaLip). We also released the technical report (arxiv.org/abs/2603.04621) covering all technical details. Linear Programming Solver is a fundamental building block for solving extreme-scale matching problems, which underline many important technical domains related to social network platforms such as ranking, personalization, item-matching and recommendation systems, as well as in LLMs. To realize the available parallelism, we develop GPU execution techniques tailored to sparse matching constraints, including constraint-aligned sparse layouts, batched projection kernels, and a distributed design that communicates only dual variables. Further, we improve the underlying ridge-regularized dual ascent method with Jacobi-style row normalization, primal scaling, and a continuation scheme for the regularization parameter. On extreme-scale matching workloads, the GPU implementation achieves at least a 10x wall-clock speedup over the prior distributed CPU DuaLip solver under matched stopping criteria, while maintaining convergence guarantees. This is the superb technical work combining ML Systems, Mathematical Optimization and Machine Learning. #Optimization #AI