Pinned Tweet
AI is compute-hungry. While it has generally relied on a single hardware vendor in the past, AMD GPUs now offer competitive memory and compute throughput. Yet, the software stack is brittle.
So we ask: can the same DSL principles that simplified NVIDIA kernel dev translate to AMD? We’re excited to introduce the newest addition to the ThunderKittens cinematic universe of kernel DSLs: HipKittens (HK) 🚀for Fast and Furious AMD kernels.
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel.
Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year.
We are deeply grateful to the developers and companies building on Modal. There's still so much more to build, and we can't think of a better time to be building software.
my journey into kernel engineering began with porting @Si_Boehm’s gemm ladder to amd - definitely feels like a full circle moment!
Replying to @simran_s_arora
our blog is inspired by the OG @si_boehm’s educational gemm ladder!
check out our blog here: rocm.blogs.amd.com/software-…
with @neoblizzz, ryan swann, @_williamhu, @seansiddens, @drewwad, julia zhang, alex underwood, alex dutu, @dylan__lim. thank you @amd @AIatamd for the support!
William Hu retweeted
we got early access and hipkittens is now running on amd helios mi455! here we share an educational gemm ladder to demonstrate how to use the new helios features.
overall, we found that the patterns identified in hipkittens for writing performant mi350/355 kernels translate well to helios making the forward port quite smooth!
🤖 Made with AI
William Hu retweeted
We’ve seen an explosion of new ML chips with unique architectural features, but software support remains the critical bottleneck
Achieving peak performance increasingly relies on hardware-specific optimizations in the kernels, but we observe that coding agents are particularly weak at this
Introducing Hawkeye, a framework that brings hardware-awareness to coding agents by grounding them in a minimal and comprehensive taxonomy of optimization strategies
For new GPU or ML accelerator architectures, you only need to write 10 unit tests and solution kernels (one per optimization strategy), and we show that coding agents can effectively scale test-time compute with this minimal supervision to write hardware-aware kernels
Hawkeye can port kernels across architectures (Ampere, Hopper, Blackwell), vendors (NVIDIA, AMD), and precisions (FP8, NVFP4, MXFP4) while consistently leveraging hardware features and approaching expert kernel performance
Work co-led with @keramakr and done in collaboration with Alexander Ingare @simonguozirui @18jeffreyma @ZishenW @simran_s_arora @Azaliamirh @profvjreddi
Big, big appreciation to the SGLang team — playing a major part in our ability to stand up great performant endpoints on day-0, model after model.
Open source is a joyous place to collaborate, thanks to these folks.
SGLang day-0 speed on Kimi K3: 423 tok/s (measured on gsm8k), plus RL support ready in Miles @radixark!
How the largest open-source model runs this fast: we natively implemented and deeply optimized K3’s new architecture with fused KDA decode kernels, DP attention, DSpark, PD disagg, and KDA-aware prefix caching. We've passed Kimi Vendor Verifier and are ready for production!
Thanks to @Kimi_Moonshot, @nvidia, @AMD, @KVCache_AI, @modal, and @baseten for building this with us, and to @googlecloud, @nebiustf, @fal, @digitalocean, @runpod, @DeepInfra and @gmi_cloud for serving K3 on SGLang.
Blog, cookbook, benchmarks in the comments.
P.S. This demo video? Kimi K3 made it itself. Play the game 👇
William Hu retweeted
Frontier models require frontier speculators. Our team moved mountains to achieve 460 tok/s with our custom DFlash model.
Try it day 0 at modal.com/endpoints.
Kimi K3 is live on Modal.
Moonshot has shipped the world's first open 3T-class model, and we're a day zero launch partner.
We trained a custom DFlash speculator for K3's novel architecture so you can run it faster, losslessly. The most capable open model we've worked with by far.
William Hu retweeted
so cool to see hybrid based linear attentions + kernelbench + full-model megakernels put to use in building and evaluating frontier models!
+ hipkittens is now delivering the fastest mla decode (github.com/ROCm/aiter/pull/3…) and dsv4 decode (github.com/ROCm/aiter/pull/3…) kernels in amd's aiter!
never doubt the impact of academic research 🚀
Try out the speculator at huggingface.co/modal-labs/In… ⚡️
Inkling by @thinkymachines is now available on Modal, backed by a custom DFlash speculator for 67% higher throughput and interactivity.
Running on Modal Auto Endpoints with SGLang today.
William Hu retweeted
Modal trained a DFlash speculator that's much faster than MTP, making it a great boost for inference speeds!
Inkling by @thinkymachines is now available on Modal, backed by a custom DFlash speculator for 67% higher throughput and interactivity.
Running on Modal Auto Endpoints with SGLang today.
Our new Auto Endpoints feature is powered by a new Modal primitive: Modal Servers.
In this blogpost, we walk through design principles and detailed architecture: @EnvoyProxy, @googlecloud Spanner config store, and a @Cloudflare Pingora-based custom proxy.
William Hu retweeted
Happy to share new ThunderKittens attention kernels for B300 GPUs -- faster than FA4! Check it out:
Excited to share new ThunderKittens attention kernels that match or outperform Flash Attention 4 on Blackwell GPUs! Currently only supports QK192/V128 shapes, but more coming soon.
Check out the code here: github.com/HazyResearch/Thun…
Shoutout to the FA4 team for the algorithmic innovations and to @stuart_sul for the helpful discussions.
Modal's been an amazing abstraction to work with! Excited to be building it out more! 🚀
Fresh blog post!
@modal partnered with @ScalingIntelLab, @HazyResearch, and @chelseabfinn's IRIS Lab to speed up research on speeding up AI research.
Read how scientists at the cutting edge are building the machines that build the machines with Modal.
modal.com/blog/accelerating-…
William Hu retweeted
Announcing Flapping Airplanes!
We’ve raised $180M from GV, Sequoia, and Index to assemble a new guard in AI: one that imagines a world where models can think at human level without ingesting half the internet.
William Hu retweeted
Much like the switch in 2025 from language models to reasoning models, we think 2026 will be all about the switch to Recursive Language Models (RLMs).
It turns out that models can be far more powerful if you allow them to treat *their own prompts* as an object in an external environment, which they understand and manipulate by writing code that invokes LLMs!
Our full paper on RLMs is now available—with much more expansive experiments compared to our initial blogpost from October 2025!
arxiv.org/pdf/2512.24601
William Hu retweeted
Happy 🦃 Thanksgiving weekend! 🍂 This year, we cooked up a new recipe for juicy fact-storing MLPs. Instead of picking apart trained models, we asked: Can we construct fact-storing MLPs from scratch? 🤔
Spoiler: we can & we figured out how to slot these hand-crafted MLPs into Transformer blocks as modular fact stores! 🧩
New work with @garctrob @ronnygjunkins @jerrywliu @dylan_zinsley @EyubogluSabri Atri Rudra @HazyResearch!
🧵👇
William Hu retweeted
Super excited for ParallelKittens led by @stuart_sul! From the Nvidia A100 to the B200, BF16 tensor core performance improved by 7.2× and High Bandwidth Memory bandwidth by 5.1×, while intra-node communication (NVLink) improved by only 3× and inter-node (PCIe/InfiniBand) by just 2×. PK helps with multi-gpu kernels!
(1/6) GPU networking is the remaining AI efficiency bottleneck, and the underlying hardware is changing fast! We’re happy to release ParallelKittens, an update to ThunderKittens that lets you easily write fast computation-communication overlapped multi-GPU kernels, along with new kernels for data, tensor, sequence, and expert parallelism!
Here’s a photo of overlapped kittens, along with things you should care about when optimizing multi-GPU kernels.
(With @simran_s_arora, @spectorb, and @hazyresearch. Generously supported by @cursor_ai and @togethercompute)





































