Stanford, CA
Joined July 2022
AI is compute-hungry. While it has generally relied on a single hardware vendor in the past, AMD GPUs now offer competitive memory and compute throughput. Yet, the software stack is brittle. So we ask: can the same DSL principles that simplified NVIDIA kernel dev translate to AMD? We’re excited to introduce the newest addition to the ThunderKittens cinematic universe of kernel DSLs: HipKittens (HK) 🚀for Fast and Furious AMD kernels.
7
36
8
157
54,578
William Hu retweeted
literally modal rn
1
2
38
1,601
William Hu retweeted
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel. Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year. We are deeply grateful to the developers and companies building on Modal. There's still so much more to build, and we can't think of a better time to be building software.
85
105
112
1,238
308,492
William Hu retweeted
t-minus 7 days till runtime @modal
6
9
2
75
5,823
my journey into kernel engineering began with porting @Si_Boehm’s gemm ladder to amd - definitely feels like a full circle moment!
3
58
7,679
we got early access and hipkittens is now running on amd helios mi455! here we share an educational gemm ladder to demonstrate how to use the new helios features. overall, we found that the patterns identified in hipkittens for writing performant mi350/355 kernels translate well to helios making the forward port quite smooth!
🤖 Made with AI
2
24
6
159
14,789
William Hu retweeted
We’ve seen an explosion of new ML chips with unique architectural features, but software support remains the critical bottleneck Achieving peak performance increasingly relies on hardware-specific optimizations in the kernels, but we observe that coding agents are particularly weak at this Introducing Hawkeye, a framework that brings hardware-awareness to coding agents by grounding them in a minimal and comprehensive taxonomy of optimization strategies For new GPU or ML accelerator architectures, you only need to write 10 unit tests and solution kernels (one per optimization strategy), and we show that coding agents can effectively scale test-time compute with this minimal supervision to write hardware-aware kernels Hawkeye can port kernels across architectures (Ampere, Hopper, Blackwell), vendors (NVIDIA, AMD), and precisions (FP8, NVFP4, MXFP4) while consistently leveraging hardware features and approaching expert kernel performance Work co-led with @keramakr and done in collaboration with Alexander Ingare @simonguozirui @18jeffreyma @ZishenW @simran_s_arora @Azaliamirh @profvjreddi
19
55
10
339
54,954
William Hu retweeted
Big, big appreciation to the SGLang team — playing a major part in our ability to stand up great performant endpoints on day-0, model after model. Open source is a joyous place to collaborate, thanks to these folks.
SGLang day-0 speed on Kimi K3: 423 tok/s (measured on gsm8k), plus RL support ready in Miles @radixark! How the largest open-source model runs this fast: we natively implemented and deeply optimized K3’s new architecture with fused KDA decode kernels, DP attention, DSpark, PD disagg, and KDA-aware prefix caching. We've passed Kimi Vendor Verifier and are ready for production! Thanks to @Kimi_Moonshot, @nvidia, @AMD, @KVCache_AI, @modal, and @baseten for building this with us, and to @googlecloud, @nebiustf, @fal, @digitalocean, @runpod, @DeepInfra and @gmi_cloud for serving K3 on SGLang. Blog, cookbook, benchmarks in the comments. P.S. This demo video? Kimi K3 made it itself. Play the game 👇
3
7
3
59
11,379
William Hu retweeted
Frontier models require frontier speculators. Our team moved mountains to achieve 460 tok/s with our custom DFlash model. Try it day 0 at modal.com/endpoints.
Kimi K3 is live on Modal. Moonshot has shipped the world's first open 3T-class model, and we're a day zero launch partner. We trained a custom DFlash speculator for K3's novel architecture so you can run it faster, losslessly. The most capable open model we've worked with by far.
19
31
12
423
53,982
William Hu retweeted
Kimi K3 is live on Modal. Moonshot has shipped the world's first open 3T-class model, and we're a day zero launch partner. We trained a custom DFlash speculator for K3's novel architecture so you can run it faster, losslessly. The most capable open model we've worked with by far.
34
58
30
672
375,173
Try out the speculator at huggingface.co/modal-labs/In… ⚡️
Inkling by @thinkymachines is now available on Modal, backed by a custom DFlash speculator for 67% higher throughput and interactivity. Running on Modal Auto Endpoints with SGLang today.
1
1
23
1,939
Modal trained a DFlash speculator that's much faster than MTP, making it a great boost for inference speeds!
Inkling by @thinkymachines is now available on Modal, backed by a custom DFlash speculator for 67% higher throughput and interactivity. Running on Modal Auto Endpoints with SGLang today.
14
44
4
445
60,223
William Hu retweeted
Our new Auto Endpoints feature is powered by a new Modal primitive: Modal Servers. In this blogpost, we walk through design principles and detailed architecture: @EnvoyProxy, @googlecloud Spanner config store, and a @Cloudflare Pingora-based custom proxy.
5
12
8
84
47,306
Open code and open weights on a platform with unparalleled observability, autoscaling, and reliability :)
It is not too late to _actually_ own your inference. Introducing: Modal Auto Endpoints.
1
14
928
William Hu retweeted
Happy to share new ThunderKittens attention kernels for B300 GPUs -- faster than FA4! Check it out:
Excited to share new ThunderKittens attention kernels that match or outperform Flash Attention 4 on Blackwell GPUs! Currently only supports QK192/V128 shapes, but more coming soon. Check out the code here: github.com/HazyResearch/Thun… Shoutout to the FA4 team for the algorithmic innovations and to @stuart_sul for the helpful discussions.
2
12
1
151
15,654
Modal's been an amazing abstraction to work with! Excited to be building it out more! 🚀
Fresh blog post! @modal partnered with @ScalingIntelLab, @HazyResearch, and @chelseabfinn's IRIS Lab to speed up research on speeding up AI research. Read how scientists at the cutting edge are building the machines that build the machines with Modal. modal.com/blog/accelerating-…
2
2
11
2,171
Announcing Flapping Airplanes! We’ve raised $180M from GV, Sequoia, and Index to assemble a new guard in AI: one that imagines a world where models can think at human level without ingesting half the internet.
337
255
239
3,585
2,259,541
William Hu retweeted
Much like the switch in 2025 from language models to reasoning models, we think 2026 will be all about the switch to Recursive Language Models (RLMs). It turns out that models can be far more powerful if you allow them to treat *their own prompts* as an object in an external environment, which they understand and manipulate by writing code that invokes LLMs! Our full paper on RLMs is now available—with much more expansive experiments compared to our initial blogpost from October 2025! arxiv.org/pdf/2512.24601
247
1,072
262
7,290
2,035,377
William Hu retweeted
Happy 🦃 Thanksgiving weekend! 🍂 This year, we cooked up a new recipe for juicy fact-storing MLPs. Instead of picking apart trained models, we asked: Can we construct fact-storing MLPs from scratch? 🤔 Spoiler: we can & we figured out how to slot these hand-crafted MLPs into Transformer blocks as modular fact stores! 🧩 New work with @garctrob @ronnygjunkins @jerrywliu @dylan_zinsley @EyubogluSabri Atri Rudra @HazyResearch! 🧵👇
8
47
14
336
66,081
Super excited for ParallelKittens led by @stuart_sul! From the Nvidia A100 to the B200, BF16 tensor core performance improved by 7.2× and High Bandwidth Memory bandwidth by 5.1×, while intra-node communication (NVLink) improved by only 3× and inter-node (PCIe/InfiniBand) by just 2×. PK helps with multi-gpu kernels!
(1/6) GPU networking is the remaining AI efficiency bottleneck, and the underlying hardware is changing fast! We’re happy to release ParallelKittens, an update to ThunderKittens that lets you easily write fast computation-communication overlapped multi-GPU kernels, along with new kernels for data, tensor, sequence, and expert parallelism! Here’s a photo of overlapped kittens, along with things you should care about when optimizing multi-GPU kernels. (With @simran_s_arora, @spectorb, and @hazyresearch. Generously supported by @cursor_ai and @togethercompute)
4
22
192
33,537