@dylan__lim

research @togethercompute | prev. @stanford @jumptrading @hazyresearch

 Stanford, CA
dylan-lim.com/
Joined April 2024
  • Tweets18
  • Following508
  • Followers333
1Media
One kernel. One llama. Mega results. Proud to share our fully fused Llama-1B megakernel! Check out the code and blog below!
Benjamin F Spector
@spectorb27 May 2025
(1/5) We’ve never enjoyed watching people chop Llamas into tiny pieces. So, we’re excited to be releasing our Low-Latency-Llama Megakernel! We run the whole forward pass in single kernel. Megakernels are faster & more humane. Here’s how to treat your Llamas ethically: (Joint with @jordanjuravsky, @stuart_sul, @OwenDugan, @dylan__lim, @realDanFu, @simran_s_arora, and @HazyResearch)
1
1
19
2,970
One of the best blogs out there on writing AMD kernels!
Simran Arora
@simran_s_aroraSep 21
we got early access and hipkittens is now running on amd helios mi455! here we share an educational gemm ladder to demonstrate how to use the new helios features. overall, we found that the patterns identified in hipkittens for writing performant mi350/355 kernels translate well to helios making the forward port quite smooth!
1
11
1,237
Getting prepped for producing tokens on Vera Rubins! We introduced VR support in ThunderKittens today. together.ai/blog/to-infinity…

To Infinity and Beyond: ThunderKittens Now on NVIDIA Vera Rubin NVL72!

We ported ThunderKittens to NVIDIA's Vera Rubin NVL72 and rebuilt our NVFP4 GEMM around the new hardware, taking it from 42% of roofline to over 22 PFLOPS — competitive with cuBLAS and CuTe DSL. Here...

together.ai
1
7
30
2,127
Dylan Lim retweeted
Dan Fu
@realDanFuSep 10
Super excited to get early access to Vera Rubin to write some kernels and play with the new tensor cores! Enjoy these astronomer thunderkittens looking into the stars and check out the write up of new features below⚡️🐱💫
Together AI
@togethercomputeSep 10
thunderkittens is now running on nvidia vera rubin! our kernels team got early access to nvl72 and spent the past few days digging through the new isa and bringing nvfp4 + fp8 gemms to life after reworking the kernels for rubin, we pushed them past 22 and 12 pflops respectively — competitive with cublas + cute dsl
3
2
26
2,448
thunderkittens is now running on nvidia vera rubin! was a smooth porting experience with some fun new updates -- checkout the blog! great work by @dylan__lim, Xinyi and the @togethercompute team!
Together AI
@togethercomputeSep 10
thunderkittens is now running on nvidia vera rubin! our kernels team got early access to nvl72 and spent the past few days digging through the new isa and bringing nvfp4 + fp8 gemms to life after reworking the kernels for rubin, we pushed them past 22 and 12 pflops respectively — competitive with cublas + cute dsl
3
17
3
121
9,999
ThunderKittens is now on Vera Rubin! It's been a fun past week playing with new kernels and hardware. Check out the blog below for more details on how we pushed our GEMMs to cuBLAS levels of performance! 🐱⚡️🔭
Together AI
@togethercomputeSep 10
thunderkittens is now running on nvidia vera rubin! our kernels team got early access to nvl72 and spent the past few days digging through the new isa and bringing nvfp4 + fp8 gemms to life after reworking the kernels for rubin, we pushed them past 22 and 12 pflops respectively — competitive with cublas + cute dsl
5
1
34
4,025
We @togethercompute believe intelligence should be abundant, not expensive. Today we announced our Series C funding of $800m @ $8.3B valuation, to continue to build the world's most efficient platform for generative AI. Thanks @nikogallogly for telling our story in @nytimes! shorturl.at/SooOP
67
87
36
488
176,057
Dylan Lim retweeted
Stuart Sul
@stuart_sulFeb 20
(1/7) We're releasing ThunderKittens 2.0! Faster kernels, cleaner code, industry contributions, and new state-of-the-art BF16 / MXFP8 / NVFP4 GEMMs that match or surpass cuBLAS! Alongside this release, we’re equally excited to share some insights we learned while squeezing every last TFLOP out of Blackwell: (with @hazyresearch & generously supported by @cursor_ai)
13
89
10
540
65,840
Announcing Flapping Airplanes! We’ve raised $180M from GV, Sequoia, and Index to assemble a new guard in AI: one that imagines a world where models can think at human level without ingesting half the internet.
337
255
239
3,572
2,242,725
(1/6) GPU networking is the remaining AI efficiency bottleneck, and the underlying hardware is changing fast! We’re happy to release ParallelKittens, an update to ThunderKittens that lets you easily write fast computation-communication overlapped multi-GPU kernels, along with new kernels for data, tensor, sequence, and expert parallelism! Here’s a photo of overlapped kittens, along with things you should care about when optimizing multi-GPU kernels. (With @simran_s_arora, @spectorb, and @hazyresearch. Generously supported by @cursor_ai and @togethercompute)
9
61
9
514
156,617
Megakernels continue with our 8-GPU Llama-70B release! Please check out below!
Benjamin F Spector
@spectorb28 Sep 2025
(1/8) We’re releasing an 8-GPU Llama-70B inference engine megakernel! Our megakernel supports arbitrary batch sizes, mixed prefill+decode, a paged KV cache, instruction pipelining, dynamic scheduling, interleaved communication, and more! On ShareGPT it’s 22% faster than SGLang.
7
659
Happy to announce that multi-GPU ThunderKittens is finally here! Help your GPU's meow better by checking out the following blog!
Stuart Sul
@stuart_sul22 Sep 2025
(1/6) We’re happy to share that ThunderKittens now supports writing multi-GPU kernels, with the same programming model and full compatibility with PyTorch + torchrun. We’re also releasing collective ops and fused multi-GPU GEMM kernels, up to 2.6x faster than PyTorch + NCCL. (Joint with @dylan__lim, @spectorb, and @HazyResearch. Generously supported by @cursor_ai)
6
753
Happy Throughput Thursday! We’re excited to release Tokasaurus: an LLM inference engine designed from the ground up for high-throughput workloads with large and small models. (Joint work with @achakravarthy01, @ryansehrlich, @EyubogluSabri, @brad19brown, @jshetaye, @HazyResearch, and @Azaliamirh)
7
48
13
206
47,868
So so so cool. Llama 1B batch one inference in one single CUDA kernel, deleting synchronization boundaries imposed by breaking the computation into a series of kernels called in sequence. The *optimal* orchestration of compute and memory is only achievable in this way.
Benjamin F Spector
@spectorb27 May 2025
(1/5) We’ve never enjoyed watching people chop Llamas into tiny pieces. So, we’re excited to be releasing our Low-Latency-Llama Megakernel! We run the whole forward pass in single kernel. Megakernels are faster & more humane. Here’s how to treat your Llamas ethically: (Joint with @jordanjuravsky, @stuart_sul, @OwenDugan, @dylan__lim, @realDanFu, @simran_s_arora, and @HazyResearch)
62
228
8
1,980
268,739
Excited to share LayoutVLM—leveraging VLMs for spatial reasoning in 3D layout generation!
Sun, Fan-Yun@sunfanyun18 Mar 2025
Spatial reasoning is a major challenge for the foundation models today, even in simple tasks like arranging objects in 3D space. #CVPR2025 Introducing LayoutVLM, a differentiable optimization framework that uses VLM to spatially reason about diverse scene layouts from unlabeled assets and open-ended language instructions 1/n
1
713
Had a super fun time building this out - always love working on distributed ML systems. Big thanks to @pearvc for awarding us the best startup prize at Stanford TreeHacks!
Aksh Garg
@AkshGarg0315 May 2024
(1/5) @CKT_Conner, @dill_pkl, @emilyzsh, and I are excited to introduce Shard - a proof-of-concept for an infinitely scalable distributed system composed of consumer hardware for training and running ML models! Features: - Data + Pipeline Parallel for handling arbitrarily large models - Algorithmic load balancing for throughput optimization - Fault tolerance for unreliable machines
1
1
11
2,393
Load more