Founding SVP of Research @togethercompute Assistant Professor @ucsd_cse Looking for talented kernel engineers and performance engineers!

Joined September 2019
Excited to share that I will be joining UCSD CSE as an assistant professor in January 2026! I'll be recruiting PhD students from the 2024 application pool - if you're interested in anything ML Sys/efficiency/etc please reach out & put my name on your application! Until then I'll be finishing up some requirements at Stanford (long story...) and hanging out at @togethercompute. Stay tuned for more!
48
40
8
580
118,138
Dan Fu retweeted
We now have a platform for SFT & RL. This is why open weight models are great
Together Post-Training is live Turn open models into custom models you own, with SFT or RL. You define the data, rewards, environments and the training loop itself. We run the hard infrastructure underneath it, tuned for quality and speed.
11
1
171
9,812
Great work from @KumbongHermann, check it out!
🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation. Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves: ⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality. 🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens. 🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps. With @ishanskhare, @IddahMlauzi, @ekellbuch, @simran_s_arora, @Azaliamirh, and @HazyResearch. More details in the thread 🧵👇
2
1
23
2,782
Next YC Paper Club: Inference Inference is no longer just: y = model.forward(x) It has now become as complex and important as training. The difference between naive inference and well tuned could be 100X in latency and cost. So we gather some of the best researchers on SOTA inference to discuss this in depth. Wednesday, 10.21, 5-9pm, MV Speaker List: - Dan Fu (Together AI) @realDanFu - Jeffrey Morgan (ollama) @ollama - Steven Arellano (Wafer) @wafer_ai - TBD from SGLang/RadixArk @sgl_project @radixark Sign up here: events.ycombinator.com/yc-pa… @ycombinator
17
19
1
320
24,308
Come chat with @hayden_prairie about Parcae or anything else looped models at COLM this week!
In SF this week. Feel free to reach out if you want to chat. Will present Parcae at Poster Session 1 #44
8
1,072
In SF this week. Feel free to reach out if you want to chat. Will present Parcae at Poster Session 1 #44
Parcae has been accepted to COLM! Feel free to reach out if you want to chat in SF!
6
1
20
2,056
Great work @nutlope and team building this, it's a joy to use!
Announcing Together Link! Run frontier open models in your favorite coding harness. Track your spending and let our Auto router pick the model for you: low-cost models for quick fixes, more capable models for harder tasks.
3
1
14
1,948
Announcing Together Link! Run frontier open models in your favorite coding harness. Track your spending and let our Auto router pick the model for you: low-cost models for quick fixes, more capable models for harder tasks.
13
26
12
145
31,482
Super excited to get early access to Vera Rubin to write some kernels and play with the new tensor cores! Enjoy these astronomer thunderkittens looking into the stars and check out the write up of new features below⚡️🐱💫
thunderkittens is now running on nvidia vera rubin! our kernels team got early access to nvl72 and spent the past few days digging through the new isa and bringing nvfp4 + fp8 gemms to life after reworking the kernels for rubin, we pushed them past 22 and 12 pflops respectively — competitive with cublas + cute dsl
3
2
26
2,445
thunderkittens is now running on nvidia vera rubin! our kernels team got early access to nvl72 and spent the past few days digging through the new isa and bringing nvfp4 + fp8 gemms to life after reworking the kernels for rubin, we pushed them past 22 and 12 pflops respectively — competitive with cublas + cute dsl
13
19
7
137
24,572
Really cool to see folks train looped models at scale! @hayden_prairie has some great work here on the fundamental architecture choices in looped models and how to make them stable (+ higher quality). More soon on the inference-time memory implications :)
It's exciting to see the rumors that Astra is a looped model. For those interested in the math on how to stably train them, I would check out our work Parcae. We also show that looping follows predictable scaling laws! arxiv.org/pdf/2604.12946
2
1
26
4,417
It's exciting to see the rumors that Astra is a looped model. For those interested in the math on how to stably train them, I would check out our work Parcae. We also show that looping follows predictable scaling laws! arxiv.org/pdf/2604.12946
new: OpenAI & others quietly using loop transformers that don't show their 'thinking' when scaled up a leap forward on performance, but sparking concerns inside & outside OpenAI re: security as this takes off
3
11
1
54
10,616
This is a really cool demo I've wanted to see for a while - cool to see the models + systems develop to really make a compelling demo ("realtime" video has been possible since 2024 with OpenSora on 8xH100, but quality was not good enough to be compelling) Great work @haoailab!
Wow, the community is moving so fast. @reactorworld just developed an infinite video stream on Twitch using FastH3, and it looks so fun! It is open source!! Thank you @reactorworld for the love with OSS 🥰🥳 code and models here: github.com/reactor-team/infi… haoailab.com/blogs/fasth3-pr…
2
1
18
4,490
Excited to lead the way on quality serving for K3!
.@Kimi_Moonshot benchmarked K3 endpoints across major inference providers. Together AI leads or ties for #1 on 3 of 4 benchmarks: OCRBench, MMMU Pro Vision, and DeepSWE. Open models like Kimi K3 have gotten genuinely good. Frontier-class good. Which makes serving them well essential. Proud of our Research team for holding the bar this high, and proud to see it verified by Moonshot. Run Kimi K3 on Together AI: togetherai.link/k3-x
1
1
26
5,829
.@Kimi_Moonshot benchmarked K3 endpoints across major inference providers. Together AI leads or ties for #1 on 3 of 4 benchmarks: OCRBench, MMMU Pro Vision, and DeepSWE. Open models like Kimi K3 have gotten genuinely good. Frontier-class good. Which makes serving them well essential. Proud of our Research team for holding the bar this high, and proud to see it verified by Moonshot. Run Kimi K3 on Together AI: togetherai.link/k3-x
11
10
5
55
14,802
Great stuff from @stuart_sul! Software is dead, really excited by what the new world brings.
(1/6) If a prompt can generate every layer of the stack, what are programming abstractions worth? GPU programming is at a weird inflection point. For decades we stacked layers of abstraction to ease our cognitive load. This year, we find ourselves constantly deleting them.
1
9
2,431
(1/6) If a prompt can generate every layer of the stack, what are programming abstractions worth? GPU programming is at a weird inflection point. For decades we stacked layers of abstraction to ease our cognitive load. This year, we find ourselves constantly deleting them.
🤖 Made with AI
9
32
6
188
18,833
It has been clear to many of us, and now it’s becoming clear more tangibly, that AI models will commoditize to various degrees. This is of course a difficult business reality if your core business depends on exclusivity on intelligence. But commodity markets are not communism. They are the largest markets on earth. Oil, grain, steel, electricity, memory: trillions clear through them every year, priced by competition among thousands of suppliers. In economic terms, communism is one provider and no price. A commodity market is the precise inverse. The world that actually resembles central planning is the one Dean argues for: a set of protected incumbents, access gated by the state, agencies instructed to manufacture FUD until every regulated buyer, and transitively every tool maker upstream, backs away from cheaper competitors. Open weights don't deter capex. They move it. When the model layer commoditizes, spend shifts to inference, data, tooling, and applications, and builds far broader industrial infrastructure rather than concentrating capital in a handful of companies. Most of our digital infrastructure today, hyperscalers included, runs on open source. The businesses built atop it keep excellent margins and compound at extraordinary rates. Open-weights intelligence will likely rank among the most important economic accelerations in history. It won't be kind to every early incumbent, Linux wasn't kind to Sun Microsystems, but it will be very good for almost everyone else. I suspect OpenAI and Anthropic, given their positions, excellent products, resources and talent density, will be just fine. They will simply hold a little less pricing power. The security theater around Mythos continues to do damage. Of course, there is no evidence for the hysterical claims. The evidence is in fact so thin that proponents of AI's existential risks now openly recommend FUD as the strategy. That should be telling.
Some observations on Kimi: 1. It's a very good model! I don't think its performance can be explained away by distillation or anything like that. In agentic coding sessions, it seems pretty much on par with the best public models of Q1 2026. In my fairly limited use, it also seemed very token hungry. It's not obvious to me that this model is actually that cheap to run. 2. I am personally surprised the Chinese state continues to allow the open sourcing of models this good, given potential risks. To be clear, I *myself* might be fine with models presenting this level of marginal risk being open weight, but I am surprised that China is fine with it. I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI). The other 25% or so is their lack of compute for customer inference (making China's open-weight strategy an unintended byproduct of US export controls) and the normal Chinese strategy of aggressive exports. For the companies, as opposed to the government, the decision to open source is partially ideological and partially because they are behind, and they know that very few people would pay for sub-frontier models from China. 3. Open-weight models are inherently decelerationist, and I'm continually surprised to see the so-called "accelerationists" so excited about open-weight models. I suspect the reason they are is that they know open-weight models are effectively ungovernable, and they simply like the overall cloak of ungovernability open-weight models create over the whole of AI. It's not a bad strategy; it reminds me of James Scott's recounting of the hill people in "the art of not being governed." Still, in the end, open-weight models deter further AI capex. 4. One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a "public good" which will ultimately be provided by the state as a kind of "digital public infrastructure." This future strikes me as a dystopian hellscape, but I've never met an open-weight models advocate who doesn't ultimately concede this is where things end. You'd be surprised how many 'accelerationists' lobbied me, while I was in government, to support an eleven or twelve-figure federally funded data center so that startups could train models at a subsidy and then give them away for free. There was no other way for AI to progress, they said. Perhaps this is the logical end state of things. Nonetheless, I find myself surprised to see supposed accelerationists excited about such an outcome. I think many of them just don't know what they're doing. Many accelerationists do not view the creation and serving of frontier models as a legitimate business. 5. I would guess that the Trump Administration will at some point realize that their best strategy here would be to create large amounts of regulatory risk around the use of open-weight Chinese models. You don't need to "ban open source" (one of the dumber motifs of AI policy discussion). You just need to direct every agency to issue soft law that creates FUD. "A Federal Reserve Advisory Bulletin found that there may be backdoors in Chinese AI models." It needn't be that well justified. You just create enough regulatory risk that every regulated enterprise backs off. You probably don't want to create so much regulatory risk that you scare off the hyperscalers from serving Chinese models; this will just drive startups to sketchier providers. There's a happy middle ground here. I'd assume they will do some version of this. 6. It's probably true that open-weight models of this capability make the world a bit more dangerous, but not so much more that you'll really notice. At some point the models will be capable enough that you will notice. "A nonliving, invisible, dangerous, and infinitely self-replicating agent escaped from a Chinese lab," you say? Color me shocked.
30
72
8
413
67,697