Limor E retweeted
If I’m not mistaken, this is the first non-Google model served in production on TPUs?
We’re running GLM-5.3 Flash on TPU7x with an optimized SGLang-JAX stack. Performance is already pretty good, but getting TPU inference competitive in production takes a lot of work, here's one example we wrote about:
zro.moonmath.ai/posts/making…
Our work builds on contributions from many talented engineers at Google, RadixArk and across the SGLang community, to whom we’re grateful.
We’ve also started upstreaming our kernels. We'll serve more models soon, just waiting for Google to approve more slices!
We are now serving GLM5.3 Flash on TPU7x.
Try it:
npm install -g @moonmath-ai/zro
zro login --api-key APIKEY
zro launch opencode --model glm-5.3-flash
Limor E retweeted
ALERT ALERT ALERT 🚨 🚨 🚨 VLLM MAINTAINERS HAVE JUST SHOWN THAT TPUv7 CAN GET 700 tok/s/user, 56% BETTER PERFORMANCE THAN NVIDIA GB200 NVL72 THROUGH MEGAKERNEL OPTIMIZATION ON KIMI K3.
As we said awhile ago, the TPU externalization of software is full steam ahead. This is ultra important to follow the progress of this.





