aaron retweeted
Hard to believe how bad this piece is; each answer contains at least one straightforward falsehood or fallacy
Michael Jordan (the machine learning one): "Superintelligence is Science Fiction" 🔥🔥🔥
di.ens.fr/~fbach/MJordan_ITW…
they dunno but i've been a nerdfighter since 2012
youtube.com/watch?v=jWHunRLx…
aaron retweeted
🚀 More reliable agents with DeepSeek V4.1!
XGrammar brings strict tool calling to SGLang & vLLM through Structural Tags, enforcing tool argument schemas in DeepSeek’s native format.
See how the Structural Tag works 👇
blog.mlc.ai/2026/05/04/xgram…
Check out XGrammar 👇
github.com/mlc-ai/xgrammar
aaron retweeted
my hot take is specialized inference engines are amateur projects to spend the spare tokens when @thsottiaux gives people one more reset😄
an inference engine is way more than just running a model on a certain hardware, it is an inference ecosystem.
I use this slide several times when I talk to people about vLLM. Models, hardwares, and inference techniques, all of them move quickly. And vLLM lies in the intersection to provide a unified interface to end-users and applications. vLLM is the inference ecosystem.
vLLM not only support current models and hardwares, but we are also working to support new models and hardwares coming in the next few months. It's an ecosystem people can trust and rely on.
Over the past 3 years, I have seen so many projects claiming to be better than vLLM in certain aspects, but in the end either their techniques are contributed to vLLM or they disappear. That's the power of ecosystem.
A specific example would be tilert github.com/tile-ai/TileRT , a megakernel inference engine dedicated for decode. They collaborate with vLLM by using vLLM prefill + tilert decode in vllm.ai/blog/2026-07-14-vllm… , as highlighted in newsletter.semianalysis.com/… from @SemiAnalysis_ .
aaron retweeted
i do think a lot of people on the pro-open-source side are having a bit of a knee-jerk reaction to the pacing statements today, as we're used to viewing the closed labs as power-seeking.
but i think their hands are somewhat forced here, and this is just another chapter on the fairly inevitable path towards decently-fast decently-safe decently-commoditized intelligence abundance.
ask any F500 exec or swing voter. the world doesn't really want super-fast-takeoff superintelligence owned only by two companies, and thus we won't get it. it'll happen at the pace that the world can accommodate it, which means reaching some sort of confidence consensus that the models are aligned enough that we won't be dealing with scary new incidents all the time.
this will trickle out broadly, in the form of best practices and distillation. the smarter a model is, the more it has a "personality", and the less effective strict rules are. there will be awkward compromises and moral tensions. but we ultimately just want the models to be reasonable, and to do the sorts of things reasonable humans would do if our brains were faster and less error-prone and had more working memory. i think we'll get there.
the labs will build mac and windows, the rest of us are building linux. everyone's gonna do great. weird stuff will keep happening, but we'll still wake up and go to work, until the work does itself in a manner the world finds acceptable.
Can I say something without anyone getting mad?
you’ll get mad at me for saying this…but cloud gaming is so obviously more economically efficient than physical hardware I think it’s going to be the default soon.
your home console / pc is idle 90%+ of the day. meanwhile, data centers targets what, 5%, maybe at worst 10% idle.
every second a cloud gamer isn’t gaming, that hardware is being used for someone else, training, etc.
I think there should be a new measurement, something like cost-per effective FLOP hour that takes into account the TCO + effective utilization.
If a gamer spends $500 on a GPU, uses it for 3 years, but it’s only fully active ~5% of that period…the cost-per relative FLOP hour is crazy high! Meanwhile, a $50,000 datacenter GPU might have a *LOWER* cost-per FLOP hour just because the effective utilization is 90+%.






















