To infinity and Beyond
Michigan, USA
Joined November 2021
- Tweets7.2K
- Following595
- Followers63
- Likes11.9K
Ventus retweeted
✅ Nathan Chen (@nathancgy4) has joined Discovery Loop.
life update: I've recently left @Kimi_Moonshot!
The past year at Kimi has been the best of my life so far. I'm deeply grateful to have worked with and learned from such amazing and supportive colleagues.
Some of my favorite memories include (in chronological order): optimizing KDA's kernels and trying every design possible with @yzhang_cs; coming up with and refining Attention Residuals with @yzhang_cs @Jianlin_S; working on optimizers, which by far taught me the most about training dynamics (and where I pulled the most all-nighters); creating the cases for K3's release with @yzhang_cs @Xinyu2ML while watching the World Cup; and more, and more.
Quoting my earlier tweet about Kimi K3’s release: it is truly incredible to have built such an open (and strong) model with a group of genuine people.
My days at Kimi are memories I'll forever cherish... thanks for everything! (especially @Yulun_Du for believing in and hiring a 16yo back then :P).
More things to explore :) 🫡
Ventus retweeted
DeepSeek just open sourced its high-performance, large-scale inference framework DeepInfer in Apache-2.0 license.
github.com/deepseek-ai/DeepI…
Supported features:
- Optimized kernels, model and sampler runtime
- Parallelism: Attention DP / balanced CP, MoE EP + EPLB
- DSpark and confidence-scheduled verification
- SWA bounded replay
- Asynchronous scheduling (zero-overhead batch scheduler)
- Batch-invariant inference
- Deterministic sampling
- Structured output
Ventus retweeted
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated.
Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
Ventus retweeted
A BIG THANK YOU to our legendary friend, @ElonMusk, for his incredible generosity in giving SpaceX Stock to millions of American children through TRUMP ACCOUNTS! This is exactly what was envisioned when we created TRUMP ACCOUNTS — Bringing together the GREAT PRIVATE SECTOR, Elon and so many more, to invest in our children, by far our greatest and most important asset, and give every American Child a piece of the American Dream. Thank you Elon! President DONALD J. TRUMP
Ventus retweeted
🎯⬇️ Jen-Hsun Huang, CEO Nvidia, trifft bei seinem aktuellen Besuch in Berlin den Nagel auf den Kopf: "Nichts schmerzt mehr, als eines der technologisch fortschrittlichsten Länder so abgehängt zu sehen."
Ventus retweeted
Old friends are the best friends. Jensen rocks.
The Star Mind computer being co-engineered by SpaceX and Nvidia is far better than almost anyone expects and will reach volume production next year.
Working with the incredibly talented Nvidia engineering team is pure joy!
Jensen Huang says he loves working with Elon Musk, and reveals their engineering teams are designing a new type of server to put data centers in space
“I’m delighted that I love working with Elon. He’s using our technology for his own AI in training models and such, deploying models, of course."
"Grok Bot is super cool. The demand for Grok Bot, the consumption for Grok Bot is really skyrocketing. And then, of course, he rents it."
"And so the combination of these three things, he’s fairly unique in doing that. I guess Google does that. That would be yet another example."
"And so I think he also has the advantage that he loves the intersection between computing and the physical buildouts. He loves that whole area. Nobody knows more about energy than he does."
"One particular new market that we’re going to go explore together is putting data centers out in space. Our two engineering teams are working super closely together to create a new type of server, a new type of architecture that might be super good in space and on Earth.”
Ventus retweeted
V4.1 Flash 1017 甚至记得 codex 的源码，能感受出来大概重训了很新的知识库 🫡
大肥鱼能飞，midtrain 能飞，RL 也能飞
mp.weixin.qq.com/s/X-5kbBked…
Ventus retweeted
🚀 DeepSeek-V4.1-Flash-1017 is here!
Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API.
🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model.
1/4
Ventus retweeted
China’s dependency on oil is dropping fast
Ventus retweeted
the most interesting paper I read this month.
sparsity comes from on-policy.
(which matches my intuition but this research explains why)
Ventus retweeted
life update: I've recently left @Kimi_Moonshot!
The past year at Kimi has been the best of my life so far. I'm deeply grateful to have worked with and learned from such amazing and supportive colleagues.
Some of my favorite memories include (in chronological order): optimizing KDA's kernels and trying every design possible with @yzhang_cs; coming up with and refining Attention Residuals with @yzhang_cs @Jianlin_S; working on optimizers, which by far taught me the most about training dynamics (and where I pulled the most all-nighters); creating the cases for K3's release with @yzhang_cs @Xinyu2ML while watching the World Cup; and more, and more.
Quoting my earlier tweet about Kimi K3’s release: it is truly incredible to have built such an open (and strong) model with a group of genuine people.
My days at Kimi are memories I'll forever cherish... thanks for everything! (especially @Yulun_Du for believing in and hiring a 16yo back then :P).
More things to explore :) 🫡
Ventus retweeted
When China figures out how to manufacture a product, their ability to scale is often greater than the rest of the world combined. We saw this with lithium batteries and solar, for example.
SpaceX is a rare counter-example, where its mass to orbit is roughly an order of magnitude greater than the rest of the world, BUT China now does a majority of non-SpaceX mass to orbit. SpaceX is an anomaly in the matrix.
Maybe RAM will be different, but RAM was a low margin commodity only ~3 years ago. LP5 and LP6 are designed to a common standard that China seems to be solving fast.
Newer HBM architectures and things like an SRAM layer below logic and 3D stacking RAM on top of logic while finding a way to keep the logic cooled will be harder to solve.
