life update: I've recently left @Kimi_Moonshot! The past year at Kimi has been the best of my life so far. I'm deeply grateful to have worked with and learned from such amazing and supportive colleagues. Some of my favorite memories include (in chronological order): optimizing KDA's kernels and trying every design possible with @yzhang_cs; coming up with and refining Attention Residuals with @yzhang_cs @Jianlin_S; working on optimizers, which by far taught me the most about training dynamics (and where I pulled the most all-nighters); creating the cases for K3's release with @yzhang_cs @Xinyu2ML while watching the World Cup; and more, and more. Quoting my earlier tweet about Kimi K3’s release: it is truly incredible to have built such an open (and strong) model with a group of genuine people. My days at Kimi are memories I'll forever cherish... thanks for everything! (especially @Yulun_Du for believing in and hiring a 16yo back then :P). More things to explore :) 🫡