Full stack open superintelligence

San Francisco, CA
Joined March 2021
Zyphra retweeted
Full-stack AI training that scales. Our AMD Data Center team has been working closely with @ZyphraAI to train an advanced reasoning model from scratch. Hear about Zyphra's experience training larger reasoning models more efficiently while supporting longer context windows: bit.ly/4ixfdaB
21
24
3
175
21,780
In Mixture of Expert (MoE) models the cost of moving tokens to their experts can dominate total runtime. At Zyphra research, we use patterns in how tokens are routed to experts to make that communication faster by up to 2.63x on @AMD MI300X GPUs, with the model unchanged.
2
28
4
241
30,657
@ZyphraAI is an open superintelligence research and product company based in San Francisco, CA on a mission to build human-aligned AI that helps individuals and organizations reach their fullest potential. Apply to join us! jobs.ashbyhq.com/zyphra
9
653
Zyphra retweeted
Open models need an open stack. @ZyphraAI VP of AI Engineering Quentin Anthony shares how access to open software libraries and direct collaboration with AMD are helping the team train bigger models, keep growing compute resources working efficiently and pave new ground in AI. Learn how Zyphra trained its ZAYA1-8B reasoning model from scratch on a full-stack AMD platform: bit.ly/4ixfdaB
3
4
1
87
21,029
How do language models develop a sense of time without explicitly encoding position in their attention operations? Zyphra Research explains how local memory layers that read nearby words help global attention layers keep track of word order. 🧵
5
15
6
118
7,729
@ZyphraAI is an open superintelligence research and product company based in San Francisco, CA on a mission to build human-aligned AI that helps individuals and organizations reach their fullest potential. Apply to join us! jobs.ashbyhq.com/zyphra
1
362