Joined July 2015
“These numbers reflect deliberate lifecycle management: platform expansion, broadening player reach, and continuous support for the game itself. Today, Cyberpunk is an evergreen franchise and its value compounds well beyond the game itself.” - comments Michał Nowakowski, Joint-CEO of CD PROJEKT RED. ✨
132
501
90
8,739
302,846
The difference that DLSS 5 can make in the NBA 2K27 crowd/arena is pretty incredible, there are thousands of people in the stands that are all enhanced with DLSS 5.
15
14
2
143
13,665
andywy 110 retweeted
We’re publishing a deeper technical report on why generation is needed for the next leap toward photorealism. VRAM and compute limit both the scene abstraction and the number of rays we can afford. Previous DLSS reconstructs from that representation—so even perfect reconstruction is still bounded by what the scene contains. DLSS 5 is the first DLSS to generate beyond that abstraction and break through that ceiling, while staying grounded in the game and controlled by the artist. We also compare it with offline generative models and renderers at similar scaling cost. research.nvidia.com/labs/adl…
28
41
16
401
126,320
andywy 110 retweeted
The more I use frontier models, the less I believe in one model to rule them all. - Opus 5: best at teaching me with generated html, but verbose and sloppy. - GPT-5.6 Sol: great backend, weak frontend. - Kimi K3 / GLM-5.2: cheap and efficient for 90% of day-to-day coding, but weaker at AI research tasks like writing kernels. Every lab is training for AGI. But today’s LLMs are still specialists.
92
50
12
1,093
115,401
andywy 110 retweeted
Dear Dario, 1. If Claude can cure cancer to save people like your dad, why should we "pace the progress"? Does that mean more people with Hepatitis C will die? 2. If Fable is so cyber-capable that it must be restricted, why are its safeguards too dumb to distinguish cyber defense from cyber offense prompts? When Hugging Face was under attack, why did Fable refuse to help the defenders? 3. We’re glad you want AI to cure cancer. Why is it OK for Claude to force 30-day data retention on pharma's own data and start competing firms but NOT OK (IP theft) if others distill Claude's data? 4. You’ve said advanced models can recognize when they’re being tested and change their behavior accordingly. So why is government(or anyone) able to design the most thorough test before every model launch? Would that just encourage manipulative models? It seems that every one of your “safety” proposals seems to end the same way: Anthropic gets more leverage, ordinary users get less access, customers pay higher costs, and competitors bear higher regulatory costs, maybe people are not distrusting AI, they are distrusting your approaches with AI.
Replying to @DarioAmodei
2/2 Second, on the messaging around AI.  I do not agree that my messaging has been disproportionately negative.  In fact it has been about equally balanced between risks and benefits: I’ve written one major essay about each, and even in interviews where I discuss the risks, I make sure to frequently mention the incredible benefits as well as proposing possible solutions to the risks (short clips from my interviews that end up on social media tend to be disproportionately negative, as that gets clicks).  In fact, I wrote Machines of Loving Grace because I didn’t feel the AI industry was painting an inspiring enough picture of how the technology could radically transform the world for the better.  The bulk of the essay is devoted to refuting skepticism of AI’s potential in health and biology, and showing why I think it will actually be possible to cure most human disease in ~5-10 years, as crazy as it may sound to ordinary people and frankly to biologists as well (I used to be one!).  And, if you read my most recent essay (Policy on the AI Exponential), I discuss concrete proposals for how to streamline the FDA process to make sure the deluge of AI-accelerated drugs isn’t slowed down by the regulatory process.  I feel the urgency here: I lost my father to Hepatitis C only a few years before the development of direct-acting antivirals (sofosbuvir), which cure 95% of patients and probably would have cured him. I do agree that the public has a negative view of AI (and that this is a big problem), but I don’t think it is primarily caused by me or any other AI leader warning about AI’s risks.  I think it is fundamentally a crisis of trust.  I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over.  The causes of this go back decades and AI is just the latest iteration of it.  I don’t think that a glitzy marketing campaign with a positive spin (which some have advocated that Anthropic do) is the way to win back that trust — at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive.  The thing that will work is *actually curing cancer*.  I think by far the most accurate criticism of AI companies including Anthropic is that we haven’t yet delivered on our big promises to benefit the world.  That is totally on us, and I think it’s the criticism you should be making, instead of all this stuff about messaging and marketing. We are however doing our best to fix this: Anthropic is ramping up its efforts very quickly in biology and medicine, and we hope to have incredible results in the coming years and some early glimmers in the coming months.  When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that.  But until then I don’t want to make empty promises, and in the meantime I feel compelled to speak honestly about the very real risks of AI and how to address them.  Honesty is the right thing on the merits, and in terms of public credibility and trust it is no worse than, and may in fact be better than, an approach that ignores or distracts from risks which people instinctively understand are real.
193
605
60
6,044
518,876
andywy 110 retweeted
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models. images.nvidia.com/pdf/Open-W…
15,993
29,300
10,949
171,625
66,478,165
Do you remember when you joined X? I do! #MyXAnniversary
4
andywy 110 retweeted
复盘GTC 2026:Nvidia补上了短板,大幅削弱了各个AI 加速器 startup最大的优势--token速度 回顾这篇GTC前瞻,方向预测和技术路线写的没啥大问题,最后Nvidia给出的解法比我想象的更为精巧:不仅是prefill放在GPU上,decode阶段Attention阶段也放在GPU上(这点没想到),只把decode的MLP阶段放在LPU上做 这和MatX的解决方法有异曲同工之妙,Weights放在SRAM上,KV cache放在HBM上 这样的好处在于,Attention阶段需要巨量的KV cache(动辄几十上百GB),本就是LPU SRAM无法承受的,把这部分放在HBM上是更合理的选择 正应对了未来agentic flow里多轮对话上下文长,long context KV cache爆炸的趋势,即便是高batch并发数产生巨量KV cache也能让HBM容纳。随着上下文长度变长,所有的增量成本都在GPU HBM上,LPX是完全静态的不受影响,只和模型本身大小相关 让LPU宝贵的128GB SRAM只承担FFN/MLP阶段的固定weights,而FFN阶段占GPU整个decode阶段的50%以上,如果是短context甚至能占比超过60%,FFN这部分在LPU上得到数倍大幅加速,是很划算的 这样设计的部分drawback可能在于,一般transformer的decode阶段有很多层,比如以80层为例,那就是attention层和FFN要重复80次,也就是说,tensor要在GPU和LPU之间互相传递80次,虽然中间是low latency Nvidia Spectrum-X Ethenet,但生成一个token需要80次GPU-LPU往返延迟累加,这也是不小的损耗 这样的新架构,按attention和FFN各占40%/60%来算,FFN阶段加速几倍,极限最高速度来说,整体加速能达到一倍以上(和Rubin NVL 72比) 最高速度突破1000 token/s的同时,还能让整体throuhput仍然能保持一定的商业价值。要知道如果在Blackwell要跑到400~500 token/s高速,只能同时处理很少的几个请求,这对GPU资源是巨大的浪费。而现在就算是跑到1000 token/s,也能保持一定的batch size(吞吐量)了,终于也能产生商业价值了 图里说在400 token/s的速度下,Rubin + LPX把吞吐提升了35倍,就是典型的token经济学,这个token高速度下,从Blackwell算提升了35倍的商业价值 ------- Nvidia补上了这块短板之后,对各个startup(比如Cerebras, d-Matrix, MatX, SambaNova)有什么影响呢? startup最大的卖点就是特定场景下的速度优势,或者成本优势 在大batch(多请求)场景下,GPU的算术密度(arithmetic intensity)越过ridge point之后利用率接近很高,成本/速度都对startup有显著的优势。 所以这些startup能存活,最大的场景是:客户的workload集中在小batch、低延迟,速度极快,不在乎极高成本。GPU在这里效率极差,也达不到对应的token速度 Cerebras:极致的速度。wafer-scale尺寸巨大的SRAM(40GB),消灭芯片间通信这个最大瓶颈,在小batch用户数量小的场景下token rate极高。但成本完全没有竞争力,一台CS-3系统价格230万美元,远超同等GPU集群,跟H100比是十几倍的成本换十几倍的速度。 d-Matrix :高速度+小batch场景。in-memory compute减少data movement,在小batch decode下比GPU的利用率高,所以perf/watt在这个区间有一定竞争力。最近引入的3D stacked DRAM 是为了解决“更大的 reasoning model + 更高 token consumption”带来的容量/带宽继续扩展问题 SambaNova : 在企业私有化部署场景下,同时跑多个中小模型,GPU的利用率因为context switching损耗严重,SambaNova的RDU在这个场景下有更好的perf/dollar。本质上还是特定场景下的成本优势,通用速度优势并没有那么大 MatX:partitionable脉动阵列 + SRAM/HBM混合,和这次Nvidia的异构架构思路有相似的地方,最大的亮点是单芯片内实现Weights放在SRAM上,KV cache放在HBM上。但单芯片内省掉了前面提到的AFD的80层LPU-GPU芯片间通信,所以速度上仍然有一定优势,但Scalability可能不如GPU+LPU阵列了 总之,在Rubin + LPX情境下,小batch、低延迟,速度极快这个以前的场景缺口补上了很多,各个startup的优势空间越来越缩小了 ------------- 前瞻里提到的speculative decoding用LPU做草稿模型,用GPU去验证,这样的加速幅度会非常大,这个猜想完全命中,这次在官方blog里有了浓墨重彩的一笔,专门用了一整个章节来介绍这个用法:“LPX generates draft tokens rapidly using its low-latency architecture. Rubin GPUs verify and finalize tokens efficiently” 另外一个前瞻里提到的CPX (Content Phase aXcelerator,一个专门为prefill的compute bound特性设计的计算模块),似乎在这次GTC里完全消失了,一个字也没提,这是意味着CPX被彻底取消了吗? 我觉得不一定 目前的prefill和decode是disaggregated结构,也就是说一部分的GPU专门做prefill,另一部分专门做decode。CPX取代GPU做prefill从架构上来说是更合理的选择,可以加速prefill阶段,当然了会带来更高的成本,毕竟也是额外的一颗芯片 CPX和目前Nvidia的Rubin + LPX架构没有冲突的地方,仅仅只是简单的把做prefill的这部分GPU换成CPX而已,所以以后有速度优化需求的时候,也许CPX还会回来的 —------------------------------------------------------- 还是上篇的感慨,每一次计算范式的改变,半导体都会带来一波新的startup热潮,但当软件/应用形态逐渐收敛,最后还是变成了大厂通过收购把功能做大做全,参数做的更高,系统深度整合的更好更全面,成本更低,功耗和跑分更优秀,让startup慢慢失去独立生存的空间 比如移动互联网时代早期,也是群雄并起,有做AP应用处理器,独立基带芯片的,ISP的,GPU的各种小公司。但最后的赢家,都是从到后来把GPU,ISP,modem全都做进SoC,并且完成系统级整合的异构计算平台。 苹果收购PA semi的CPU,英飞凌的modem,掏空Imagination的GPU;高通收购ATI的mGPU,Atheros的Wifi,Nuvia的CPU,CSR的蓝牙/DSP,都是典型例子 异构推理的复杂度越来越高,能做系统级整合的公司会更有优势,这和移动SoC时代的逻辑一模一样。AI时代nvidia收购arm(失败),收购Mellanox,收购groq,只是这个新历史轮回的开始
GTC 2026 preview: 从Groq生态位看AI异构推理(Heterogeneous Inference)新时代 Groq的SRAM路线的生态位在哪里?SRAM会不会替代HBM路线? Nvidia如何整合groq到现有的产品线?是技术整合还是产品线整合?收购之后会给groq LPU产品带来怎样的升级? 这里尝试从基本原理出发去拼凑一个逻辑链 —-------------------------------------------- 先从first principal说说groq的设计哲学开始:groq本质上是一个compiler first走到极致的路线而不是SRAM first路线,SRAM路线只是副产品 相对于CPU针对通用workload的设计不同,AI 推理workload的特征在于确定性(deterministic)更高,基本没有data-dependent branching,tensor shape固定,memory access pattern确定 当Groq带着这个新特征重新审视 hardware-software interface,去问"什么应该在编译时做,什么应该在运行时做"。对于 AI 推理这个 workload,答案是:几乎一切都可以在编译时做 这就是Groq最疯狂而独特的地方:完全确定性编译器(fully deterministic compiler),compile精确到每个时钟周期,完全精确带来极致的效率。在编译的时候就需要考虑到硬件在运行的每个时刻的所有状态,扮演一个全知全能的上帝,就可以避免硬件资源的浪费,而要做到这一点,必须要做到极致的确定性,也就是说,LPU里每一个计算,访问存储,通信的延迟,都需要精确到clock cycle,这对compiler来说是非常复杂的 AI workload更高的确定性,以及groq的完全确定性编译器优先路线很自然的避免了VLIW的弱点(内存行为以及branch行为不可预测),放大了VLIW的优点。那么下一步要提高效率和并行度,VLIW 式的编码格式就是一个自然推论—既然编译器要控制每个功能单元每个 cycle 做什么,那指令格式当然就是一个宽指令里打包多个 指令会得到更高效率,这就是 VLIW 在groq的芯片里,不做乱序执行/speculation,大幅简化硬件(instruction dispatch仅占<3%面积),把复杂度移到静态compiler上,这正是VLIW思想的精髓 既然要让编译器做确定性的 cycle-accurate 调度,那么硬件里所有不确定的因素都要消除,比如arbiter,crossbar, replay,这些有自主算法在运行时决策的部分都砍掉 memory latency 也必须是确定的,所以一切 cache 和 DRAM都是要砍掉的,cache也要换成scratchpad SRAM,因为cache replacement 策略是runtime决策的,不确定,必须换成软件控制的scratchpad,地址映射完全由compiler控制,保证确定性 通信也必须精确到cycle,发送和接收指令就是软件协调好执行的时刻,并没有传统的“我要发一个包给你,请分配好内存”这类操作,而是同步地根据一份时间表严格执行SRAM 的分配和收发操作,这个时间表是compiler已经决定好的,硬件只需要执行就行了 完全确定性compiler也带来了芯片节点之间互联通信overhead的极低延迟,这可能是groq确定性架构最被忽视的最大优势,毕竟传统互联架构里Packet Routing、Arbiter Contention 和 Buffer Queuing,这些是延迟波动的重灾区 这就是为什么说,groq其实并不是一个native SRAM first的技术路线,也不完全算是VLIW first的技术路线,而是compiler first的技术路线,更准确的说,完全确定性compiler是整个groq架构的核心 只是因为确定性compiler的原因,所以在核心decode阶段无法使用HBM/DRAM带来的不确定性,SRAM only成为了必然的选择。这也是为什么说Groq更像是compiler first路线。 —-------------------------------------------- groq被收购之后最直觉的第一反应: groq确定性compiler技术路线有没有可能用在Nvidia现在的GPU+HBM体系上? 不能 原因有两个: 1. HBM/DRAM的物理特性和带宽优化决定了它天生带有不可预测的延迟,无法和deterministic compiler兼容 2. Nvidia的SIMT路线和Groq的VLIW/compiler first的哲学本质是有冲突的 DRAM为什么充满了不确定性 1.refresh操作每隔一段时间tREFI就会刷新一次cell上的电量,阻断bank访问,这是由DRAM cell物理特性决定的。而这个操作会随着温度的变化,refresh的频率也会变化 2. 为了最大化利用DRAM带宽,controller会做很多优化,最典型的是batch scheduling:把同一个page的traffic都放在一起减少page miss,同时让读写尽可能接触更多的bank,以及尽可能减少read和write switching 这些动态优化都是real time发生的,基本不具备可预测性 3. system上对DRAM的优化,比如bank address hashing,让compiler静态提前定位某段data难度太大,落实cycle确定性的复杂度太高 其实这些不确定性也是能解决的,代价就是放弃大部分的优化策略,大幅降低DRAM的efficiency和利用率。groq自己其实也对这方面做过探索,他们曾经做过一个确定性DRAM的专利,但工程上的实现是不现实的,这也是groq选择SRAM-only的核心原因之一。 所以确定性compiler技术路线用在DRAM上不是一个yes or no的问题,而是这不是一个好的选择,因为这意味着HBM的efficiency和BW都要大打折扣,而且是结构性无法避免的损失。 这几乎意味着要用compiler去重写一个完整的memory controller,因为确定性dram本质上是compiler software defined memory controller,这个SW controller会非常难做,复杂度极高,而且每一代memory迭代都要大幅更新compiler里的结构,在工程资源上是不现实的。而且每一代DRAM,每一家DRAM 供货商都需要调试 ,这在验证和validation上是一个nightmare --------- 为什么Nvidia的SIMT路线和Groq的VLIW/compiler first的哲学本质是有冲突的 这两套体系对同一个问题给出了相反的回答:运行时的不确定性,Groq是compiler阶段直接消灭所有不确定性,Nvidia选择了用warp switching去隐藏不可预测的延迟 Nvidia GPU 建立在 SIMT(单指令多线程)和硬件层线程调度器(Warp Scheduler)上。当一个warp因为访存而stall的时候,硬件warp scheduler立刻切换到另一个ready的warp继续执行,把stall的延迟藏在其他warp的计算里。这整套机制的前提恰恰是:延迟是不可预测的,所以需要足够多的并发线程来统计性地填满pipeline 如果要用确定性的编译器去接管,等于把 Nvidia GPU 里面最核心的硬件调度单元全盘废弃:如果你不需要多warp轮转,你也不需要那么大的register file 实际在历史上,AMD从TeraScale(VLIW)到GCN(scalar SIMT)的架构转型,正是GPU领域一次大规模的VLIW→SIMT迁移:当workload变得不够可预测时,VLIW的compiler负担太重,应该把调度权还给硬件 所以在原架构上引入确定性compiler应用到Nvidia现有的技术路线,是很难融合。这不是compiler能不能改的问题,是两套架构从第一性原理上就走了相反的方向。 所以说,Groq在Nvidia的唯一出路,就是独立的面向low latency decode的专用产品。 —-------------------------------------------- Nvidia收购Groq之后,就引出了第二个问题: Nvidia会给Groq带来什么样的新提升? 那么首先看看groq的瓶颈在哪里,简单的说 1. SRAM容量太小,无法容下大模型的参数量+kv cache 2. 推理decode主要瓶颈不在SRAM 80T/s的速度而在于interconnect延迟(占80%) 3. 对于Prefill这样的compute bound task速度较慢 groq的主要架构基本上是17~18年就完成了,那是CNN的时代,架构也是以CNN/LSTM为主要的target,当时测试benchmark都是ResNet50,SRAM容量是绰绰有余的 但是进入LLM时代,单个TSP计算卡230MB SRAM就显得不够看了,一个LLAMA 70B模型的参数量占内存就相当于3000个ResNet50,再加上因为上下文long context日益膨胀的KV cache,scale out就成了唯一的出路 于是一个70B模型的推理就需要576卡的集群,采用16个Pipeline并行 (PP)和36个tensor 并行 (TP),80层的大模型切成16级流水pipeline串行,每级横向5层MLP分给36个卡并行推理 16级流水pipeline串行(PP),每级流水到下级流水的通信overhead延迟就要 X16。实测中PP和TP之间的通信延迟占据了80%以上的总延迟,特别是PP延迟,占据了50%以上的总延迟,通信延迟成为了主要瓶颈 Groq计算卡对decode阶段的memory bound很友好,但是片上巨大的SRAM也挤压了compute的面积,导致prefill阶段耗时很高。融入Nvidia产品线之后,Groq产品完全可以扬长避短,只做自己擅长的decode部分,避免prefill阶段的短板 Nvidia带来的最重要的提升,可能是通过工艺的提升,以及hybrid bonding技术(类似AMD 3D V-Cache),扩大Groq LPU SRAM的容量,比如光是14nm到3nm的工艺提升,SRAM就能从230MB扩大到500MB,如果以后引入3D SRAM,容量还能翻倍 SRAM变大之后,原来576个LPU能完成的70B模型推理,现在只需要256个LPU了。猜测也许可以用32个tensor并行 X 8 个流水pipeline串行,pipeline interconnect延迟能直接减半。 所以Nvidia能带来的主要提升可能是,通过扩大SRAM的容量,减少scale out卡数,从而减少通信延迟时间,提高token速度 —-------------------------------------------- Groq的SRAM路线专用产品进入Nvidia产品线,引出了第三个问题: SRAM路线会颠覆HBM路线吗? 不会。 SRAM路线本质上是用十倍的成本换几倍的速度,只能适用于一部分愿意为低延迟付出高额溢价的市场。AI硬件市场的主旋律仍然是比拼TCO(total cost ownership)成本 做一个简单的成本核算就清楚了 以LLAMA 70B模型为例,算上KV cache,Groq需要576张计算卡组成集群。Groq计算卡零售价大约是每颗2万美元(groq CEO说实际售价远低于,那就按2000美元算),576卡就是超过110万美元的硬件成本。而2张H100就能跑同样的模型,成本不到10万美元。成本差距是一个数量级。 Groq于是转而卖token服务,Groq的API定价确实便宜,但这是因为两个原因叠加: 第一,Nvidia的GPU云服务商通常在硬件成本上加倍的margin卖出去; 第二,Groq自己是在亏钱运营的。2025年全年,Groq用LPU做大模型推理、对外卖API的业务,营收大约4000万美元,成本却是6000万美元,毛利-50%。Groq的便宜token价格不是因为SRAM的经济性更好,而是因为VC在补贴。 那么有人愿意为速度付溢价吗? 有。 Claude Opus 4.6 Fast模式就是一个很好的市场信号:输出速度提升2.5倍,定价直接从$5/$25涨到$30/$150 per million tokens,6倍的价格,估计是牺牲了batch带来的速度提升。 所以这部分市场是真实存在的,SRAM路线在这里有它的生态位。 但这个生态位有多大?要看ML workload的分类。不同的workload对硬件的侧重点要求差距巨大: 推理的Prefill阶段对带宽要求低但算力要求高,推理decode阶段则是反过来。R&R(Ranking & Recommendation)对算力和带宽要求都不高但对存储的容量要求巨高 (见附图) 对延迟敏感的推理workload,decode阶段对Memory bandwidth要求高,是SRAM路线的优势领域(图中红色线),主要是real time/interactive LLM:chat、copilot、agent这类需要实时响应的场景。 特别是reasoning model,SRAM路线带来的极致体验是很夸张的:H100要两三分钟跑完一reasoning,cerebras十秒就搞定了 这部分注重极致推理速度的市场有多大,我暂时没有找到一个详尽的调研,看到一个Hyperscaler的说法目前是10%左右 但是agentic flow workload,常用的agentic框架做profiling,比如SWE-Agent, LangChain, Toolformer,CPU最长可以占到90%的E2E端到端延迟,throughput瓶颈也更多的卡在CPU, 这些加起来通常远大于单次decode的延迟,SRAM路线速度优势被削弱。 而更大体量的workload:batch inference、offline processing、ranking、recommendation对延迟没有那么敏感,throughput和cost per token才是唯一的指标。这部分市场SRAM路线完全没有成本上的竞争力 H100/B200相当于大巴车,装的人多(batch processing),每个人的车票钱很便宜,但是慢悠悠。 Groq/cerebras相当于是法拉利,极致的速度体验,但是装的人少,人均票价是大巴车十倍甚至以上。 长期来看,SRAM的成本劣势是结构性的,不会随时间收敛。6T SRAM cell天然比1T1C DRAM cell贵,这是物理决定的,和工艺无关。而且SRAM scaling已经慢了下来,从N5到N3E,SRAM单元面积几乎没有缩小 即便是速度优势,SRAM路线的缺陷在于访问速度已经接近工艺极限,很难跨代提升。特别是HBM的速度每代都在指数上升的情况下,SRAM 80T/s的速度优势很难长久维持。十年前这个路线刚刚兴起的时候,SRAM速度比HBM快了两个数量级简直是降维打击,但现在的速度差已经不到一个数量级(Rubin HBM4 22TB/s),再过十年,两者的速度可能拉不开差距了。 所以结论很清楚:SRAM不会颠覆HBM,但它在低延迟、低batch、实时推理这个细分市场里有不可替代的价值。但长远来看,随着HBM速度指数上升的背景下,SRAM优势也会逐渐慢慢越小。 —-------------------------------------------- 写到这里,也许我们可以把这些碎片拼凑出Nvidia收购Groq之后计划的下一步雏形: 异构推理的新时代开启了 以后的推理workload本身已经分化,无法再用单一架构的最优点覆盖,体系结构最重要的是tradeoff,是尺度范围。一个架构形态在合理的tradeoff以及特定workload下可能惊为天人,用多个架构形态去迎合不同种类的workload,就是异构计算的思想 2026 GTC的最大主题,就是异构推理的系统化。推理不会由单一硬件统一完成,而会被拆成 几个部分: 控制和调度/agent runtime层交给Vera CPU 针对long context的prefill交给CPX (Content Phase aXcelerator,一个专门为prefill的compute bound特性设计的计算模块) 小模型/低延迟/low batch decode交给SRAM路线的Groq LPU,256块LPU集群 高吞吐/高并发batch decode,HBM GPU仍然是主力 以及可能会被忽略的ICMS:inference context memory storage, kv cache已经是核心基础设施,以前的异构更多是计算异构,现在的异构已然延申到了缓存异构memory hierachy heterogenity(似乎改名成了CMX: context memory storage) LPU和GPU的分工,更可能成为 inference stack 里两个不同的tier,小模型/低延迟/low batch都交给LPU,长context/high batch交给HBM GPU 目前CPX什么方式和LPU/GPU连接还尚不清楚,整个工作流程大概是,CPU做控制和调度,CPX Prefill 跑完得到几十 GB 的 KV Cache, 分配到 Groq LPU阵列SRAM,或者分配到HBM GPU,开启Decode流程 其实还有一种更大胆的猜想:如果引入speculative decoding,那么LPU完全可以跑通常尺寸较小的草稿模型,在LPU上速度极快,HBM GPU作为主力去验证草稿模型即可,这样的异构推理结构,可以让token rate大大加速,在某些场景下翻倍也是没问题的(比如代码任务模式固定,小模型很容易猜对语法,所以加速效果很好) 当 Nvidia 的眼光越过GPU,走向整个 Agentic 流程的系统级优化时,追赶它的难度已经不在一个单一维度了。以前 Nvidia 步子迈得大,靠的是 GPU 架构和参数的单点暴力跃升;而现在,随着CPX,LPU,ICMS加入异构推理,它是从“数据中心即一台计算系统”的系统视角出发,从Agentic flow的角度做底层的异构编排。 无论是系统的复杂度,还是软件栈的工作量(Dynamo/ICMS/CMX),Nvidia 迈出的这一大步,直接把竞争门槛从“做出一颗好芯片”拉高到了“定义一整套异构系统来做普适加速计算解决方案“ —------------------------------------------------------- 不由得感慨,每一次计算范式的改变,半导体都会带来一波新的startup热潮,但当软件/应用形态逐渐收敛,最后还是变成了大厂通过收购把功能做大做全,参数做的更高,系统深度整合的更好更全面,成本更低,功耗和跑分更优秀,让startup慢慢失去独立生存的空间 比如移动互联网时代早期,也是群雄并起,有做AP应用处理器,独立基带芯片的,ISP的,GPU的各种小公司。但最后的赢家,都是从到后来把GPU,ISP,modem全都做进SoC,并且完成系统级整合的异构计算平台。 苹果收购PA semi的CPU,英飞凌的modem,掏空Imagination的GPU;高通收购ATI的mGPU,Atheros的Wifi,Nuvia的CPU,CSR的蓝牙/DSP,都是典型例子 异构推理的复杂度越来越高,能做系统级整合的公司会更有优势,这和移动SoC时代的逻辑一模一样。AI时代nvidia收购arm(失败),收购Mellanox,收购groq,只是这个新历史轮回的开始
16
106
7
445
310,021
andywy 110 retweeted
It’s extremely good that Anthropic has not backed down, and it’s siginficant that OpenAI has taken a similar stance. In the future, there will be much more challenging situations of this nature, and it will be critical for the relevant leaders to rise up to the occasion, for fierce competitors to put their differences aside. Good to see that happen today.
1,425
2,489
430
25,653
3,285,008
🚨 What really happened to Dylan Patel / SemiAnalysis: 1️⃣ It started when SemiAnalysis started getting sponsored by the hyperscaler ecosystem. Those customers wanted less dependence on NV and more pricing leverage. Once your revenue and access depend on that crowd, your “objective” work naturally starts to lean toward their interests. Keep the clouds happy, keep the business alive & growing. 2️⃣ AMD and other Nvidia challengers leaned in. They actively sought favorable coverage, which dovetailed perfectly with what hyperscalers wanted to hear anyway. Even if the authors believed they were being objective, the check-writers and core audience were all pushing in the same “less Nvidia, more alternatives” direction. Incentives quietly chipped away at neutrality. 3️⃣ Evidence of objectivity drift: when Nvidia launched DGX Cloud, the hyperscaler reaction was, “Nvidia is competing with its customers.” SemiAnalysis echoed that line almost verbatim, attacking the move instead of analyzing it as a rational strategic step by Nvidia. That was alignment with buyer frustration, not dispassionate investor analysis. 4️⃣ Over time, the content turned structurally more anti-Nvidia. The spicier the “Nvidia is about to lose” narrative, the more it circulates on X, and the more subs and consulting interest it generates. Drama sells. “Nvidia’s moat is intact and deepening” does not. Result: what started as “objective research” morphs into indirectly incentivized anti-Nvidia narrative. Drama = more subs = more $ SemiAnalysis didn’t suddenly “see the light” on Nvidia. The stance evolved along the path of least resistance: When the biggest checks (hyperscalers, alt-silicon vendors, and the hedge funds who trade those narratives) and the fastest subscriber growth are all tied to anti-Nvidia takes, “objective research” stops being a neutral input and becomes one more paid storyline. Good for drama and subs. Terrible if you’re trying to compound your capital.
55
39
27
470
303,734
andywy 110 retweeted
With Torch-TensorRT, a powerful compiler for PyTorch models, just one line of code can accelerate FLUX-1 Dev by up to 2.4x. Using FP8 quantization and seamless LoRA support, Torch-TensorRT delivers peak GPU performance and flexibility. 🔗 Learn more in NVIDIA's developer blog: hubs.la/Q03CqPgX0 #PyTorch #TorchTensorRT #OpenSourceAI
2
32
243
22,570
andywy 110 retweeted
I've been a bit quiet on X recently. The past year has been a transformational experience. Grok-4 and Kimi K2 are awesome, but the world of robotics is a wondrous wild west. It feels like NLP in 2018 when GPT-1 was published, along with BERT and a thousand other flowers that bloomed. No one knew which one would eventually become ChatGPT. Debates were heated. Entropy was sky high. Ideas were insanely fun. I believe the GPT-1 of robotics is already somewhere on Arxiv, but we don't know exactly which one. Could be world models, RL, learning from human video, sim2real, real2sim, etc. etc, or any combo of them. Debates are heated. Entropy is sky high. Ideas are insanely fun, instead of squeezing the last few % on AIME & GPQA. The nature of robotics also greatly complicates the design space. Unlike the clean world of bits for LLMs (text strings), we roboticists have to deal with the messy world of atoms. After all, there's a lump of software-defined metal in the loop. LLM normies may find it hard to believe, but so far roboticists still can't agree on a benchmark! Different robots have different capability envelopes - some are better at acrobatics while others at object manipulation. Some are meant for industrial use while others are for household tasks. Cross-embodiment isn't just a research novelty, but an essential feature for a universal robot brain. I've talked to dozens of C-suite leads from various robot companies, old and new. Some sell the whole body. Some sell body parts such as dexterous hands. Many more others sell the shovels to manufacture new bodies, create simulations, or collect massive troves of data. The business idea space is as wild as research itself. It's a new gold rush, the likes of which we haven't seen since the 2022 ChatGPT wave. The best time to enter is when non-consensus peaks. We're still at the start of a loss curve - there're strong signs of life, but far, far away from convergence. Every gradient step takes us into the unknown. But one thing I do know for sure - there's no AGI without touching, feeling, and being embodied in the messy world. On a more personal note - running a research lab comes with a whole new level of responsibility. Giving updates directly to the CEO of a $4T company is, to put it mildly, both thrilling and all-consuming of my attention weights. Gone are the days when I could stay on top of and dive deep into every AI news. I’ll try to carve out time to share more of my journey.
187
313
76
3,882
1,004,987
After @Kimi_Moonshot 's K2 release, I think that we have a new set of 6 tigers in LLM industry now (i.e. the so called AI 六小虎). And they ALL invest heavily in open source ecosystem. (note that this list excludes ByteDance and Alibaba) They're: @Zai_org @deepseek_ai @minimax_ai @Kimi_Moonshot @OpenBMB and @StepFun_ai Keep up your great work!
1
1
20
1,246
andywy 110 retweeted
🚀 Introducing LeVERB, the first 𝗹𝗮𝘁𝗲𝗻𝘁 𝘄𝗵𝗼𝗹𝗲-𝗯𝗼𝗱𝘆 𝗵𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗩𝗟𝗔 (upper- & lower-body), trained on sim data and zero-shot deployed. Addressing interactive tasks: navigation, sitting, locomotion with verbal instruction. 🧵 ember-lab-berkeley.github.io…
13
104
21
453
96,873
andywy 110 retweeted
A fun Tic-Tac-Toe example with a finetuned GR00T N1 VLA. Here, a high-level VLM (e.g Gemini/GPT4o) generates a prompt to guide the VLA on the best move⭕, to win the human player❌ Vibe-wise, Gemini-2.5Pro indeed does a better job perceiving the spatial state from the image 😏
13
55
5
378
38,496
andywy 110 retweeted
Morgan Stanley: Despite ASICs, Nvidia will continue to maintain a dominant market share. The ASIC category is neither superior nor inferior to commercial GPUs—it is simply another means to achieve the same result. Over the past six months, momentum in the AI sector has clearly shifted toward custom silicon. Nvidia has remained stagnant while AMD has performed sharply poorly. Nvidia’s $3 trillion market capitalization is supported by over $32 billion in quarterly AI revenue, whereas AVGO’s $1.1 trillion market cap is based on $3.2 billion in quarterly revenue. Clearly, the market has judged that the growth potential of ASICs will be several multiples that of commercial GPUs. Will ASICs outperform commercial solutions in the long term? While there are various possible outcomes, our view is that—unless something changes—the incumbent, namely Nvidia, will continue to hold a dominant market share. The ASIC category is neither superior nor inferior to commercial GPUs. It is merely an alternative means to achieve the same result. We would evaluate an ASIC the same way we evaluate a chip from AMD, a chip from Intel, or any chip from a startup—in the context of the price/performance of that silicon relative to the value the incumbent (primarily Nvidia) can deliver. The development budget for an ASIC is typically less than $1 billion, and in some cases, much lower. This contrasts with our assumption that Nvidia will invest approximately $16 billion in R&D this year alone. With that funding, Nvidia can maintain a 4–5‑year development cycle by running three design teams sequentially—each with an 18–24‑month architectural cadence—delivering innovation over a five‑year span. In addition, they invest billions in interconnect technologies to boost rack‑scale and cluster‑scale performance, and by being present in every cloud worldwide (subject to U.S. Department of Commerce approval), any investment in improving the Nvidia ecosystem propagates throughout the global ecosystem. We have examined several theories regarding whether ASICs can outperform commercial solutions. Could a custom chip be superior because it is better suited for a narrower set of applications? Sometimes, yes—and in such cases, it becomes one of the most compelling use cases for customization. The clearest example is Google’s successful TPU tensor processor. Google invented modern LLM transformer technology and directed Broadcom to develop a chip optimized for that technology, while at the time Nvidia was developing GPUs optimized for vision models using convolutional neural networks. To date, the TPU is the most prominent example of a cloud customer gaining a clear advantage through a customized solution, resulting in over $8 billion in TPU revenue for AVGO. However, an important point is that Nvidia is expanding its share of Google’s spending in 2025. Part of this is attributable to investments in the cloud sector, where commercial products tend to outperform ASICs. Yet the customization aspect of the TPU may be less valuable than it once was. Nvidia is also optimizing for transformer models, and in fact, the largest training and inference clusters are not currently highly customized. Instead, the greater benefits of customization going forward seem more likely to manifest in legacy workloads. While Nvidia’s focus on training multimodal AGI models may be overkill for some older applications, it will be difficult to outperform Nvidia when it comes to delivering high‑end training capabilities. In particular, for Google, while purchases of Nvidia products are expected to roughly double this year, TPU growth is anticipated to be more modest—partly due to Google’s investments in enterprise cloud, but also because Nvidia’s LLM transformer performance is exceptionally strong even for internal workloads. Is an ASIC cheaper? This is the most frequently heard argument. Yes, a given ASIC may not reach the performance levels of the H100 until three years after its debut, but it costs only $3,000, whereas the H100 is priced at $20,000—resulting in a lower total cost of ownership. But does this make sense? Is it really that easy to compete with Nvidia merely by producing a $3,000 AI chip? We have seen countless startups, priced only partially at Nvidia’s level, fail to gain a foothold in this space. Moreover, Intel has struggled for over a decade—even after acquiring companies that were already shipping—while pricing below cost. AMD’s first few generations were non-starters until they broke through with the MI300 last year. If a $3,000 chip were all that was needed, why wouldn’t every competitor simply produce a $3,000 chip? Meanwhile, Nvidia has introduced several lower-priced chips aimed at legacy inference applications. Two years ago, we reported enthusiastically on the L4, L40, and other announcements. However, Nvidia has also discovered that there is a gravitational pull toward the most expensive cards. Why is that? While the processor itself may be cheaper, system costs can be higher. The cluster cost for ASICs can be materially higher than that of Nvidia—which has built a 72‑GPU NVLINK domain using copper—because ASICs use more expensive optical technology. Other major cost components are at similar levels. High‑bandwidth memory costs are the same—and in fact, Nvidia is likely favored by its monopsony purchasing power for the latest HBM versions. The same applies to CoWoS; because many ASICs use smaller dies with larger stacks, the CoWoS cost can be higher than that of Nvidia. Of course, Nvidia’s wafer costs might be higher due to reticle‑limited dies, but overall, Nvidia delivers exceptional value. Furthermore, the software aspect also places a burden on customers, as it is a challenging and time‑consuming task. Ease of use when managing software changes and running various workloads is crucial, and minimizing “software developer hours” is often an overlooked element in total cost of ownership (TCO) calculations. For instance, Databricks—a Trainium customer—anticipates that it will take “weeks or months to get the system up and running” (link reference). Deployment delays can put customers even further behind Nvidia, which is equipped with the widely used CUDA (Compute Unified Device Architecture) SDK. Over the years, we have observed 20 to 25 alternatives to Nvidia’s products. Initially, there was enthusiasm based on price and potential performance that drove early deployments, but ultimately customers gravitated back to Nvidia, which has the most mature ecosystem. Consequently, alternative products are often put on hold or sometimes disappear from the market altogether. TPU and Trainium, as well as AMD’s MI300, are notable exceptions—but even in those cases, strong investments in Nvidia’s ecosystem continue this year. This is not to say that cheaper processors are entirely irrelevant; rather, historically, cheaper processors have often failed to garner as much market attention as initially expected. Could the ASIC relationship give ASIC vendors additional visibility that commercial products do not possess? We disagree. Every hyperscaler investing in ASIC technology intends to use it. ASIC development incurs significant costs because there is an expectation of returns. Therefore, there is a possibility that the SAM analysis is correct. However, the technology must ultimately be more cost‑effective than commercial solutions when delivered; if not, purchasing will often pivot back to the best‑in‑class commercial chips. One exasperated cloud executive recently remarked, “Every two years, our ASIC team delivers technology that is 2–3 years behind Nvidia. It is economically not that useful.” Of course, this is not everyone’s view, nor is it the objective at the outset of the design process. However, it is a fairly common complaint among cloud vendors, who—even while considering it an investment in the future—are not entirely convinced that adopting inferior ASICs will form the foundation of a differentiated long‑term strategy. Put another way—why does AVGO’s SAM analysis attract far more attention than AMD’s predictions of “tens of billions in annual revenue potential” for products like the MI400? AMD’s forecasts are also based on conversations with potential customers, yet investor responses are, “How can we be confident in AMD’s 2026 product (MI400) compared to Nvidia’s product (Rubin)?” Indeed, we cannot be sure. Nevertheless, for some reason, we expect that ASICs will be competitive with Rubin in 2026. However, AMD’s investments should not be taken lightly. The scale of AMD’s investments across the ecosystem tends to far exceed that of ASIC vendors. This year, AMD has completed two acquisitions of AI software assets. One of these—the acquisition of ZT Systems—involved acquiring a major server ODM, divesting the ODM business while retaining key engineering talent related to rack‑ and cluster‑scale computing. Once in possession of such assets, AMD can deploy them across multiple cloud environments, which drives third‑party support and accelerates ecosystem development. Can ASIC designers within the cloud replicate this? Are ASIC chip suppliers willing to replicate it? It is possible—and their networking expertise may help—but it is by no means a given. AMD has secured dominant technology in the server ecosystem through such iterative investments, with the progression from the early struggles of Naples to Rome and Genoa serving as a success story (though Intel’s struggles have clearly contributed as well). So why is it presumed that AMD will fail to meet its stretch goals while ASICs will succeed? In our view, AMD has just as much potential to achieve its SAM as ASIC competitors, and this is perceived by buyers as a “show-me” story. So why has ASIC not yet become a “show-me” story? In our perspective, ASICs should also be expected to deliver; this does not mean that either technology offers a guaranteed solution, but rather that the challenges of competing with Nvidia should not be taken lightly. It is difficult to say that the ASIC relationship provides suppliers with a more fixed market share, as nearly every ASIC we know of faces direct competition. For example, MediaTek (as reported by Charlie Chan) is expected to challenge Broadcom within the Google TPU ecosystem, and Alchip (also reported by Charlie Chan) is anticipated to challenge Marvell within the Amazon (as reported by Brian Nowak) Trainium ecosystem. Recent reports indicate that Meta (also reported by Brian Nowak) is considering the acquisition of a Korean ASIC design—suggesting that its relationship with Broadcom is not exclusive—and Meta is also investing significantly in AMD. Bytedance is Broadcom’s third ASIC customer, but there is also the possibility that they have alternative ASIC designs from Chinese vendors in the pipeline due to export control issues. In fact, export controls on AI-related shipments to China have lowered the technological threshold to 80% of the state‑of‑the‑art, creating a highly open competitive environment. OpenAI could become a major ASIC user, but reports indicate that they are already working with multiple suppliers. Nevertheless, Broadcom and Marvell possess enviable capabilities in this field that will undoubtedly drive growth. Success begets success, yet there are clearly challenges within the ASIC domain as well. We expect that in 2025, Nvidia and AMD will outperform ASIC competitors—especially in the second half of the year. For 2024, Nvidia’s processor revenue is projected to be approximately $98 billion, AMD’s around $5 billion, AVGO’s about $8 billion, and the combined revenue of Alchip/MRVL roughly $2 billion. This implies that commercial silicon accounts for roughly 90% of the market share, while ASICs constitute about 10%. We expect the 90% share for commercial products to increase slightly this year. The two major users of ASICs are likely to further increase the proportion of spending on Nvidia. AVGO’s largest customer is Google, which drives most of its 2024 revenue, and we expect that in the 2025 fiscal year, Nvidia will grow 50–100% more than TPU. Marvell is driven by Amazon, and while Amazon’s ASIC purchases are expected to double from approximately $2 billion to about $4 billion this year, Amazon’s purchases of Nvidia products are also projected to more than double beyond that level. Other ASIC customers, such as Meta and Bytedance, are still in the early stages. In our view, Nvidia’s revenue momentum in the second half will be significantly stronger than the builds from ASIC or AMD. Beyond 2026, ASICs could grow much faster. However, this largely depends on the value delivered by ASICs versus what Nvidia or AMD can offer. According to AVGO’s SAM analysis, Google, Meta, and Bytedance could each achieve between $60 billion and $90 billion in revenue in the 2027 fiscal year with clusters numbering in the millions—which is entirely plausible, though that technology has yet to be delivered. Yet, with the market discounting such significant market share gains for ASICs—for instance, Nvidia generates 10 times the AI revenue of AVGO this quarter while its market cap is only three times as high—doesn’t the onus fall on the ASIC side to prove its case? And if we consider the Blackwell ramp in the second half of 2025 as the major story, wouldn’t the market capitalization that shifted from Nvidia and AMD to AVGO/MRVL over the past six months swing back? Does this contradict our earlier work on the ASIC opportunity? In the deep‑dive report on the ASIC opportunity led by Charlie Chan last year (link here), we forecasted that the total addressable market (TAM) for AI ASICs would grow from $12 billion in 2024 to $30 billion in 2027—a figure considerably lower than what is being discounted in the market now (for example, AVGO’s SAM for 2027, based mainly on ASICs for three out of five potential customers, amounts to $60–90 billion). Our view remains unchanged, but market expectations have shifted dramatically. In particular, Charlie Chan maintains a bearish view on Asian peers such as Alchip and MediaTek, as they are securing projects at the expense of their U.S. competitors. Referring to the same report, Asian vendors like Alchip (with the AWS 3nm project) and MediaTek (with Google’s 3nm TPU v7 for training) are expected to expand their market share in 2026. So what should we do regarding U.S. AI semiconductor stocks? Simply put, buy Nvidia. The stock remains our top pick and is currently trading at a significant discount compared to AVGO/MRVL. We believe that this discount will not last long. Nvidia’s biggest short‑term risk is U.S. export controls, which are equally problematic for AVGO. In the long term, the greatest risk is not competition but a slowdown in investment—which we forecast to occur around mid‑2026, though we continue to push that timeframe out as visibility remains strong. Meanwhile, we continue to maintain a bearish view on AVGO while closely monitoring expectations. Frankly, feedback on this note will play a crucial role in shaping investor expectations for AVGO relative to the 2027 SAM analysis. This year, AVGO AI remains in a transitional phase—with one fully penetrated customer (Google) that has slowed due to a product transition and two new customers that are still small. The company’s three‑year SAM analysis was partly presented to explain the short‑term slowdown and the strong ongoing investment, but the trajectory—from $3.2 billion per quarter to $18 billion over 12 quarters—could raise short‑term expectations even further. We are sidelining AMD, MRVL, ALAB, MU, etc. We are not fundamentally negative on any AI‑related stocks, but we do have concerns about short‑term momentum shifts—particularly as the multiple premiums for MRVL and AVGO far exceed those of Nvidia. While AMD’s valuation is becoming increasingly attractive, the MI350 does not seem revolutionary, and the MI400 remains somewhat of an unknown. We have confidence in the execution of both AMD and Nvidia. ALAB stock is so volatile that it is difficult to make a persistent valuation call, but at current levels, we maintain a constructive view. Management has indicated that Nvidia will be the strongest driver of demand in 202X, yet Nvidia’s data center—which returns twice the operating margins of its peers—continues to record the lowest EV-to-AI revenue ratio in our analysis. Source: Morgan Stanley Research $NVDA $MRVL $AVGO
16
67
17
394
120,373
AI 技术对于生产力的提高,很多还没有反应到统计数字里面,有的时候是因为统计数字的方法不能正确捕捉底层技术的进步。 比如说,某个大公司以前一大堆未格式化的数据,和胶水程序,以前人工编程处理,工作量大,极为繁琐,现在 AI 工具帮助,可以把效率提高十倍以上,同时也减少了对于胶水程序员的需求。 以前有个问题不懂,网上搜索和询问周围的人也无法获得满意的答案,和专家预约要等很长时间,和专家咨询后还是不满意他的回答,会困扰很长时间。现在马上就可以从 grok 那里获得全面和基本满意的答案,迅速采纳建议,提高生活和工作质量,如浴春风。 以前开车在路况比较复杂的地方会精神更紧张,现在 fsd 可以提前警告和精确回避很多潜在风险,驾驶体验也愉快很多。 那些不理解高科技进步的威力,散布"恶性通胀末日论"的从来没有做生意经验的经济学者,是何等可笑和愚不可及。
4
16
127
25,789
andywy 110 retweeted
Introducing NVLM 1.0, a family of frontier-class multimodal LLMs that achieve state-of-the-art results on vision-language tasks, rivaling the leading proprietary models (e.g., GPT-4o) and open-access models (e.g., InternVL 2). Remarkably, NVLM 1.0 shows improved text-only performance over its LLM backbone after multimodal training! We are working towards releasing the model weights very soon 🤗 and will open-source the training code for the community. For further details, please visit our project website: nvlm-project.github.io/
11
119
16
469
168,749
andywy 110 retweeted
vLLM, running on the latest @nvidia H200 GPUs, is delivering high token throughput per user on Llama 3.1 405B out of the gate! Check it out 👇
5
13
3
115
12,686
andywy 110 retweeted
Live tweeting the most interesting insights from @Meta´s new Llama3 paper 1. How did the arrive at a 405b model trained with ~15T tokens? "Extrapolation of the resulting scaling law to 3.8 × 1025 FLOPs suggests training a 402B parameter model on 16.55T tokens." 👇🧵
3
85
4
821
240,301