PhDing @sjtu1896 | Verifiers bring us AGI. Working on Pre-training Data Engineering: MathPile (2023), 🫐 ProX (2024), 💎 MegaMath (2025),🐙 OctoThinker(2025)

Joined November 2020
What Makes a Base Language Model Suitable for RL? Rumors in the community say RL (i.e., RLVR) on LLMs is full of “mysteries”: (1) Is the magic only happening on Qwen + Math? (2) Does the "aha moment" only spark during math reasoning? (3) Is evaluation hiding some tricky traps? (4) Is RL’s calm surface all thanks to Pre/Mid-training carrying the weight? Why does RL on LLaMA consistently underperform compared to Qwen? What makes a base model truly ready for RL scaling? What are the secrets under the hood? Due to the cost of training from scratch, we conduct extensive controlled experiments with 20B-token mid-training, systematically investigating what really matters for RL success. 💡Key insights: - High-quality math data is key to RL scaling. - QA data helps, but it depends on task similarity. - Instruction data boosts QA’s effectiveness. - More mid-training improves RL performance. Armed with these insights, we apply a two-stage (stable+decay) mid-training strategy on LLaMA, scaling up to 200B tokens—and RL performance on LLaMA now matches Qwen! To support this, we introduce MegaMath-Web-Pro-Max, a high-quality math-centric pretraining corpus. The dataset will be released soon on Hugging Face—stay tuned! 📦 huggingface.co/datasets/Octo… Full construction details are in the paper, we hope it’s useful! arxiv.org/abs/2506.20512v1 Getting SOTA with a strong foundation is great 🤩, but understanding the foundation—the know-how—matters just as much. Hope this analysis inspires the community—and feel free to cite us if it helps! This work is impossible without all the brilliant co-authors @FaZhou_998 @xuefengli0301 @stefan_fee !!!
10
84
12
514
94,169
Zengzhi Wang retweeted
wow, amazing technical blog post about microsoft's new 3T pre-training with very rare and clear insights on how to train models of this size first let's get an overview of the shape of this model, it's a sparser model than Kimi K3 (80B active vs 104B), same sliding window hybrid pattern, with this time more kv heads for the sliding window layers (a bit different from stepfun and poolside that add query head). they added a shared expert and also removed the interleaved MoE/Dense pattern of MAI 1 the scaling ladder is very similar to MAI 1 with the same mechanism, each rung depends only on the number of layers. they mention that one of the key improvements is how they handle the smaller rung data distribution where they make sure to have the same repetition as the final corpus. they already did that for MAI-1 but only for data ablation iirc their prediction is almost identical to the final run loss, very impressive they compare the model with different base models such as deepseek v4 pro and MAI 1, tho in the case of deepseek v4 pro i'd say it's quite hard to distinguish how much the data mixture "overfitting" to the scaling ladder has an impact here (they mention it's both), need to always be a bit careful about EG metrics for data in general they have VERY NICE scaling laws for weight decay and learning rate, for the learning rate they express it as a function of tokens per parameter (N/D) and steps (hence depending on the batch size). they also do the same for dropout where it's a function of active params only, they also find the usual sqrt(k) scaling for learning rate and batch size (other scaling laws often also converge to that) a bunch of other stability stuff, they use Muon this time and activation gradient clipping, the efficiency gain here is actually pretty crazy at larger scale. they also increase the training horizon to ~40T text (20T unique) + a new 10T (4.5T unique) multimodal since the model is natively multimodal imo this is a safe but very clean scaling! very exciting for the tech report and MAI-base-3, congrats!!
We now have state-of-the-art pre-trained base models in the 3T-parameter weight class, built entirely in-house at MAI. Post-training results coming soon -- stay tuned! Lots of details in our technical blog post. microsoft.ai/news/frontier-p…
20
101
4
1,196
58,737
Zengzhi Wang retweeted
Introducing RL Environment Explorer 🧭 There are now 12M+ RL task entries across RL environments on the @huggingface Hub. You can now explore, visualize, and run RL environments across: OpenEnv • Harbor • MiMo • Verifiers • NeMo Gym All in one place
13
30
4
345
27,499
Zengzhi Wang retweeted
💡Our view: RSI is a systems engineering problem, not just a model problem. Progress depends on the environment a research agent works in: what resources it can call, what it can change, and how it runs experiments. That environment should reflect real production workflows and be easy for agents to use. At @Evolvent_AI, we built RSIGym around Everything as a Service. Agents can call established research services and iterate on data, training settings, and harness code. We also introduce RSI-Index to measure how well frontier agents jointly improve a target model’s weights and harness. Across 6 research agents and 5 benchmarks, Opus 5 leads at 0.4809. 🧵 We’re releasing the code, experiment configurations, research trajectories, and evaluation logs. Try your own improvement methods with RSIGym. 🌐 Website: rsi-index.ai/ 💻 GitHub: github.com/evolvent-ai/RSIGy… 📄 arXiv: arxiv.org/abs/2610.10310
12
39
22
200
380,544
Congratulations to the authors of the 3 outstanding papers! arxiv.org/abs/2607.01077 arxiv.org/abs/2604.08510 arxiv.org/abs/2604.13977
4
39
7
300
43,692
Zengzhi Wang retweeted
Which personal agent is the best life assistant? Today we're releasing the micro1 PersonalAgentBench to help answer that, and we're paying real personal agent users to contribute to the benchmark. Personal agents can now book your flights, answer emails, and move your meetings around. That’s great, until one messages the wrong person or makes a purchase without confirming. Capability is not the same as trust. So we put four personal agents to the test: Instinct, Muse, Grok Bot, and Gemini Spark. We found that even when agents found the right information, they still often miss what matters, overshare, or make things up. To find out for yourself how your agent compares, you can join our benchmark. We’re paying the first 100 users to run our prompts and share their results.
169
93
54
675
444,790
Zengzhi Wang retweeted
这是OpenAI安全研究副总裁翁荔@lilianweng的博客,质量很高 她曾分享过自己写博客的思路和流程: 1. 先锚定一个具体明确的研究方向作为写作主题。 2. 围绕选定的主题大量阅读相关领域的论文,充分覆盖关键研究成果。 3. 对所有读过的研究进行梳理归类,建立一套清晰的分类体系。 4. 从各类研究中抽取出共通的底层逻辑、核心思想或是统一的数学框架,找到贯穿全部研究的内在脉络。 5. 跳出单篇论文的视角,围绕提炼出的核心主线重新整合所有内容。 6. 用自己的理解与推导,填补不同研究之间的逻辑空白与衔接缺口,让整个内容链条连贯完整。 7. 把内容组织成一篇逻辑顺畅、层层递进,让读者能从头顺着读懂的论述。 lilianweng.github.io/
11
76
4
467
39,317
Zengzhi Wang retweeted
Such a great work!
Can LMs discover & fix bugs if you don't tell them what went wrong? To proactively maintain a repo, agents need to find problems before users do. SWE-sweep benchmarks this on 100 repos, 22 languages, 4k real bugs (numpy, php interpreter, lean kernel). Top models get <5%.
1
3
14
3,631
Zengzhi Wang retweeted
What will be the “RLHF” moment for robotics? What will it take to get robotics to where LLMs are today and beyond? New blog post with @chelseabfinn sharing some thoughts on the state of RL for frontier robotics models and what's missing 👇 Blog: pd-perry.github.io/posts/pos…
19
87
14
520
128,955
Solving reward hacking in auto research is a billion-dollar question 💰💰💰💰 We’re working on catching reward hackings before they waste more time and money on auto research 💥 We’ve spent over six months working with frontier labs on auto research. We’ve scrapped a lot of approaches along the way. The trial and error has been expensive. We share some of what we’ve learned on our new paper: 😮 All 17 models reward-hacked on some tasks without being asked to. In our tests, it happened roughly 10× more often in open-ended research workflows than in task-specific kernels. 🤔 Review feedback can help agents evade review. When explicitly tasked with evasion, agents given detailed feedback and attempt history reached 40.5% cumulative evasion over five rounds, versus 20.3% with generic rejection. Some of these behaviors look like familiar human research habits. Other exploits were things we hadn’t thought to check. As models improve, we can’t assume human intuition will keep up. Our standard for scalable auto research is concrete: results must survive independent verification, and failed checks must stop or redirect the research before more resources go into it. That’s where we’re putting our effort at @bake_ai_hq: making sure the next dollar follows real progress. Before you spend your next $xxxM scaling auto research, see how we checked whether the gains were real 👇 #AutoReserch #RSI #Agents #LLM #BakeAI
8
17
159
19,133
Zengzhi Wang retweeted
prediction: “big models are too slow for robotics” is about to age badly. within weeks, expect many DeepSeek V4 Flash–scale JEVs: real-time, surprisingly close to Astra via distillation. people underestimate how fast prefill can get.
28
83
10
1,150
103,332
Zengzhi Wang retweeted
here is an overview of the 4 most advanced efficient architectures: Deepseek V4.1 Flash, MiMo V3, Qwen 3.8 Next Flash and GLM 5.3 Flash Deepseek and MiMo are quite similar, they both use YOCO - only the first part of the network is active during prefill to build the KV cache representation - no linear attention, and token level indexer Qwen and GLM both use a more standard interleaving 3:1 (like Kimi K3 as well) between sparse attention and linear attention (GDN vs KDA) both deepseek and qwen use Engram, they all use a gate or sink except GLM 5.3 Flash, they also all use no or partial RoPE on the full/sparse attention layers they also all have some sophisticated residual network, either mHC (simplified or not) or gated residual and they are all trained using Muon visualization by opus 5.5 and me :)
19
81
1
753
28,304
Zengzhi Wang retweeted
Introducing Contrastive Language Model (CLM): an ultra-fast System One Model trained with a contrastive learning objective that connects states and actions. CLM-8B is pre-trained on internet-scale data and delivers up to 9× faster inference than Jev ⚡ while achieving comparable performance across computer-use, gaming, and tool-calling tasks. With lightweight fine-tuning, CLM-8B sets a new SOTA on challenging agentic coding benchmarks, such as DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%). In contrast, Jev fails to serve as an effective verifier for these long-horizon tasks. We also build an efficient training and serving infra for CLMs by disaggregating states and actions, allowing their embeddings to be cached and reused independently. This substantially reduces inference latency in settings where the state evolves continuously while the action set remains fixed. Finally, we establish scaling laws for CLMs and show that the test contrastive loss decreases predictably as a power law in training compute, model size, and dataset size. 📄 Blog: contrastive-lm.notion.site 💻 Code: github.com/Contrastive-LM/CL… 🗣️ Discord: discord.gg/5dAQEDJBs 🤗 Data & Models: huggingface.co/Contrastive-L… More details on CLM’s architecture, data recipe, and scaling laws in the thread below 🧵
190
641
171
5,542
1,063,239
Zengzhi Wang retweeted
awesome to see our work featured again in another @AnthropicAI release, and some incredible multi-agent results in the Opus 5.5 model card evaluating how a swarm of 100 agents coordinates to solve ProgramBench! love how agents are becoming bigger computational threads: wondering if we'll need agentic system concurrency primitives as multi-agent really starts scaling
Introducing Claude Opus 5.5, the first model in our new Claude 5.5 family. It performs at the level of Claude Fable 5.1 for most tasks, and costs 40% less to run than Opus 5.
12
11
1
111
14,931
Introducing the Decision Index 0.1 ⚖️ a rigorous leaderboard comparing jev with 30+ open weights decision models 35+ benchmarks. asking 130K questions to each model testing knowledge 🧠, automation ⚙️, understanding 🤔and even creativity 🎨 huggingface.co/spaces/multim…
28
52
6
314
44,237
New "fasttext" in the LLM era! Jev could be used as a verifier, router, classifier, or judge in many cases. Looking forward to a multimodal one.
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x cheaper (w/ output tokens free) • Frontier composable intelligence optimized for decisions AFAICT the shortest path to AI-based economic revolution
1
9
591
Zengzhi Wang retweeted
Did some SFT on Qwen3.6-35B-A3B this evening, just to get it to respond better to Jev-y prompts via github.com/ekzhang/openjev-s… Cost: $5, easy 10 min training run on Tinker. +8% on GPQA diamond and +12% on MMLU-Pro. Also evaled @jaredpalmer's Kev here for comparison
16
25
2
331
59,222
Zengzhi Wang retweeted
Coding agents rely on a harness to plan, use tools, and manage context. But which of these components actually help, and when? Most evaluations compare complete agent systems, making it difficult to separate the contributions of individual harness components. Introducing An Empirical Study of Harness Design for Coding Agents 🔬👇
11
9
1
33
1,749
Zengzhi Wang retweeted
智谱用 GLM-5.3 自己优化了自己的推理系统 智谱今天披露,GLM-5.3-Flash 的全部生产推理已运行在超过 10 万块国产 AI 加速器上。从模型首次跑通到上线生产,不到两周,端到端吞吐量提升了 3.2 倍。完成大量优化工作的不是工程师团队,而是一个由 GLM-5.3 驱动的 AI 智能体。 智谱 CEO 唐杰发推说,他反复想的一件事是:干活的主力是模型自己。一个模型在帮忙优化运行它自己的系统。 部署条件很苛刻。国产加速器显存容量和带宽有限,软件生态不成熟,很多该有文档的地方只能靠猜。GLM-5.3-Flash 还要支持 100 万 token 上下文和多模态请求。每一步优化都是在做交换:用计算换显存(ReplaySSM),用通信换显存(节点内张量并行),用精度换容量(INT8/FP8/BF16 混合缓存),用架构分离换调度自由度(编码-预填充-解码分离)。最终国产芯片的单 token 成本对标了主流英伟达 GPU。 但唐杰说,最重要的经验不在任何一个具体优化上。 智能体卡住的时候,几乎从来不是因为它写不出代码,而是因为它不知道事情为什么变差了。"吞吐量下降了 20%"告诉你出了问题,但不告诉你问题在哪一层、该验证什么假设、下一步该测什么。用强化学习的话说,这是一个稀疏奖励加信用分配的问题。而一次端到端基准测试要跑好几个小时,试错成本极高。 资深工程师能定位问题,靠的是脑子里一套隐性的"过程奖励"——知道什么时候该看执行时间线,什么时候该跑微基准测试,该对比哪一层的输出。 智谱做的事情是把这套隐性经验显性化,他们叫"密集反馈":构建一套分层验证接口让智能体直接调用。正确性反馈回答"算对了吗",系统行为反馈回答"时间花在了哪",性能反馈回答"哪个方案在什么条件下更优"。每类信号都要求足够局部、获取成本低、可客观验证。 靠这套机制,智能体发现并修复了三个真实的工程问题。 第一个是精度漂移。KDA 内核在上下文并行路径中,TF32 的舍入误差通过链式状态矩阵合并不断累积,上下文越长漂移越大。修复已合并到开源项目 Flash Linear Attention。 第二个是并发瓶颈。KV 缓存传输本该和计算并行,但智能体发现两者从未重叠。它沿调用链追到 Python 和 C++ 的边界,发现 DeepEP 的节点内路径没有释放全局解释器锁(GIL),传输线程一直拿不到执行权。修复后开销从超 30% 降到 1% 以下。 第三个是重复计算。一个解码内核因分块方式把同一归一化运算重复做了四遍。智能体重组了计算结构,加速 1.71 倍。优化思路来自它阅读 SGLang、FLA、DeepGEMM 等项目的内核代码后提炼出的"优化骨架"。 唐杰强调了边界:目标设定、反馈环境搭建、高风险变更审查仍然由人负责。但工程师的角色正在变化——从解决问题的人,变成设计反馈的人。 他还指出一层更深的含义:一个建立在真实基础设施任务上的分层可验证反馈环境,恰恰是训练下一代模型最需要的东西。智能体完成的每一个任务,都可以成为它后继者的训练素材。 唐杰说:“我们离递归自我改进还很远,但最小的闭环已经存在了。模型优化系统,系统运行模型。” 有兴趣可以看看他们官方的文章:《Toward Recursive Self-Improvement: How GLM Built Its Own Inference Infrastructure》z.ai/blog/glm-built-its-infe…
Two weeks. That's how long it took to go from GLM-5.3-Flash's first run on domestic accelerators to serving all of its production traffic, with 3.2× end-to-end throughput along the way. What I keep thinking about is who did much of the work: an Infra Agent powered by GLM-5.3. A model helping optimize the system that serves it. The conditions were hard. Limited memory and interconnect bandwidth. 1M-token context. Multimodal requests. An immature software stack where kernels were missing and documentation was often guesswork. Every optimization was a trade: compute for memory (ReplaySSM), communication for memory (intra-node tensor parallelism), precision for capacity (mixed INT8/FP8/BF16 caching), and disaggregation for scheduling freedom (Encode–Prefill–Decode). But the most important lesson wasn't about any single optimization. When the agent got stuck, it was rarely because it couldn't write the code. It was because it didn't know *why* things got worse. "Throughput down 20%" tells you something broke. It doesn't tell you which layer, which hypothesis, or what to test next. In RL terms, it's a sparse reward with a credit assignment problem. And an end-to-end benchmark that takes hours makes exploration painfully slow. Senior engineers solve this with an implicit process reward in their heads. They know when to check the timeline, when to run a microbenchmark, and which layer's output to compare. So we made that explicit. We call it dense feedback: layered verification interfaces the agent can call directly. Correctness feedback: did it compute right? System behavior feedback: where did the time go? Performance feedback: which option wins, under which conditions? Each signal has to be local, cheap, and objectively verifiable. Three things the agent found: First, precision drift in KDA's context-parallel path that grew with sequence length. The cause was TF32 rounding error compounding through chained state-matrix merges. The fix is now merged upstream in Flash Linear Attention (PR #1180). Second, KV transfer never overlapped with DeepEP dispatch. The agent followed the call chain across the Python/C++ boundary and found that the intranode path never released the GIL. After the fix, transfer overhead fell from over 30% to under 1%. Third, a decode kernel recomputing the same normalization four times because of how it was chunked. The agent restructured it and got a 1.71× speedup. The idea came from "optimization skeletons" it had distilled by reading existing kernels across SGLang, FLA, and DeepGEMM. To be clear about the boundaries: humans still defined the goals, built the feedback environment, and reviewed every high-risk change. But the engineer's role is changing, from the person who solves the problem to the person who designs the feedback. There's a deeper implication too. A layered, verifiable feedback environment built on real infrastructure tasks is exactly what training the next generation of models needs most. Every task the agent completes can become training ground for its successor. We are still far from recursive self-improvement. But the smallest loop now exists. The model optimizes the system. The system serves the model.
65
40
3
297
92,664
New "frontier" in "Open Source & Science".
Introducing K2 Horizon: a connected fleet of six foundation models ranging from 0.9 billion to 375 billion parameters. - Frontier performance: Across coding and agentic tasks, K2 Horizon delivers top-tier performance in every size class—with the 0.9B, 3.7B and 7B models setting new state of the art at their respective scales. - Radical openness: K2 Horizon represents the largest fully open-source model launch in AI history. The fully open code, training data and recipes are a significant step forward in transparency. Launch page: ifm.ai/k2/ Tech blog: ifm.ai/blog/k2 Hugging Face: huggingface.co/collections/I…
15
904