来自 OAI 员工的点评
I’ve been testing 6.2 Astra from internal ckpt. for about 3 weeks before I went vacation. The improvements I got compared to 6 Astra are:
1. It’s less lazy
2. It writes less coding slop.
3. Generally easier to work with than 6 Astra.
Some needs improvements are
1. On writing, it lacks of a high level progression narrative.
2. Implicit instruction following, understand what is not in the prompt verbatim, is still not perfect.
Very different from the sentiment I got today in X.
1
236
灰机仔 retweeted
repeat after me: oneshotting a threejs "game" is not indicative of real world model performance.
i'm tired.
灰机仔 retweeted
China’s response, translated into plain language:
AI is a shared asset of humanity, not ammunition for the next Cold War.
If ideology determines who is allowed to develop, access and benefit from AI,
then AI is no longer serving humanity.
It is serving power.
灰机仔 retweeted
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me.
I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA.
I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation.
Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO.
Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours.
Jensen
Claude Code 团队分享了他们如何让 Agent 验证自己的工作的经验。众所周知,目前Claude Code 团队约 90% 的代码由 Claude 编写和评审。
Claude Code 团队认为,验证回路决定你能委托出去多少工作。没有验证回路,你只能同时运行一两个 Claude,因为每件事都要自己检查。有了验证回路,你可以并行运行数十个到数百个。在内部测试的在结构化文件生成任务中,独立评分器按评分标准检查 Claude 的产出,任务成功率最多提升 10 分,最难的题目提升最大。同时成本也在发生变化:测试、基准和规范现在可以由Agent快速生成,生成后作为软件反复运行,消耗的 token 很少。验证结果还能回流改进Agent,例如把重复失败的日志检查写成 hook 或 CLAUDE.md 规则。
三类值得投资的测试
作者先给出两个失败模式。测试不稳定时,Agent会反复重跑测试,而不修复失败的代码。测试慢时,Agent会减少每批的测试数量,问题定位变难,因此重型测试应放在提交改动之前运行。
1. 端到端测试
只能选一类测试时,选端到端测试。它描述真实用户流程,例如把商品加入购物车、结账、进入确认页。
它有效的原因是难以糊弄:整个功能要么可用,要么不可用。它有明确的完成标准,并能具体说出坏在哪里。单元测试更快也更稳定,但可能通过测试而功能仍然坏掉,因为Agent和测试都没有理解完整任务。单元测试只显示 Error: Timeout after 30 seconds,端到端测试可以给出截图和故障描述。
做法是给Agent真实运行系统的操作能力和观察能力:用 skill 启动、驱动、检查应用,工具包括 CDP、模拟器、API 测试框架。功能地图可以充当指南和评分标准,它从用户视角列出功能、路径、快捷键、选择器和预期结果。
2. 属性测试
多数测试基于你预想的故障。属性测试覆盖人们容易漏掉的边界情况,而Agent常在这些地方失败。
做法是先写一条你认为不会失败的陈述,再用测试库生成数千个输入去推翻它。库包括 JS/TS 的 fast-check、Python 的 Hypothesis、Rust 的 proptest。
常见属性有四类:
往返编码:编码 X 再解码,结果仍是 X。解析和序列化同样适用。
不变量:必须始终成立的约束,例如账户余额不得变为负数。
幂等性:迁移、归一化、重试执行两次的效果与执行一次相同。
等价性:优化后的代码与朴素但正确的代码产出相同结果。
属性库用与原始代码不同偏见的边界情况约束 Agent。规范已经偏离生产行为、不再是可靠验证来源时,它对重写工作特别有用。
3. 差异测试
差异测试把旧系统当作判定标准,适用于功能应保持不变的迁移场景。做法是用相同输入分别运行旧系统和新系统,结果不同就说明存在缺陷或破坏性变更。
关键点有三个:输入用随机生成或生产流量抽样;归一化空格这类无关因素;为应当不同的情况维护一份已批准变更清单。
团队的验证实践
验证等于运行改动。 端到端运行应用比测试更重要,就像 iOS 改动必须在模拟器里试过才能发布。Claude 可以像人或 QA 工程师一样测试整个应用:网页改动用 Claude Chrome 扩展,移动端用 iOS 或 Android MCP,桌面端用 computer use,后端在本地运行服务。
落地方式是建一个名为 verify 的 skill,放在仓库的 .claude/skills/ 目录,尽量靠近相关代码,数量按需增加。Claude 知道在提交改动前使用它。之后要投入让它快且可靠:让 Claude 迭代它并做压力测试。它是冒烟测试框架,必须足够稳固。Anthropic 几乎所有改动都进行这样测试。
失败测试需要人工介入。 没人能读完每个改动,因此用失败来决定哪些需要人。Claude 运行检查并尝试修复失败项。同一检查连续三轮仍失败时,Claude 停止并引入人。
代码所有者编写自己的检查。 拥有代码的人编写检查,并反复打磨到检查能标记出所有者会标记的问题。检查引入评审人时必须说明原因,评审人不必从 diff 里自己推断。
可靠性检查按 diff 触发。 团队维护一份 39 项的可靠性清单,每项包含可从 diff 读出的触发条件、检查内容、背后的事故或 PR。其中一项是「--resume 加载旧会话记录」:会话存储或消息类型变更时,追问新字段能否安全加载,因为旧记录没有它。Claude 读 diff,只检查触发条件匹配的条目,通常是 3 到 10 项,没人需要记住历史事故。
一个评审人,多轮评审。 团队不靠叠加评审人来改进评审。在主评审人之上再加一个更贵的评审人,效果通常不如调优主评审人。团队用 REVIEW.md 写明这段代码该关注什么。各轮评审要互相衔接:把前几轮的上下文和历史失败指向每一轮,而不从头重查,这样更便宜也发现更多。
改动不得修改自己的检查。 改动能修改判定它的检查时,这个检查已经失效。Code Review 从基线分支或默认分支读取 REVIEW.md,不从 PR 的改动里读。团队确保门禁规则所在分支不被该改动修改。
评审只对提交生效。评审进行时有人继续推送,评审结果就不再有用;过期提交上的通过评审没有意义。
团队还先跑便宜的检查。Agent没在本地测试就推送,等于消耗共享的评审和 CI 时间,而缺陷本可以自己发现。团队要求 Claude 在开 PR 前运行测试、构建和 linter。可以用 hook 在 Claude 结束时运行测试,或用 /goal 要求测试套件通过来强制执行。
新门禁从观察模式起步。 团队给每个提交打上验证方式标记,标记包含 "none",这样可以查出未验证的提交。团队还建了提交门禁:会话中没有 /code-review 或 /verify 时可以阻止提交。门禁有 off、observe、enforce 三种模式。observe 模式记录本会被阻止的提交,用于复核误报并确认无人受影响。enforce 模式阻止提交,提交信息里可以写明未验证的原因。同一会话被阻止两次后,下一次放行,避免 Claude 卡住——Claude Code 并不完美。
用真正重要的案例测试你的检查。 团队定期评审评审器本身,跟踪三项数据:合并后被回滚的改动数量、维护者在合并前拒绝的改动数量、一份规模匹配且保持干净的改动子集。
也可以做正式评估。Code Review 团队基于历史事故和回滚建立评估集:把仓库还原到评审当时的状态,为每个案例提供一句话的答案要点,只有正确描述该缺陷的发现才得分;用已合并且稳定的改动作为对照组,防止过度标记;调优集与用于评估系统的固定测试集分开。团队还分别评估原始发现、验证后、最终评审三个阶段的召回率,这样能看出系统是否丢掉了已经找到的缺陷;把三者合成一个端到端分数会丢掉这个区分。
除有效性外,团队还关注改动的重要性。好的评审人能抓住技术上正确的问题,但人们在意某个缺陷属于必须修、应该修还是小瑕疵。Code Review 会打上 Important、Nit、Pre-existing 标签,Pre-existing 表示评审人不应让作者为早已存在的缺陷负责。团队把召回率调到再提升就会过拟合评估集、而不再真正抓到缺陷的那个点为止。
把遗漏变成规则。 团队有两种方法。第一种是从人工评审意见和 PR 回滚中提取模式:问题必须在至少两个 PR 中出现,并且能从 diff 识别,才算候选规则;三个Agent尝试推翻它;最后由人在 PR 中评审并批准。第二种是把三个缺陷发现器对关联已知事故的 PR 重放:至少两个发现该缺陷,就算可检测缺陷;小 diff 上漏掉该缺陷,就值得写进 REVIEW.md 成为规则。
不稳定测试同样处理。Claude 不得跳过它们、加 sleep 或重试到通过。找不到根因时,Claude 删除该测试并写下原因。
规则也需要清理。测试套件、验证步骤和 .md 文件放着不管只会增长,因此团队例行整理。
结论
给 Agent 一条能验证自身工作的强反馈回路,效果差别巨大。文章建议把验证层当成一个系统来维护,具体有四条:
1. 把校验放在它所判定的改动够不到的地方。
2. 用对你真正重要的缺陷给校验打分。
3. 校验漏掉问题时写下经验,避免同一问题发生第二次。
4. 定期修剪这些经验,让它们始终值得运行。
做到这些之后,你的工作从读完每个 diff 变成定义「完成」的标准并审阅证据。这正是能同时运行多个 Claude 的前提。
1
390
灰机仔 retweeted
I got red pilled about
NYT, WSJ, The Economist's China coverage during the HK terrorism. Before then, I thought the journalists didn't have the language skills or knowledge of China to write a balanced narrative but I didn't think they were liars.
But in 2019, all these journalists said the protests were peaceful when I saw with my own eyes the targeted bigoted violence against mainlanders. Molotov cocktails. Well organized, pipe wielding teenagers beating the shit out of people and destroying businesses.
I realized then these media outlets didn't act in good faith. It wasn't about language skills or lack of knowledge. These outlets were imperialists and bigots. They were every bit as bad as those who bombed innocent babies in the name of American hegemony wrapped up in democracy.
Then my red pilling doubled down on Xinjiang with these outlets allegations about genocide borne from people literally paid by the US government to contain China. Adrian Zenz. Look at his background on what he calls gays and non-believers.
I traveled all over Xinjiang over many trips and got pilloried for reporting the truth. Some harsh policies, stemming from fear of more terrorism, but certainly no genocide.
It was clear to me then the western media again wasn't acting in good faith. That's twice.
Now the Jensen story. Allegations about spying becomes truth without evidence due to alleged tips from Marco Rubio and Anthropic.
The same western media that lied about HK and Xinjiang are now condemning Jensen and writing a narrative that while ultimately might be true now doesn't have proof. We all know that the likelihood of proof coming out is virtually non-existent.
I long wanted to have my books and research accepted by these western media outlets.
No longer. These are immoral people. Imperialists and bigots. Some might be racist.
I want them to hate me. All good, moral people should want evil, immoral people to hate them
The Economist. The NYT. The WSJ. I hope Jensen sues them into oblivion and makes them shut down just like Peter Thiel did to Gawker.











