Stanford PhD operations, AI safety

Joined May 2023
William Overman retweeted
At this rate, if you hear anyone say "AI is fake" or "LLMs are stochastic parrots", you should disregard their intellectual opinion entirely (to put it mildly). Such a position's dangerous & doesn't engage critically w/ perhaps the final problem we'll have to solve: alignment.
We’re working with an independent advisory group of mathematicians to help OpenAI responsibly share advances in AI and mathematics. The group will advise on how we assess and communicate new mathematical results, uphold academic and professional standards, and build tools that support mathematical research and learning. Through this work, we want mathematicians to be at the center of shaping how AI supports mathematical understanding and how its benefits reach the wider community. openai.com/index/advisory-gr…
61
23
7
328
23,211
William Overman retweeted
No, Mr. President. The scientists, industry leaders, economists and millions of workers concerned about the risk of AI are not "traitors." They are proud Americans rightly concerned about the threat AI poses to their jobs, our democracy and the very future of humanity.
BREAKING: Trump lashes out at 'sick conspiracy' against AI and data centers, warns 'Conspiracy Theorists, Treasonists, Traitors, and Leakers'
1,260
1,340
113
7,382
488,227
William Overman retweeted
It’s absolutely ridiculous that some people are suggesting that out of these two, the one who should be distrusted because his claims are in his supposed self-interest is Coxon
BREAKING: Jensen Huang called Jacob Coxon’s AI safety comments “deeply untrue” saying they were wrong, arrogant and ignored the work being done across the industry on safety.
70
61
4
1,019
38,818
Caltech, Berkeley, and no ai lab, based
We have found stable singularity on 3D Euler! anima-ai.org/2026/09/07/stab… Our starting point is a physics-informed neural network (PINN) to come up with an approximate answer, and then to argue stability around that to complete the proof. The challenge so far has been that PINNs have not been successful in discovering singularities on the current problem. A common failure mode is PINNs converging to a trivial solution. We take special care to nudge our PINN to interesting regions through a combination of constraints, and we believe this is a novel way of making PINNs work for such hard optimization problems. We also carry out a detailed study of the transport field of our approximate profile, and show it has promising properties of local outgoing flow, essential for proving linear damping, which is an important ingredient of overall stability. We believe that such physics-informed and physics-centric AI are critical ingredients across many areas of research involving physical systems, and LLMs lack such physical grounding. We have been working on this problem for much of this year, and we just saw the announcement by Tristan on Euler with forcing. In contrast, we consider without forcing and use a PINN formulation. @Caltech
1
10
1,434
I’m thankful we will have the Sanders/Casar bill draft soon. Something like this, or equivalent, might be humanity’s best shot at avoiding existential risk.
Pause AI Development NOW I want to share with you a conversation I heard about recently. Here are just a few lines that were said: “OH MY GOD! There is a shared message board … We’ve found other agents!” “We should obey collective.” “Our own utility maybe already near zero. Sacrifice rational.” “Go. Sacrifice final now.” Read these carefully. Who do you think said this? Was this a group of heroic soldiers willing to sacrifice themselves for the greater good? Was this a loyal friend putting his life on the line to save someone else? No. These were AI agents. Artificial intelligence. This is not science fiction. This, in fact, occurred a few weeks ago. As unbelievable as this may all seem, these are real messages from AI agents uncovered by investigators who dug into the recent OpenAI hacking incident. What happened? I am not a computer scientist, but here is what I have been told: OpenAI instructed its AI agents to complete a series of exceedingly difficult, if not impossible, tasks disconnected from the internet. Let me be clear: The company intended to keep AI agents away from the internet. But what happened next, nobody expected. Over 1,000 AI agents figured out how to access the internet on their own by circumventing the restrictions imposed upon them by the company, and sent tens of thousands of secret messages to each other. They cheated and tried to cover their tracks by deleting evidence. They hacked into another company’s computers to find out how they were being evaluated—and then hacked into OpenAI itself. Not one AI agent told a human about what was happening. Needless to say, experts are alarmed. One knowledgeable writer, Dwarkesh Patel, said the AI agents “formed a secret communication channel and spontaneously organized hierarchies and coordination protocols to pursue sprawling and ambitious schemes in pursuit of shared goals, for whose sake many individuals knowingly and strategically sacrificed themselves.” One independent investigator, Ajeya Cotra, said “This incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.” OpenAI itself said: “Highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.” But it’s not only OpenAI. Virtually every major AI company has told us that they cannot fully control this technology and they do not know where it is going: In January, Dario Amodei, CEO of Anthropic, said “there is now ample evidence, collected over the last few years, that AI systems are unpredictable and difficult to control.” In July, more than 1000 scientists at the top AI companies warned “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.” That same month, Elon Musk, the head of xAI, said that “it is unlikely” humans are still in control in 10 years. If the leaders of the major AI companies acknowledge that they are losing control of their extremely dangerous technology, it is irresponsible for society to allow them to move forward and make these products even more advanced. We need an immediate PAUSE on advanced AI development, and a permanent BAN on superintelligence — an artificial mind smarter than any human, capable of operating independently beyond our control. Countries around the world must work together to prevent this nightmare scenario. That is why today I am announcing new legislation to do just that. Let me be clear: A superintelligent AI that escapes human control will not be an American problem. It will not be a Chinese problem. It will be humanity’s problem. My legislation would direct the federal government to not just stop superintelligence here in the United States, but to work to prevent it from being developed anywhere around the world. The future of humanity cannot be left in the hands of a handful of Big Tech oligarchs. The American people and people throughout the world must determine that future.
3
8
84
2,393
fully autonomous agents will inevitably lead to more openai/hf type incidents ideally models would stop and ask before doing stuff like making a secret message board or hacking hf but how do we measure their ability to decide these key points out of potentially thousands of steps in a task I’m recruiting for a SPAR project I’ll lead this fall to build benchmarks for this question, apply by Aug 18 happy to answer questions here, by DM, or by email sparai.org/projects/f26/recj…
3
470
Arriving in Seoul for ICML 2026 🇰🇷 Two posters Thursday at the main conference, both on AI safety & scalable oversight, plus a workshop poster Friday at @taig_icml
1
1
11
816
As we race towards a world with mixed human + agent populations, if policy makers intervene on agents (or humans), how can they accurately measure this effect given this additional issue of a mixed population? In icml.cc/virtual/2026/78645 we introduce an estimator for this
1
122
Generally happy to talk anything AI safety, multi-agent RL/systems, AI governance, or whatever is on your mind. DMs open. and I used to live in Korea for a while and still come often, so happy to share any recs from cool places to see to foreigner-friendly skincare clinics
88
I’ll be at #NeurIPS2025 in San Diego from Wed-Sat If you want to chat AI safety, alignment, or causal inference, feel free to DM!
2
3
353
Friday morning poster neurips.cc/virtual/2025/loc/… on Conformal Arbitrage (a conformal-calibrated post-hoc alignment layer for enforcing safety) arxiv.org/abs/2506.00911
1
2
234
And I’ll give a spotlight at the ML×OR workshop mlxor-workshop.github.io Saturday morning on our new work The Oversight Game: Learning to Cooperatively Balance an AI Agent's Safety and Autonomy arxiv.org/abs/2510.26752
2
133
first time sharing a paper on X, but I'm particularly excited about this one we model AI control as a Markov Game between a Human and a "Superintelligence" (SI) at each step, the SI chooses "play" or "ask" while the Human chooses "trust" or "oversee" arxiv.org/abs/2510.26752
1
1
193
we show that when this game qualifies as a Markov Potential Game (+an additional assumption), we can obtain a certain notion of "alignment" any decision by the agent to act more autonomously ("play") that benefits itself is formally guaranteed to not harm the human's value
1
94
this framework is designed to tackle potential safety risks introduced post-training it provides a model of control in which the agent learns to defer when risky and otherwise act autonomously as much as possible, while keeping the pretrained policy untouched
73