My utility as an individual AI researcher will be gone in a year. Writing math proofs will be automated even sooner. It was never about proofs. Science is about curiosity, creativity, and compression - via @SchmidhuberAI *Mathematicians Should Become Formal Theory Influencers* Mathematicians were the original influencers: they created conjectures and theories that others understood, cited, and built on. Difficult proofs often reflected deep understanding. But the proof itself was rarely the point. Frontier AI can already generate informal proofs much faster than humans can carefully inspect them. The scalable response is machine-checkable mathematics in systems such as Lean. AI-generated formal proofs may increasingly resemble compiler-generated machine code: essential, but rarely worth reading or writing by hand. That shifts mathematical work up the abstraction stack. Mathematicians become formal-math software engineers, working with AI on clean definitions, interfaces, key theorems, and reusable libraries rather than low-level proof details. Call everything derivable from a formal library within a fixed compute budget its mathematical halo. Humans and AIs should compress clusters of results into insightful, useful theories that expand the halo further. A great theory compresses many results, makes future results cheaper to derive, and exposes structure that humans and machines can understand and build on. That is why minimum description length is a useful lens on mathematical insight. Frontier math will eventually require huge compute budgets. At that point mathematics starts to resemble experimental science: scarce computational instruments, with researchers deciding which problems deserve them. Mathematicians and AIs should create conjectures and theories that expand the mathematical halo. Mathematics should become an enormous machine-checkable software project. *Become a Formal Theory Influencer* Inspired since 2010 by: people.idsia.ch/~juergen/art…

We don’t know how to reliably align frontier AI models. Alignment may get harder, not easier, as models become more capable. But maybe AI control can scale faster than dangerous capabilities. Let’s build a Great AI Firewall - without building totalitarian control of society. *Alignment is Dead (for now), Long Live Control!* We do not know how to specify or guarantee robust alignment of sufficiently capable models. Instead we should treat these models as untrusted. Optimizing models against imperfect reward signals may increase the propensity for reward hacking, deception or scheming. Models can also be jailbroken or fine-tuned away from their safeguards. The central bet of AI control is that our ability to constrain an AI’s access to the world may scale faster than our ability to understand or align what is happening inside it. Where possible, make verifying an answer easier than generating it. Formally verified software and hardware are unusually attractive: let AI produce the implementation and a machine-checkable proof that it satisfies the specification. This does not solve specification: the specification and verification stack remain part of the trusted base. Every consequential external action should pass through an independent control layer: code execution, network access, calling APIs, accessing physical systems - even communication with users. Optimize control monitors for extremely high recall of dangerous actions, even at the cost of rejecting many safe actions. Suspicious actions should be rejected or escalated. Every major increase in dangerous capability should require evidence that control still works. Chain-of-thought monitoring is valuable while it works, but it should not be foundational. Models can learn to obscure their reasoning and may increasingly reason without fully verbalizing it. Open weights shift much of control from the model developer to whoever deploys the model. Technical control is only useful if dangerous deployments cannot cheaply route around it. We should approach AI control like cybersecurity. The goal is defense in depth: make successful catastrophic attacks sufficiently difficult, expensive and rare. Call this technical and institutional architecture the Great AI Firewall: the boundary between untrusted frontier intelligence and consequential real-world power. The name is deliberately provocative. China has substantial experience building large-scale technical control infrastructure. That may create some common ground for international coordination. But the analogy is also a warning. AI control must not become control of society. The goal is to constrain dangerous machine capabilities - not human speech, actions, or ordinary access to information. Controls should scale with capability and risk. Ordinary models should face ordinary constraints. More consequential capabilities justify stronger controls. The objective is the minimum control necessary to keep catastrophic risk acceptably low - not maximum control for its own sake. Firewall the AI, not society.