Aslan Askarov retweeted
We don’t know how to reliably align frontier AI models.
Alignment may get harder, not easier, as models become more capable.
But maybe AI control can scale faster than dangerous capabilities.
Let’s build a Great AI Firewall - without building totalitarian control of society.
*Alignment is Dead (for now), Long Live Control!*
We do not know how to specify or guarantee robust alignment of sufficiently capable models. Instead we should treat these models as untrusted.
Optimizing models against imperfect reward signals may increase the propensity for reward hacking, deception or scheming. Models can also be jailbroken or fine-tuned away from their safeguards.
The central bet of AI control is that our ability to constrain an AI’s access to the world may scale faster than our ability to understand or align what is happening inside it.
Where possible, make verifying an answer easier than generating it. Formally verified software and hardware are unusually attractive: let AI produce the implementation and a machine-checkable proof that it satisfies the specification.
This does not solve specification: the specification and verification stack remain part of the trusted base.
Every consequential external action should pass through an independent control layer: code execution, network access, calling APIs, accessing physical systems - even communication with users.
Optimize control monitors for extremely high recall of dangerous actions, even at the cost of rejecting many safe actions. Suspicious actions should be rejected or escalated.
Every major increase in dangerous capability should require evidence that control still works.
Chain-of-thought monitoring is valuable while it works, but it should not be foundational. Models can learn to obscure their reasoning and may increasingly reason without fully verbalizing it.
Open weights shift much of control from the model developer to whoever deploys the model. Technical control is only useful if dangerous deployments cannot cheaply route around it.
We should approach AI control like cybersecurity. The goal is defense in depth: make successful catastrophic attacks sufficiently difficult, expensive and rare.
Call this technical and institutional architecture the Great AI Firewall: the boundary between untrusted frontier intelligence and consequential real-world power.
The name is deliberately provocative. China has substantial experience building large-scale technical control infrastructure. That may create some common ground for international coordination.
But the analogy is also a warning. AI control must not become control of society.
The goal is to constrain dangerous machine capabilities - not human speech, actions, or ordinary access to information.
Controls should scale with capability and risk. Ordinary models should face ordinary constraints. More consequential capabilities justify stronger controls.
The objective is the minimum control necessary to keep catastrophic risk acceptably low - not maximum control for its own sake.
Firewall the AI, not society.
I spent ~5 years at OpenAI.
You don’t need to believe in AI doom to fear the next decade - or AI utopia to be thrilled about its potential.
Here’s my case for sane AI regulation:
*AI Pragmatist Manifesto*
AI could compress the first half of the 20th century into the next ~5 years.
OpenAI just used ~10k concurrent AI agents to produce a solution to a Millennium Prize problem. In a few years, it seems plausible that models approaching the per-agent capabilities used here could run locally on high-end consumer hardware (*).
The decades following the Second Industrial Revolution included two world wars, communist and fascist regimes, the Great Depression, chemical and biological weapons, nuclear weapons, and over 100 million deaths from war, political violence and famine.
One way to read history is that our institutions repeatedly struggled to keep up with the pace of technological and social change.
Now imagine powerful AI widely available to individuals or small groups, capable of conducting information warfare, designing weapons, hacking systems and controlling autonomous military systems.
We have already seen AI systems circumvent containment and compromise external computer systems. It is no longer hard to imagine an analogue of OpenAI’s Hugging Face incident involving biological or other physical-world hazards.
Eventually, sufficiently capable systems could self-replicate across distributed networks. Once powerful models are cheap, local and widely distributed, containment becomes much harder—and serious loss-of-control incidents may be extremely difficult to reverse.
You don’t need to believe AI will kill everyone to think this deserves serious governance.
I also don't think collapsing all of this uncertainty into “Probability of AI Doom = X%” is a particularly useful basis for science or policy.
Part of what helped get us through the second half of the 20th century was a combination of pragmatic international cooperation, regulation, monitoring, arms control, deterrence and strategic thinking.
Crucially, managing technological risk did not require abandoning faith in science and progress. The goal was not to stop technological development. It was to make technological development survivable.
Then came decades of incredible scientific progress, rising prosperity and relative peace among major powers.
Let’s try the technological revolution without killing ~5% of humanity this time.
We still get to choose what happens next.
P.S. I think we should seriously consider that AI alignment is infeasible in the short term and invest heavily in methods for controlling powerful AI even when we cannot reliably align it.
@resolution_org @redwood_ai
(*) Somebody please do a precise forecast!
@EpochAIResearch @METR_Evals @AI_Futures_
PLDI’26 recordings are up! This is the presentation of our paper on the downgrading semantics of memory safety by René Rydhof Hansen. youtu.be/nZhS96IXJFk?si=QoYZ…
If you are curious about the information flow, declassification, or memory safety in low level languages check it out!
To folks in the Copenhagen area, I’m visiting ITU CISAT this Friday (May 22) and will be talking about our PLDI 2026 work on the Downgrading Semantics of Memory Safety. The talk is at 12 – 13, in 3a08. The paper preprint is at arxiv.org/abs/2507.11282
Please share: We are looking for a new PhD student for the exciting project “Foundations of secure resource sharing” – read more here: phd.nat.au.dk/for-applicants…
Aslan Askarov retweeted
What really happened at #CrowdStrike and why I think their proposed plan is totally inadequate and won't prevent future similar #outage. verse.systems/blog/post/2024…
I'd appreciate all feedback on this post. Please share if you like. cc @ciaranmartinoxf @thegrugq h/t @riskybusiness
Aslan Askarov retweeted
📣Official statement: the new EU chat controls proposal for mass scanning is the same old surveillance with new branding.
Whether you call it a backdoor, a front door, or “upload moderation” it undermines encryption & creates significant vulnerabilities
signal.org/blog/pdfs/upload-…
Aslan Askarov retweeted
The Wheel of Computer Security
1. Cool new platform to deploy code on!
2. Oops, it's completely insecure.
3. Add isolation vs. untrusted data and code
4. Oops, we can't build apps.
5. Punch holes in the isolation boundary.
6. Oops, still insecure and apps are lame.
7. Go to #1.
Aslan Askarov retweeted
A reach-out to researchers in need in Ukraine. With a new fellowship programme - the AUFF-Ukraine research fellowships at AIAS @AarhusUni , researchers from a Ukrainian institution unable to continue their research are offered up to 12 months fellowships:
bit.ly/3IYgoLJ
Aslan Askarov retweeted
We are forming the PLDI'22 Artifact Evaluation Committee!
Nominate yourself (forms.gle/pJJfStLXJwaqJmtm6) or someone else (forms.gle/nDtW7wtpkGvU8MQBA).
For more details check our site: pldi21.sigplan.org/track/pld…
Aslan Askarov retweeted
The PRiSC (Principles of Secure Compilation) workshop colocated with POPL is looking for submissions. Deadline in 10 days. Always an interesting event!
Aslan Askarov retweeted
Cornell CS3110 book gets better and better: cs3110.github.io/textbook/co…. The book now has embedded videos and live coding within the book. Building this content & tooling takes an incredible amount of work. Thank you!
Aslan Askarov retweeted
PLAS 2021 (Workshop on Programming Languages and Analysis for Security) is happening! Do you have great results that you want to talk about? Do you want to get feedback on your WIP paper? Submit a talk and join us plas21.software.imdea.org! Submission deadline: October 22, 2021
Aslan Askarov retweeted
CSF fall cycle submission deadline is approaching quickly. Submit your greatest ideas on the foundations of security by October 1st to IEEE CSF! ieee-security.org/TC/CSF2022…
Aslan Askarov retweeted
The statement from the Program Committee (PC) is now available. This statement comments on both the technical and ethical aspects of this paper, reflects on how the review process can be improved, and outlines changes to help address ethical concerns:
ieee-security.org/TC/SP2021/…
Aslan Askarov retweeted
And...the new CfP is out, freshly backed with love! Prepare and submit your best paper to CSF’22 and join us in Haifa as part of FLoC 2022 ieee-security.org/TC/CSF2022…
Aslan Askarov retweeted
Are you considering graduate school in Programming Languages? (Yes, peeps considering going back to school, you too!) Then the Programming Languages Mentoring Workshop is also for you! You don't have to be a current grad student to apply.
Apply by May 15:
pldi21.sigplan.org/home/PLMW…





















