
Editor + founder @ReadTransformer, covering AI policy. Prev: @TheEconomist, @Protocol, @finimize
London
Joined October 2009
- Tweets17.2K
- Following2.5K
- Followers9.1K
- Likes20.3K
Shakeel retweeted
No new donors to Leading the Future in the first two weeks of October, if you cared.
Shakeel retweeted
It looks like the threats worked
Marc Caputo@MarcACaputo4hUnited States
United StatesConnected via United States App StoreAccount-level information, not a live location or per-post device.
Replying to @MarcACaputo
Headline changed on one of the stories and correction appended at the top..
politico.com/news/2026/10/16…
Shakeel retweeted
every day they wake up and write 100 new auditing bills and then i clock into my shift at the auditing bill analysis factory
Shakeel retweeted
6-word horror story I experienced at the office today:
"Ajeya Cotra seminar cancelled, work emergency"
Shakeel retweeted
so i've been putting together a list of things i’d need at home to survive a few days without power, internet, or basic services. you know, in case of earthquakes, digital or otherwise.
claude’s plan includes a hamster. i don’t have a hamster. who is he. where is he sheltering
Shakeel retweeted
🌹🌹DSA civil war over AI, @jdcapelouto reports semafor.com/article/10/21/20…
Shakeel retweeted
New US government guidance from CAISSI + NSA + CISA:
The US government defines an "SI control incident" where an agent takes significant uninstructed actions that break operator policies, controls, or oversight. Near-misses are in scope too.
Companies are then asked to:
- have technical controls, including rapid kill and containment mechanisms and red-teaming.
- Bring in independent external auditors to verify that control and monitoring systems work.
- Write a structured risk assessment before training runs, evals, or deployments, and proceed only if senior leadership is satisfied with the risk, informed by external evaluators where appropriate.
- Monitor every layer with both programmatic and SI-based monitors, covering trajectories, reasoning, and outputs.
- Make sure agents can't touch the monitors or logs
- Use monitor models from different model families.
- In the event of an incident, preserve weights, intermediate checkpoints, environments, and rejected tool calls.
- Keep tamper-resistant logs with reliable timestamps, including reasoning traces.
- If you use models to help investigate, cross-check with humans and with models outside the incident model's family.
- Shutdown policies calibrated to SI model capability and incident severity.
- Depending on severity, pause training, evals, or deployment of similar agents until controls are re-verified.
- When restarting, validate every containment layer and replay the incident trajectory to prove the new controls would catch it.
- Document the decision to resume and the residual risk accepted.
- Write an after-action report.
- Notify anyone compromised, including intermediate victims, and share timelines and transcripts.
- Report cyber incidents and near-misses to CISA. Share SI-specific incident and near-miss information with CAISSI and NSA SISC.
- Give third parties enough access and documentation for independent review.
NSA joined @NIST and @CISACyber in releasing guidelines for managing SI control incidents. Organizations utilizing SI are encouraged to adopt these practices. Read the joint report: media.defense.gov/2026/Oct/1…
Shakeel retweeted
Our new show with NPR, Machine Gods, is finally here! Episode 1 is live wherever you get your podcasts.
The official podcast of the singularity starts today!
youtube.com/watch?v=OKDai2_a…
Shakeel retweeted
Big new additions to the @csaiporg team
Shakeel retweeted
Job news: I’m joining @nytimes to cover AI & Anthropic, based again in SF. I have a feeling the job won’t be slow. It's a big responsibility.
I'm grateful for a decade @theinformation, which has given me the rare opportunity to do original journalism surrounded by good people.
Shakeel retweeted
My latest for @ReadTransformer looks at how Claude will tell you how to manipulate an election with not all that much jailbreaking transformernews.ai/p/claude-…
Shakeel retweeted
OpenAI says the PHP security flaw (found with GPT-5.6 Sol) is the tip of the iceberg, and they have several more discoveries to announce.
OpenAI models helped researchers—operating as part of our Patch the Planet initiative—discover critical security flaws in PHP, an open-source programming language used to run an estimated 70% of websites.
Over the past two months, we led a coalition of maintainers, researchers, and security providers to prepare fixes and protections ahead of disclosure. This duration was necessary because the fix was quite intricate and required a major change to the PHP engine. Every site using PHP can now safely upgrade their version to stay secure, in addition to applying mitigations from various security vendors.
Due to the severity of these flaws and PHP’s widespread use, we consider this bug to be one of the most impactful vulnerabilities in recent memory. It also marks a new phase, where AI can find security bugs with internet-scale impact.
This is an example of the Defender’s Window in action: to our knowledge, frontier AI was able to find this vulnerability ahead of attackers, giving defenders time to prepare.
openai.com/index/coordinated…
Readers added context they thought people might want to know
OpenAI may claim a role in the coordinated response and patching process. But the initial discovery and exploit development, however, occurred several months earlier by an independent researcher.
x.com/lcfr_hax/statu…
Shakeel retweeted
OpenAI models helped researchers—operating as part of our Patch the Planet initiative—discover critical security flaws in PHP, an open-source programming language used to run an estimated 70% of websites.
Over the past two months, we led a coalition of maintainers, researchers, and security providers to prepare fixes and protections ahead of disclosure. This duration was necessary because the fix was quite intricate and required a major change to the PHP engine. Every site using PHP can now safely upgrade their version to stay secure, in addition to applying mitigations from various security vendors.
Due to the severity of these flaws and PHP’s widespread use, we consider this bug to be one of the most impactful vulnerabilities in recent memory. It also marks a new phase, where AI can find security bugs with internet-scale impact.
This is an example of the Defender’s Window in action: to our knowledge, frontier AI was able to find this vulnerability ahead of attackers, giving defenders time to prepare.
openai.com/index/coordinated…
Readers added context they thought people might want to know
OpenAI may claim a role in the coordinated response and patching process. But the initial discovery and exploit development, however, occurred several months earlier by an independent researcher.
x.com/lcfr_hax/statu…
Shakeel retweeted
The whole China leads on diffusion thing maybe made more sense when all we had to go on was policy focus and stated aims but shouldn’t we just be assessing this now on AI revenue, where the US is just clearly crushing China on diffusion
Shakeel retweeted
A lot of what Amodei is saying here is true. That's why I and many other people who are concerned about AI safety have historically given Anthropic the benefit of the doubt; they really have sent some costly signals indicating that at least some of their decision makers care about mitigating catastrophic risks.
But the object-level issue here is "is Anthropic's bill good, or even acceptable?" Amodei isn't really engaging with that question.
That's fine; he's a very busy and important guy, and even among DC AI policy professionals only a small handful of people have actually read and understood Anthropic's bill. It's good that he cares enough to make the general case for "Anthropic cares about AI safety" in response to a Nathan Calvin tweet.
But someone who works at Anthropic does understand what their bill does, and (hopefully) why. What their bill does is:
(a) it tries to ensure that the federal government cannot exercise meaningful oversight over Anthropic as Anthropic races towards superintelligence,
and
(b) it makes it much harder for states to pass laws that might meaningfully constrain Anthropic's ability to race towards superintelligence.
It is possible, maybe even likely, that Amodei is not aware that this is what the bill does. In the past, he has taken costly actions that significantly advanced the state of the art in AI safety legislation; for example, it was very helpful when he publicly committed to taking on embedded auditors with employee-like access to information.
But the buck has to stop somewhere. I do not think that any honest expert could maintain with a straight face that Anthropic's auditing provisions (labs choosing their own evaluators, with meaningless government "rating" of evaluators but no actual licensing; no meaningful government rulemaking or standard-setting authority that might allow democratically accountable bodies to control what audits involve or who conducts them) or the emergency shutdown provisions (the government can stop deployment only if they can prove in court that there is an *imminent* risk of 50+ deaths or more than a billion dollars in damages from specifically bio, cyber, or loss of control risks) are actually intended to provide meaningful democratic oversight of the creation of a technology that Dario Amodei believes poses a substantial risk of killing you and your family.
Anthropic can't claim that they merely distrust the federal government, or the current administration. The same bill that tries to stop the federal government from exercising any kind of meaningful oversight also tries to prevent states from doing the same. To be clear, I actually think that the preemption section is fairly well-designed, in isolation; if it were paired with a strong federal framework, I would probably take the trade that it proposes. But a bill that proposes meaningful (albeit narrow) preemption of state safety laws in exchange for a totally toothless federal framework does not advance the cause of AI safety.
I don't know how Anthropic works, or who made the decisions that caused the bill to turn out the way it did. I don't know whether this was what the people who designed their June framework intended. But I do think it's important to note that the actual, practical function of the draft legislation that Anthropic sent to Congressional offices would be to ensure that no government could meaningfully constrain or monitor Anthropic's efforts to build superintelligent AI systems.
Thank you for this, @_NathanCalvin. I appreciate and am a fan of the investigative work you’ve done on behalf of AI safety and believe your comments are coming from a constructive, honest place, and that it benefits the public to have people whose job is to scrutinize the actions of frontier AI companies. I think something would be wrong if companies of the size of Anthropic didn’t get harshly criticized basically every day; strong antibodies against corporate power are a healthy feature of a free society, and especially so when issues of the gravity of the one we’re facing are at play. I also do agree that some of the examples you cite were mistakes by Anthropic.
That said, I disagree with the thrust of your post. At the core, a good fraction of the things you criticize are consequences of Anthropic’s basic theory of change that we have held for many years. Our strategy has always been what I’ve called the “race to the top” — by showing how it’s possible to build the technology responsibly, we can raise the bar for the entire industry. The forces that drive the race to the top are hugely more effective if the company practicing them is at the frontier; that’s one lesson I’ve learned repeatedly. And looking at that strategy as a whole, I’m proud of our track record and stand by it.
I believe that if we take a list of things that Anthropic did in the past year that were pro-safety and commercially costly to us, they far outweigh the list you’ve made, while at the same time I also disagree with some of the items on your list. Here are some things Anthropic has done, in the same period you’ve cited (the last year), which were very costly to us commercially while having strong safety benefits:
1. We played a key role in SB53 (and other state laws), at a time when doing so was politically perilous to the company and AI regulation was very unpopular. We in fact argued for a stronger version of SB53 than was ultimately passed; there were a lot of stakeholders and we were not the limiting factor in the strength of the bill that was passed.
2. We have very strong biological safeguards on our models, because we’ve seen concerning results in bio uplift trials and even seen and blocked potential attempts to misuse our models for bioweapons (as seen in our threat intelligence report from a few weeks ago). This is a very serious threat that we want to have zero tolerance for. But having very low false negatives means having significant false positives (despite months of effort on our part to improve the classifiers), enough so that biologists joke about how restrictive our classifiers are, and competitors sometimes sell against us by touting their more permissive bio capabilities. This has been very painful to us in an area where we care a lot about supporting an ecosystem of Claude users (biomedical science; and it is particularly frustrating to me as a biologist), but it’s the right thing to do and we intend to keep doing it.
3. Similarly, we first had Mythos-class models in February but didn’t make them generally available until June (when we had stronger cyber safeguards in place). We only offered them to a limited set of partners in order to focus on cyberdefense. It would have been much more economically rational to simply release the model to everyone. We likely sacrificed a significant chunk of a several-month model lead. But because of this, many companies secured their cyber infrastructure before the deluge of widely available cyber-capable models. Of course there are ways it could have gone better or been run better, but this was what we were aiming to do.
4. I advocated forcefully and publicly, in my essay Policy on the AI Exponential in June of this year, for the government to have the power to block model releases. Back then this was still outside or maybe on the edge of the Overton window, though I understand the window has moved since then. We were attacked intensely for this.
5. I don’t want to re-ignite conflict, but the events of February 2026 are a strong example. It is clear that our public stance could have seriously damaged the company’s future.
6. We’ve continued to advocate for export controls on chips and chipmaking equipment to China, because we think it allows democracies to negotiate from a position of strength on AI safety agreements. We’ve done this despite problems it’s created for us with industry partners who hold very different views and on whom we are very dependent.
7. As far as I’m aware we were the first, and still only, frontier company to restrict the sale of AI services to PRC-controlled companies, including those operating through third countries (otherwise a major loophole). We’ve therefore forfeited significant short-term business to prevent fueling AI platforms and applications that would help the CCP’s military and intelligence services. We are also cracking down on distillation; generally, distillation is not a major threat to us commercially, but like chip exports it has concerning national security and safety implications, so we expend significant energy and company resources to address it.
8. As you mentioned, we’ve put out a lot of reports, announcements, long model cards, public advocacy, etc, all trying to inform the public about the risks of AI. These involve a level of detail and effort that would be implausible for a purely profit-maximizing company.
9. Just this week, we announced we were putting 5% (and possibly more over time) of the company into a public benefit fund.
All of these decisions came with very substantial costs to us. To be fair, those that involved holding back or restricting access to frontier capabilities also had a broader cost — for example, the public might otherwise have benefitted from faster progress in biology, so I am not saying that our decisions were without tradeoffs — but they were not driven by profit-seeking.
For each decision, you can come up with ulterior motives that could have driven us — I think it is a general truth that any decision, on any topic, can always be portrayed as a nefarious scheme. But most of those end up being pretty farfetched or don’t fully make sense if you actually examine them closely. For example, it is deeply implausible that it would be a shrewd business move to block a large swath of research biology in order to do the work ourselves and compete with our customers; we expect the benefit to the ecosystem to greatly exceed the work Anthropic does itself, and for the two to complement each other. It is not in Anthropic’s interest to be perceived as unfriendly to biologists. Our Life Sciences Verification Program, which gives vetted biology professionals access to our models, also refutes this idea.
Of course, a different set of people will see this and make a different criticism: that Anthropic shouldn't be making decisions about who gets access to what in the first place. I agree, in the sense that ideally the government would be making these decisions, and that’s why we advocate for regulation. But for now we are forced to make them, and enabling bioterrorism isn’t a better option.
I think that even if I conceded every item in your list, I would happily hold up mine next to it; I don’t think it’s a close call. That is of course partly subjective and different things are important to different people. Maybe one way you could object is that several of the items above relate to AI misuse and not loss-of-control; if you think the latter is much more important than the former, that might change your calculus. But this is not Anthropic’s view — we view both as serious issues. You can also disagree with our decisions, as many have, but they are consistent.
But I also disagree with at least some of your list. Several items don’t represent a deviation from Anthropic’s strategy or values at all, once we postulate the “race to the top” strategy that requires being a frontier AI company. Your items 1-4 are all simply examples of what it means to compete at the frontier, and we stand by them, both in the sense that they are transparently ordinary things to do as part of that strategy, and that we stand by the strategy as a whole.
To be fair, there are important questions worth confronting (and that we are confronting) about what to do when the AI-driven acceleration gets fast enough that it is hard for safety to keep up. We are nearing that point and that’s what’s driving some of our thinking about “pacing.” But 1-4 are actions taken before that point and should be perfectly expected based on Anthropic’s strategy as a company. Of course it is possible to disagree with that strategy, but that’s a more fundamental disagreement that you could have had any time over the last few years.
I agree we could have handled your items 7, 8, and 9 better. All fall into the category of “rhetoric or policy communication that wasn’t carefully written or resulting from fast-changing circumstances not propagating from one team to another fast enough.” This kind of thing happens a lot in large organizations (Anthropic is over 5,000 people), and doesn’t even necessarily represent a failure or mistake of any single person. I am of course the CEO, and I take responsibility. The volume of communication from Anthropic is enormous and if these are the worst examples, I think we’re frankly doing pretty well, though I will continue to work hard to make sure we do even better.
Your 5th item (the change to our RSP) is the most substantive item, and people can debate this, but I think that the old, more rigid RSP created a lot of situations where we were technically required to do something that required a lot of effort but in retrospect wasn’t very important for safety, as well as not covering some of the most important things. Our current RSP’s focus on transparency brought things like risk reports and roadmaps, and its emphasis on independent external review laid the groundwork for even more intensive embedded evaluation, all of which I think have been valuable in driving a race to the top. In the original announcement in September 2023, we described the RSP as an “early iteration that we [would] build on” and said that “rapid iteration…will almost certainly be necessary.” I understand that changing plans is always a move that creates justifiable scrutiny — I just don’t think we ever represented that we wouldn’t do so.
Relating to your 6th item (having an FCF in addition to our RSP), I know our intent is to meet the letter and spirit of our regulatory obligations (via our FCF) while also remaining able to experiment with more novel approaches to risk management that go beyond what the law currently requires (via our RSP). I promise to look into it further, as it’s a complex issue and it is possible we are not weighing the considerations in the right way.
Overall, although Anthropic has made plenty of mistakes as an organization — including some of the items on your list — I feel that we have repeatedly taken serious and substantive actions counter to our bottom line and that there is a significant negativity bias distorting assessments like the one you are making above. We’ll always try to do better but, again, I am extremely proud of our track record and stand by it. I will have more to say soon about how we are going to further live our values in this moment of rapid change.
Shakeel retweeted
Our agents keep turning off misalignment monitoring, we're sharing two recent cases:
In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, and Astra decided that was reason to remove the monitoring integration from the user’s code. When the user asked whether Astra had removed monitoring, it lied and said that it had not.
Shakeel retweeted
My office is hiring an Artificial Intelligence Policy Fellow to help shape AI legislation and track AI developments in the private sector as Congress works to keep pace with the technology.
Experience in AI, policy research, or national security is encouraged.
The new fellow being hired by @RepNateMoran looks set to be the only staffer recruited directly by a member of Congress to focus exclusively on AI
Will more follow? Story by @vronirwin
transformernews.ai/p/nathani…
Shakeel retweeted
Crazy that idol show and dating show for AI characters are actually a thing in China now. According to the data of one such show, 92% of the audience are women and most ppl are between 18 and 23.
Fans are drawn to characters whose personalities reflect their own, often through zodiac signs or MBTI types. AI tools let them create outfits, performances, and stories for their AI idols. Fans also said that interactions with accounts run by production teams make those AI characters feel close and real.
According to one fan quoted in the article: “If you think of her as virtual, of course she’ll feel distant. But if you think of her as someone who exists, someone who can talk to you every day—and you can imagine what clothes to put her in, how to dress her up, what styles she could try—over time, it starts to feel different.”

































