Here's how I see the "doomers vs normal cybersecurity" discourse/situation: People who think reducing AI extinction risk should be an urgent global priority (sometimes called "doomers," but people sometimes read this as "people who think we're definitely doomed," so I'll say "the x-risk people" instead) have seized upon HuggingFace and similar incidents as evidence for their core claim, which is that AI agents will evade human control (and this becomes increasingly dangerous as the models get more powerful). So, note that this claim decomposes into: they will evade human control if given the chance (i.e. we won't "solve alignment"), and they will be given the chance (i.e. we won't "solve control"). Some cybersecurity-world people who disagree with the x-risk people (I guess I'll call them the cybersecurity-disagree-ers) think something like: they were given the chance this time, but that's because the companies had incompetent cybersecurity. They sometimes claim that the x-risk people only focus on alignment and neglect control; I've posted a few times about how this is not true. But the point is, they think the "solve alignment" part is a distraction, and the more important (or tractable) work is to "solve control," or really to implement the normal kinds of strict cybersecurity measures that the labs have thus far failed to do. As far as I can tell, they mostly do not dispute the "they will evade human control if given the chance" part. They just think it will be relatively easy to not give them the chance. I think it does seem possible to design sufficiently strong control measures to stop, or adequately contain, agent incidents for systems that are pretty significantly beyond human capabilities. But this does not reassure me that companies will in fact maintain control, for two reasons. 1) "Possible" doesn't mean it will definitely happen. Right now, as the cybersecurity-disagree-ers say, the frontier AI companies have woefully inadequate cybersecurity. But they seem to in fact have very little interest in solving this. Instead, their attitude toward the incidents has been to patch specific vulnerabilities very narrowly; to disclose basically as little as possible, usually only when their hand is forced by an external party (and a letter from senators did not suffice for this), until the Friday-afternoon news dump; and to briefly pause some aspects of training but mostly proceed with making more and more capable agents at breakneck pace without seriously rethinking their security posture. Or at least, I have seen no evidence from the frontier companies that these incidents have prompted a serious change in attitude that would be commensurate with the rate of incidents and the rapidly increasing capabilities of the agents. So insofar as the cybersecurity-disagree-ers are like, "the x-risk people think we're doomed, but we're going to be totally fine, even for much more capable agents, if the companies implement such-and-such control measures," I'm like: well, that's a big if! Are they going to do that? Voluntarily? Maybe liability will eventually force them to, but this will stop being an adequate disincentive once the damages become judgment-proof-scale. 2) Controlling systems "pretty significantly beyond" human capabilities does not go far enough. Even if they had adequate control now, we will soon be in pretty novel terrain. It does seem like they could do a much better job controlling thousands of agents with summer-2026 capability levels by implementing existing practices. Maybe they could get there within a few months. But where should they turn in, say, 2029, for techniques for controlling many millions of agents, each of whom might be more capable than the best human hackers, which instead of being poorly "sandboxed" are deployed commercially throughout the economy with access to the internet and all kinds of sensitive data? What about 3 years after that? Will the R&D for securing these models actually outpace their capabilities to evade controls, including as they themselves increasingly speed up AI capabilities research? Will implementation of the control measures also keep up? And how sure are you about that? To be clear, I think it's really, really great that cybersecurity professionals are turning their attention to this problem. I hope more of them do, and that they shame the companies into doing a better job, go to the companies and try to fix this situation, and/or contribute new ideas for solving the novel problems this creates. But I disagree with the vibe that this whole problem is easily fixed with a little bit of effort.