National security in the age of AI is incompatible with the level of international conflict we're used to. I think controlling this technology will ultimately call for nothing less than the end of war. In the final installment of my five-part essay "The Singularity Is Clear," just posted on @NonzeroNews, I elaborate: Part V: The Epic Challenge of International Governance For years the term “warning shot” has had a special meaning within the AI safety community. It refers to a high-profile event that could focus attention on the catastrophic potential of artificial intelligence and thus on the need for preventive measures. Typically, the envisioned event involves significant death and suffering. “Chernobyl-sized” is the way some put it, referring to the 1986 nuclear accident that killed dozens of people in the short run and an estimated tens of thousands via cancer in the long run. Others imagine a higher death toll—maybe millions—as the necessary price for enlightenment about the dangers of AI. Against that backdrop, some of the most unsettling AI stories of recent months look like good news. We learned in April about the superhacking abilities of Anthropic’s Mythos, and three months later came the Hugging Face story—about the swarm of OpenAI agents that escaped their “sandbox” and then broke various other rules. Neither of these developments, so far as we know, entailed any deaths, and both significantly expanded awareness of AI risk. Indeed, in the wake of the Mythos story, President Trump—a skeptic of international governance, to say the least—agreed with Chinese leader Xi Jinping to initiate a formal dialogue on AI safety between the two countries. And the aftermath of the Hugging Face incident saw not only the “pacing the frontier” letter, but a subsequent statement by Dario Amodei embracing—in principle at least—the idea of a slowdown. Still… there’s a ways to go. It’s great that the phrase “international coordination” is now more common in the context of AI, but even the more optimistic observers have modest expectations about the fruits of US-China dialogue. The wildest commonly expressed hope is for the eventual emergence of something comparable to a US-Soviet arms accord during the Cold War—a narrow agreement about specific capabilities that doesn’t involve very intrusive monitoring and doesn’t much alter the fundamentally adversarial nature of the relationship between the two nations. That would be progress, but not as much progress as the US and China, and the whole world, are going to need in the long run. This technology is only beginning to demonstrate its power—or, more to the point, its powers. Given the number of dimensions along which its capability will grow, and the novel nature of the policy challenges it will pose, preserving international and intranational stability will demand much more thorough and creative policy collaboration between the US and its superpower rival than materialized during the Cold War. No issue more clearly highlights the degree of innovation in governance that AI will eventually demand than the issue of “open source” (or, strictly speaking, “open weights”) large language models—models whose internal parameters are publicly available… [You can read the rest of Part V at nonzero.org, and you can read the whole essay, from the beginning, at thegodtest.net/singularitycl…]

NEW: Here’s a clue about these mystery books: Papa may be California’s next governor. I’m Years ago, Kitty Felde launched young adult novels with a lead character modeled after a girl she mentored. For the dad, she depicted an ex-LA congressman much in the news lately…

This morning on the DC beltway! #nokings