Excited to share a new working paper, joint with 25 interdisciplinary coauthors: “The Feasibility of a Hardwired Pause of Frontier AI Training" 63% of voters in a recent WSJ poll want to pause AI development. But can it be done globally? How long could it last? 🧵

I'm in AI Frontiers today arguing that countries and companies need to wake up to AI-driven takeover of outer space. Due to the compounding returns of being the first mover and tapping into massive infrastructure in space, a winner-take-all outcome is looming. Rival states, who lag in launch capacity as well as frontier AI, would have an incentive to strike first, before such a pioneer establishes enduring hegemony. Here's why the first-mover advantage could be so durable: First movers in space will benefit from an industrial flywheel, wherein space-based solar power enables more compute and the construction of yet more space-based solar and thus more compute and furthermore. Moreover, many of the engineering problems that make spacecraft fragile and costly today—and which cause people to instinctively dismiss rapid space takeover as science fiction—will be solved using AI-accelerated engineering. Brutal present-day latency in robotic operations (e.g., a radio signal can take 40 minutes to and from Mars...possibly for a single instruction) will be circumvented by on-board intelligent operation (i.e., full self-driving for orbiters, landers, and rovers). The result is that a lead in AI capabilities could have huge, huge industrial returns for an actor that first manages to spin up infrastructure in space. That includes the orbital analogs to workshops and shipyards. Laggard countries and companies simply will not benefit from these dependencies, such as orbital factories and depots for assembling structures that wouldn't fit into one rocket fairing (e.g., too wide for the hold of Starship) and which therefore would be infeasible for rival states to produce. And herein lies the wicken security dilemma: The optics for laggard nations are awful. As in MAIM, a country just isn't used to thinking about kinetically striking foreign soil merely to stop a commercial computing operation or, in this case, because some really cool and innovative company is collecting energy and putting lasers and robots into space. It feels wholly inappropriate to exert violence on a bustling business. But such a bustling business is exactly what we will see as one actor (whether private or government) is spinning up history's largest industrial flywheel—extraterrestrial infrastructure providing energy, compute, and robotic manipulation to carry out all its other ambitions in space. Finally, the holy grail of space expansion will be to launch von Neumann probes or specialized development convoys into space to establish colonies and proliferate themselves. In other words, a space virus. Hmm...how could this ever go wrong? 1) If these AI envoys are aligned to the interest of just one man, other countries had better hunt that shit down. 2) If these AI envoys are hastily, sloppily developed and thus misaligned with civilization overall, we had better hunt that shit down. Nation states must wake up, because there may be a point of no return.