“Our experiments suggest that small AI models may be months away from being capable enough to self-replicate on commercial IoT devices, like smart ovens and off-brand Android TV boxes. A replicator like this would be challenging to shut down, since there’s no centralized point to shut off - though it may be slow (with a doubling time of days). Unlike non-AI malware, it could also figure out unfamiliar devices on its own. In our evals, a 9B model got inference running on 8 of the 9 hardware types we tested. Models of 30B parameters and under can already complete many of the individual steps for replication today, though they make mistakes when composing them end to end. In our simplified evals, GLM-5.2 and stronger models are capable of end to end replication. If small models improve at their historical rate, they’ll match this capability level by the end of this year. Small weight-based replicators may be possible soon after.” -@bucketofkets @willzhou97 anthropic.com/research/self-…

We are terminating Chris Degnan for unethical conduct involving Cognition. The last few months have seen incredible progress in AI capabilities. San Francisco has flourished as new companies that solve new, more ambitious problems are finding great success. Generally, it is a wonderful time to be building. We at @FactoryAI have seen overwhelming interest in our model-agnostic coding agents and our software factory product. Our team has 10x’d in size, while our revenue has 100x’d year over year. This momentum has been unprecedented. A much larger competitor, Cognition (makers of Devin) has fallen behind us on the capabilities that matter most to customers: cost, quality, and security. Instead of competing in the market, Cognition engineers feigned interviews with us to pry information about our product. Not finding what they were looking for, Cognition has decided to throw their weight and money at people with direct knowledge of our most confidential plans. It feels as though ethics is being thrown out of the window in the AI era. People are willing to do anything, including exploiting privileged information and violating ethical boundaries. I think this is unique to our time, and I don’t think it’s right. Integrity still matters. Yesterday, I made the decision to immediately terminate Christopher Degnan’s roles as a Board Observer and Advisor to Factory, after over a year of service. Prior to this decision, Chris told me he had a casual conversation with an executive at Cognition AI. When I questioned his intentions, he reassured me that ethics aside, he had “made too much money” and was “too lazy to go work for Cognition,” which I trusted and believed. On Monday, Chris spent time advising the Factory team on a handful of confidential board-level matters. That evening, he called me to say that the conversation that was initially described as casual and one-off was actually formal and recurring. For weeks, while he sat in our board meetings and advised our leadership team, he was also confiding with executives of our largest competitor. Chris was subject to confidentiality obligations in connection with his work with Factory. We do not know the extent of the information he shared, but it puts his timely questions about our product roadmap and what the parity gap involves into a new light. It is sad to sever a relationship with someone who has been a trusted advisor - and even a close friend - for over a year. But Chris’s conduct is unacceptable to me. Trust in Board Membership is one of the sacred bonds in the Silicon Valley, one that helps the startup ecosystem thrive. With it comes an enormous responsibility. That trust was violated. Competition is good. I respect and in many cases admire our competitors. San Francisco is a beautiful, singular place where the bold and ambitious go to defy the norms and precedents of the past. But certain principles that must remain. Violating ethics to seek advantage turns what should be positive-sum into zero-sum. We all love technology. And we all love to compete. But we must hold ourselves to a higher standard. The future of software engineering comes with abundance that will impact every person on Earth. Building that future comes with immense responsibility. We must build and compete with integrity.

JUST IN: Rumors are circulating in Silicon Valley that some Anthropic engineers have begun “worshipping” Claude as a god. — The Spectator

OpenAI's Noam Brown says air-gapping the computers may not stop a misaligned AI, because two air-gapped machines can still talk by running a CPU hot and reading the temperature change "But I think the major takeaway from the incident is that people underestimated the AI. And we never want to be in a situation again where we underestimate the AI. It's a weird world, because AI progress is so fast that people are consistently underestimating the AI." "So to be in a situation where you don't underestimate it again, when it comes to safety and alignment, you have to have a very, very, very high bar." "You could even go as far as to say, "Well, we should air gap the computers." And I'm not convinced that that would be sufficient." "There are studies, and this is mostly academic, where you can have two computers next to each other that are air-gapped and they're still able to communicate with each other because they have temperature sensors." "One of them is able to run their CPU really hot, and then the other one can actually detect the temperature change, and then that actually gives them a mechanism to communicate." _________ Link and more key quotes from OpenAI's safety related conversations: firesidealpha.substack.com/p…

I really enjoyed managing you and it was sad to lose you, but I think you're doing the right thing We need talented safety expertise holding AGI labs to account. You're one of the most promising researchers I've worked with, and I have no doubt will do great work at METR

Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead. You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement. I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible. But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier. Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want. Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well. So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it. If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.

🚨 Four no doubt painful years later, @YannLeCun’s transformation into becoming a perfect clone of @GaryMarcus is now complete! 🚨