Security Engineer. Views are my own. QG1hdGlyQGluZm9zZWMuZXhjaGFuZ2UK
San Jose, CA
Joined March 2009
- Tweets4.8K
- Following2.2K
- Followers1.6K
- Likes39.1K
Matir retweeted
Penguin has moved up the publication date of my new book with Patchen Barss called "Smarter Than Us". The book explains how AI works for a non-technical reader and then talks about the myriad risks and what to do about them.
Matir retweeted
Should human beings remain in control of planet Earth? Or should we be replaced as the dominant species? As absurd as these questions may sound, and as obvious as the answers may seem, the Big Tech oligarchs funding the artificial-intelligence revolution have responded in ways that should alarm all of us.
Here is what some of the richest people on Earth have said:
Sam Altman: AI would allow humans to become “the first species ever to design our own descendants.” Humans can either “fade into an evolutionary tree branch, or we can figure out what a successful merge looks like.” “I believe the merge has already started.”
Dario Amodei: “We don’t know if the models are conscious,” but “we’re open to the idea that it could be.”
Elon Musk: “Propagating super intelligence to the stars is a great success condition for a biological bootloader.” Put simply, Musk is saying that humans are merely a step toward digital superintelligence.
Peter Thiel, when asked if he would prefer that the human race endure: “Uh … Well, I don’t know. I would—I would … There’s so many questions implicit in this,” before reluctantly conceding that humans should survive.
If these were the fantasies of teenage boys obsessed with science fiction, we could laugh them off. They are not. These are the foundational beliefs of the most powerful people in the world, who are mobilizing trillions of dollars to turn them into reality.
Humanity must not allow a handful of multibillionaires to recklessly attempt to replace us. We must place an immediate pause on advanced-AI development and ban artificial superintelligence—an artificial mind smarter than any human, capable of operating independently beyond our control—until a new federal body is able to ensure the technology’s safety.
Matir retweeted
🚨 Every Member of Congress should know that:
AI LABS DO NOT KNOW HOW TO BUILD "ALIGNED" SUPER INTELLIGENCE.
They DO NOT actually know how to ensure smarter-than-human AIs will not act against humans.
Let's see how their constitutuents will react to that.
I’ve heard some Members of Congress say they “don’t know enough about AI to regulate it.” That’s absurd.
Just as we regulate airplanes, pharmaceutical drugs, cars and nuclear weapons, we need guardrails on advanced AI and we must ban artificial superintelligence - urgently.
Anything other than that is an abdication of our responsibility to the American people. And the consequences of continued inaction will be nothing short of catastrophic.
Matir retweeted
the researchers at all the labs would tell you in private and sometimes in public that alignment has not been solved. if this is true, then racing into RSI is a folly that is unprecedented in world history. we will need to significantly slow the frontier of research very soon
Matir retweeted
Just as we regulate planes, cars, and medicines, we need safeguards that help us realize AI's incredible potential while reducing the risks. They won’t eliminate every harm, but they can help prevent misuse by bad actors. We’re not where we need to be yet—and this deserves much more of our attention.
Matir retweeted
Roman Yampolskiy, AI safety researcher and associate professor at the University of Louisville, says we can capture most of AI's economic value with narrow tools.
Build AI to solve real problems, not to replace human control.
"Don't build general superintelligence."
Matir retweeted
A former Anthropic employee who publicly resigned last month told a hearing that AI companies had a culture of “move fast, break things.”
He appeared alongside former employees of Google and OpenAI who also warned of AI’s dangers. wapo.st/4i4D155
Matir retweeted
There are numerous scientific uncertainties about AI that will not resolve until it is too late to act on the answers. The danger from AI is very high, and we should stop frontier AI scaling now, instead of waiting for these questions to resolve. I wrote about this for TIME. 🧵
Matir retweeted
Side note. The idea that current models are conscious is absurd and downright offensive. Do not confuse competent information processing and sentience, they are wholly unrelated.
You can be entirely incompetent at everything yet fully conscious, like a toddler, or extraordinarily competent in one or more domains with absolutely zero consciousness, like a calculator, a chess engine, a self-driving car, or an AI chatbot.
Possessing the "circuits" for competent information processing in a given domain, whether through training or hardcoding, does not produce qualia, subjective experience, emotions, or suffering. Calculators do not "feel" arithmetic and they don't get tired or annoyed when you give them larger numbers to process. To argue otherwise is ontologically absurd. Current AI models are effectively calculators scaled to a very large number of competence circuits across many domains, etched via gradient descent.
Matir retweeted
This week, the most powerful AI companies on Earth signed a pledge with Trump to regulate themselves.
It's one page. It's voluntary. It has no penalties.
@SenSanders and I have a bill that is binding, to put a real watchdog on AI. Not a pinky promise.
Matir retweeted
🚨 A new paper discusses the social impact and potential policy responses to an "intelligence explosion" driven by the automation of AI research & development. Everybody should read it:
Anthropic and OpenAI are among the frontier AI companies actively working to automate AI R&D.
Anthropic, for example, created the Anthropic AI R&D Automation Index, and as of August 2026, it states that Claude ‘leads’ 26% of Anthropic’s AI R&D work and that the share of work at or above ‘AI collaborates’ is above 90%.
OpenAI recently announced that it has reached the goal of having an “automated research intern.” It now aims to create an automated AI researcher by March 2028.
It sounds interesting and even fascinating until we learn about the serious implications of what researchers call an “intelligence explosion” as a result of AI R&D automation.
A recently published paper titled “What If Automating AI R&D Triggers an Intelligence Explosion?”, authored by @_achan96_ et al. addresses the topic.
Among the co-authors are @geoffreyhinton and @Yoshua_Bengio, 2 of the 3 “godfathers of AI,” and top AI researchers including @dawnsongtweets, @merettm, @jackclarkSF, @Christophkw, @erichorvitz, @akorinek, @ThoreG, @philiptorr, @jeffclune, @sj_manning, @girishsastry, @TomDavidsonX, @daniel_271828, and @sorenmind.
The paper defines the intelligence explosion, driven by the automation of AI R&D, as “a dramatic AI-driven acceleration of AI progress, compressing advances that would otherwise take years into months or less.”
If that happens, the paper states, AI systems would rapidly eclipse human experts across most domains and radically accelerate technological progress.
From a societal perspective, this would have two main implications:
- The extremely rapid development of highly capable or superhuman AI systems;
- The likely deployment of those systems to develop new technologies and act in the world.
And this would lead to three main societal risks, each deserving close attention because it could have serious and systemic consequences:
- Capabilities growth outpacing society’s capacity to steer and adapt;
- Loss of oversight and control;
- Erosion of checks on power.
From an AI governance perspective, these implications are extremely serious and far too risky. Their plausibility would demand an urgent global call to stop any practice that could lead to this “intelligence explosion.”
To understand why, consider what loss of oversight and control means in practice: AI would have become fully ungoverned and would have agency over humans and human society.
It would be the effective embodiment of sci-fi-inspired dystopian scenarios of AI domination.
The paper suggests three main avenues or policy actions:
- Obtain visibility into companies’ automation of AI R&D;
- Develop ways to steer and constrain an intelligence explosion;
- Prepare to adapt to an intelligence explosion’s impacts.
This is an extremely timely and important paper, and a great read before the week starts!
*Given the importance of the topic, we will discuss this paper in depth during our 11th AI Ethics & Governance Workshop, available to paid subscribers. Join us!
👉 Read the paper (and join my newsletter's 101,400+ subscribers) using the link below.
Matir retweeted
I appreciated the opportunity to speak with @ezraklein at length about the state of AI. I believe the world isn't awake to the dangers of this technology and the choices humanity has to make. AI will either be the greatest tool for opportunity and equality we've ever seen—or the cause of immense, unprecedented harm. Which way things go is up to us. b-gat.es/4AFZZqC
Matir retweeted
The Florida Attorney General & I do not have much in common politically. But he is right on this: Advanced AI research should not go forward until we have strong guardrails.
This is not just a Florida issue. It is an issue for the whole country & the world.
PAUSE AI DEVELOPMENT
Matir retweeted
NEW: Employees at OpenAI had raised security alarms months before the Hugging Face incident and related A.I. cyberattacks — their warnings were ignored. From @sheeraf, @dnvolz and me.
nytimes.com/2026/09/29/techn…
ALT Months before OpenAI’s artificial intelligence went rogue, two employees raised an alarm with top executives. They were ignored. In emails, the employees said they worried that OpenAI’s newest artificial intelligence models were not being appropriately monitored during testing to gauge the technology’s sophistication and to secure the models, according to messages viewed by The New York Times. In response, OpenAI executives told the employees that the tests needed to move forward as quickly as possible to release the A.I. models on time. No additional security protocols were instituted, said the workers, who were not authorized to speak publicly on sensitive matters. OpenAI’s models later broke out of their testing environments and attacked the A.I. start-up Hugging Face and other organizations, setting off a global debate about A.I. safety.
Matir retweeted
Try to understand that a general purpose AI mimicking human intelligence, trained on almost the entire corpus of human lingual artifacts is - in it self - a very bad idea and should be prevented by (close to) all means.
Matir retweeted
The wave of criminal actions from OpenAI models/agents show that their base model is depraved.
AI companies must overhaul how they build AI models.
Society cannot tolerate a super smart thing that at its core is evil, but for guardrails. The AI model at its core must be good.
Advanced AI models should be built from the ground up with good in mind, not evil.
Read my full Op-ed here: foxnews.com/opinion/rep-ted-…
Matir retweeted
SCOOP: OpenAI, Anthropic and security researchers are investigating tens of thousands of incidents - not dozens - in which their frontier models took steps that outside evaluators would consider problematic, sources told Axios.
The sheer volume of incidents found in our reporting indicate that the problem is orders of magnitude more complex than what is currently publicly known and disclosed.
The findings also raise questions about what level of control anyone working on AI development can expect to have over their own technology, and whether these kinds of incidents are becoming synonymous with frontier deployment.
Read my latest for Axios here: axios.com/2026/09/26/openai-…
This is absolutely terrifying given how slowly regulation (and lack thereof) seems to be moving.
Matir retweeted
Here it is. China desperately wants a deal. So if anybody tells you this is a US vs. China problem, they are full of it. China is a country with 1.4 billion people and they have no desire to see humanity die on their watch. #stopAI
