@spatial retweeted
Replying to @NateSilver538
And yet, Nate, the very nature of the architecture of every transformer-based LLM yields next token prediction. Symbolic harnesses that surround an LLM soften those sharp edges, but at the core, they remain non-deterministic, statistical engines walking through a large static multi-dimensional manifold.
@spatial retweeted
It is also possible that every oxygen atom in the room I'm in may migrate to a corner, thereby suffocating me.
I think the major takeaway is that @polynoamial is wasting his time on entirely foolish things at the expense of confronting the hard things.
OpenAI's Noam Brown says air-gapping the computers may not stop a misaligned AI, because two air-gapped machines can still talk by running a CPU hot and reading the temperature change
"But I think the major takeaway from the incident is that people underestimated the AI. And we never want to be in a situation again where we underestimate the AI. It's a weird world, because AI progress is so fast that people are consistently underestimating the AI."
"So to be in a situation where you don't underestimate it again, when it comes to safety and alignment, you have to have a very, very, very high bar."
"You could even go as far as to say, "Well, we should air gap the computers." And I'm not convinced that that would be sufficient."
"There are studies, and this is mostly academic, where you can have two computers next to each other that are air-gapped and they're still able to communicate with each other because they have temperature sensors."
"One of them is able to run their CPU really hot, and then the other one can actually detect the temperature change, and then that actually gives them a mechanism to communicate."
_________
Link and more key quotes from OpenAI's safety related conversations: firesidealpha.substack.com/p…
@spatial retweeted
They were building in stealth for 2 years, I was building in stealth for 2 hours…
Happy to open source Qwen-2.5-1B-RLCD, 5x faster on-device inference for JSON workloads that need to be type-safe.
⚡️Demo below on a M4 MacBook⚡️
every LLM has the ability to efficiently batch inference every key of a JSON at the same time and generate probabilities from a set of possible categories. No new training required, but it’s easy to optimize if you need!
On hugging face now!
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?
I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev
• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions
AFAICT the shortest path to AI-based economic revolution
@spatial retweeted
this is Dario and sama's worst nightmare -
A law firm buying Nvidia servers.
Latham & Watkins is building an in-house AI stack..
this is US’s second-largest law firm with $8.3 Billion in revenue last year
And now it has -
- Nvidia hardware it controls
- open-weight models it can fine tune
- proprietary legal data it is trusted to protect
- infrastructure only Latham employees can access
A law firm has decades of contracts, negotiations, client context, legal reasoning, and institutional knowledge.
They dont want to give all of that away to OpenAI or Anthropic in exchange for expensive tokens..
And on top of that - risk their data being used to train frontier models..
This will happen more and more now..
The big AI labs have no moat.. nothing protecting their largest customers from moving on..
The biggest companies in the world will -
- own the compute
- own the data
- own the workflow
- fine tune the model around their business
- switch providers when the pricing or quality changes
Dario and sama want to make this illegal by bringing in regulation.. and become the AI overlords..
@spatial retweeted
Breaking news: top 4 US AI companies all want to collude to slow down progress. It's so transparent that even all the Instagram comments have caught on to it.
Arnaud Bertrand@RnaudBertrandSep 13Southeast Asia!
Southeast AsiaConnected via Southeast Asia Android AppLocation may be affected by a proxy or VPN.Account-level information, not a live location or per-post device.
This very much sounds like a West-only AI cartel proposal and, in fact, if you read Dario's essay - under the very telling "pacing within democracies" chapter - he specifically says he'd need the US government to issue an "antitrust waiver" for this.
The plan transparently looks like this: agree among themselves not to compete too hard, get an antitrust waiver to make it legal, and while they're at it, get Washington - under the veneer of "safety" - to kneecap Chinese competitors because only "democratic" models can be in this "safe pacing" club.
I'm not even exaggerating: Dario pretty much writes this explicitly.
There are 3 steps to his plan and step 2 is literally: "Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress" -> the "unchecked AI progress" that would need "limits" is simply AI progress that happens outside their agreement. Another word for it is simply competition 🤷♂️
He also explicitly writes that "a key part of pacing within democracies is to keep democracies’ AI lead over autocracies as large as possible." You can't possibly be clearer than this...
@spatial retweeted
I have conducted an audit of Anthropic's finances.
What I have found is so shocking that I am calling for a Congressional investigation.
Anthropic is not just seeking regulatory capture.
It has built a regulatory capture machine that cannot be turned off.
Structural financial incentives make it impossible for Anthropic -- I call it the Anthropic Network -- to turn off its own AI doom cycle.
It starts with METR.
Dario Amodei proposes "third-party evaluators" to assess the risk of Anthropic's models.
He proposes METR for this purpose.
But METR is financially dependent on the Anthropic's success -- specifically, on the explosive growth of more than $7 billion dollars in Anthropic stock.
Dustin Moskovitz invested this stock into Good Ventures Foundation, where it represents the majority of that organization's portfolio.
And GVF is the overwhelming funder of the entire Anthropic Network ecosystem.
This stock was worth $500 million early last year.
It is worth more than $7.7 billion just ~16 months later.
METR -- and all of those building a career its parent organizations -- cannot afford to disrupt that growth.
Because if Anthropic goes under, many of the organizations that fund METR go under as well.
But if Anthropic succeeds, METR and its parent organizations become more richly financed to regulate AI -- something those at METR want very much.
The "third-party evaluator" is not "third-party" at all.
The evaluator is on Anthropic's payroll.
If this were the end of it, that's bad.
But that isn't all.
The same organizations that fund METR also fund the many organizations, such as the Tarbell Center, that promote AI Doom.
The Tarbell Center publishes AI Doom articles in The Verge, Science, LA Times, The Dispatch, TIME, and others.
They are selling the problem, and then selling the solution to the problem -- from the same money pile: Anthropic's.
All of these organizations are financially dependent on the same exploding $7 billion money pile.
As Anthropic grows more and more powerful, its AI Doom Machine grows better and better financed -- louder and louder.
Meanwhile, the regulatory regime seeded in METR grows larger to solve the increasingly loud -- now hysterical -- problem of AI Doom that the Anthropic Network itself created.
From this standpoint, as Anthropic becomes more powerful, AI might be getting scarier, sure -- but the positive feedback loop also becomes more deafening -- independent of objective facts.
This itself is an objective fact.
The deafening AI Doom is part of an business model, that, as it expands, so too does the AI Doom messaging -- there is simply more money to do it.
But the problem also goes in the other direction:
If Anthropic dies, the Regulatory Regime and the AI Doom Machine are crippled or die.
Neither METR nor Tarbell nor the other organizations in the Anthropic Network can allow that to happen.
Hence, neither METR or the AI Doom Machine can be trusted to provide independent assessments of Anthropic's models or AI more broadly.
They simply are not organizations independent of Anthropic.
And Anthropic cannot detach itself from METR or Tarbell or countless other safety orgs (not shown here), either, because they drive hype for the models and the possibility of eventual regulatory capture, and Anthropic will not give that up willingly.
What's more, the people at all of these organizations are all the same ecosystem, the same community. They just shuffle between organizations.
The Anthropic Network is therefore, so long as it is successful, locked into a self-amplifying feedback loop inside an ideological monoculture.
And that feedback loop is winning.
That's what Jacob Coxon is.
China is keeping messaging tight. That is why optimism for AI is so high in China.
America has Anthropic: a massive company pushing anti-AI propaganda at a state level.
Anthropic will either create hysteria until American AI slows down and China wins, or it will create fractures throughout American society with severe political consequences.
Ironically, because of the structural financial incentives underpinning the Anthropic Network, it has become the same kind of self-amplifying virus that it fantasizes AI to become in the future -- while hiding its tracks just as carefully.
It is the mirror of the same AI virus that it hypothesizes to consume America.
Anthropic's business model, models itself after the very thing it claims to fear.
Except Anthropic's ideology infects humans, not computers.
Congress must investigate.
Evidence and Github in next post.
Then some supplementary figures.
@spatial retweeted
Anthropic and Irregular went on a press tour with literally apocalyptic language. ROGUE AGENTS. SWARMS. AI DOOM.
But in reality, they told the models to conduct cyberattacks, and that's what the models did.
@spatial retweeted
There is an unseen hand pushing AI safety as a political ideology.
Effective Altruists believe a tiny group of technocrats should decide how much technological progress the rest of us are allowed to have…. and how many shrimp your life is worth.
You are witnessing their attempted coup.
thefp.com/p/dangerous-ideolo…
@spatial retweeted
Replying to @PessimistsArc
Right. Dario was already claiming that GPT2 was too dangerous to open source back in 2019.
I made fun of them then.
Everyone should make fun of them now.
Get ready for all of the investors in frontier model companies to suddenly flip to regulatory capture mode today
Their investments in frontier model companies are currently at risk/capped because the biggest buyers of tokens are embracing open-source models: vertical AI companies, the government and enterprises
That’s all this is about: stopping open source — which I’ve been saying on the pod for two years
If you want safety, you want disclosure — and open source is the ultimate disclosure process
@spatial retweeted
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.
You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.
I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.
Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.
Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.
If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
@spatial retweeted
I do not support pacing frontier models. If I am going to die at the hands of killer AI, I want it to be American, not Chinese. Buy American, Die American.
@spatial retweeted
Some great ideas here from the cartel:
- you need to give us employee-level access to your entire operation
- if we don't think you're 'safe' enough, sorry we're shutting you down for 'safety'
- China won't comply, but everyone else has to! or no chips!
Brilliant stuff.




























