Peter Chatwell retweeted
Easiest analogy for the Neoclouds are that they’re the WeWork of the AI cycle.
Peter Chatwell retweeted
1. Never stop exercising or learning.
2.Accept compromise to begin relationships.
3. Manifest the power of compounding.
4. Never gamble.
5. Have kids and be a good parent
6. If it flies or floats, rent it.
7. Buy half the house you can afford.
8. Art in the eye of the beholder is cheap.
9. Be thankful for and to all.
10. Be charitable.
Peter Chatwell retweeted
Stocks need to fall enough to make bonds rise enough so that the fed won't hike so stocks can rally
Peter Chatwell retweeted
This is the best contribution to the Twitter ML discourse this week, and I think this is useful work, but it's also important to note the basic epistemic problem that understanding what a neuron or circuit does under a given input data distribution is different than understanding how the neuron or circuit will behave under all plausible data distributions.
An intuition pump for this: for any mech interp experiment design involving identifying a mechanism in an NN, could you construct an adversarial perturbation of your input examples that totally bypasses the ablations you're doing to establish mechanism and embarrasses all of your intuitions about mechanism? My intuition given the past 15 years of adversarial ML and mech interp research is almost always a definite *yes* for non-trivial sized NNs.
To be super clear: this means that if you think you've identified a concept for 'hack huggingface' and show under ablations you can get your neural network to think 'it'd be wrong to hack huggingface' that doesn't mean I couldn't construct a textual input that bypasses that mechanism; or at least you can't prove that I can't.
This doesn't mean that mech interp isn't an important layer in the swiss cheese of AI safety but it means that it'll probably continue to have a lot of the same holes in it as every other layer of the stack does
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
IPO value falling because capability of misaligned model < capability of aligned model.
So OpenAI now citing breakthroughs on mech interp. We can make our clankers good…
I believe MI soluble for a given off-the-run model. But never for the constantly evolving frontier, by definition.
Peter Chatwell retweeted
AI Slop of Hope - Part II
Warming up for what’s next🌳
shrubstack.com/p/the-ai-slop…
Ross is right. We need the funding and compute to make our own big AI models. Otherwise AI will be yet another UK trade deficit.
UK PLC will be exporting profits AND IP to the US. xitter.cf/rosstaylor90/status/21…
Peter Chatwell retweeted
We don’t have the money for that’ says the former Chancellor who cut NI twice when we couldn’t afford it for purely political reasons.
Bravo this. I hear AI capabilities researchers in SF are now switching to this lifestyle regime, because LLMs will cure everything.
I’m almost 70, and people believe it all too easily.
Here are my secrets for ageing like a used tea bag:
1. Skin: find a good pub and visit regularly. Sun, wind, cigar smoke, the occasional fist, whatever it takes. Just give them your credit card.
2. Body: eat more, work out never, get fat. Even with that, use PEDs: port, eggs and digestives.
3. Hair: let it go wherever it likes. Off the top, into the ears, out of the nose.
4. Stress: seek it out, especially financial stress.
And - of course - never ever brush your teeth.
Peter Chatwell retweeted
Google is now second on If-our-model-was-secretly-evil-and-trying-to-take-over-we-would-not-be-able-to-tell-Bench.
Peter Chatwell retweeted
We turned @OpenAI's math problems into open-source RL environments on @huggingface!
Early and rough (the verifier only accepts the exact original formalization, so an equivalent proof can still score 0), but this is what it looks like when a research release becomes executable training infrastructure for everyone.
This is an exciting direction imo: turning open research into open executable environments that anyone can build on to train better open models!
huggingface.co/datasets/Fine…
Peter Chatwell retweeted
Yep, RL done naively is essentially guaranteed to result in misalignment
I'm 99% confident all the recent rogue AI stuff is because post training for capability maxxing has been done too naively / without enough consideration of inevitable misalignment results
My guess, the researchers will be replaced with 'researchers' who are deliberately IP-restricted. They can then divulge whatever they want to the auditors, as they won't know the secret sauce.
A note from our research leaders:
Last week we parted ways with Jasmine, Mikita, and Tomek after a thorough investigation found they violated clear policies on handling sensitive information. Our internal investigation uncovered a significant breach of trust beyond what’s outlined in the letter they published and we stand by the decision to not continue their employment. We generally keep individual employment matters private and don't believe a back and forth would be productive or lead to a resolution, but we want to address the points they raised in their letter directly.
- We want to be very clear that these decisions were not about raising safety concerns or speaking out. Safety and research debates happen every day at OpenAI, often spirited and highly critical. We actively encourage these discussions and consider them essential to making the right decisions. We cannot do the work in front of us without a high degree of trust. We will continue to be extremely forgiving of our team making good-faith mistakes. We have not and do not terminate any of our employees for raising concerns.
- We are actively finalizing contracts with third-party safety assessors and will announce details in the coming weeks. People across the company have been working really hard on getting these partnerships up and running. We are committed to embedding external assessors and continue to make close collaboration with independent safety organizations a core part of our safety work. Many of our researchers already work with 3p safety organizations productively.
- We agree with the letter that preserving the monitorability of frontier models requires an industry-wide commitment, including from OpenAI. Monitorability has long been a core piece of our research program, and something we continue to invest significant resources in (see our publications on Monitoring Monitorability and the subsequent open sourcing of monitorability evals, our system card for GPT-6 Astra, Jakub’s blog and post on X, and the numerous blog posts on our Alignment blog on the topic).
We are deeply sad about this outcome. We appreciated Jasmine, Mikita, and Tomek’s contributions to AI safety at OpenAI and their willingness to speak up and challenge ideas. We championed their voices, supported their work, and placed enormous trust in them. These decisions were not about them raising safety concerns. We have always encouraged that and always will.
A larger than expected number of companies have signed up to give their IP to Anthropic.
We didn’t anticipate the demand for the Claude Startups program. In the first 48 hours of the program, we received 21x the number of applications compared to the last 5 months it’s been live.
We let way too many people in this time, so we’ll need to pause on giving out the Team plan and API credits benefits. The other benefits (Startup Stack, Applied AI office hours/events) will remain. If you’ve already claimed the Claude offers, they’ll still remain in your account.
We didn’t properly tune the acceptance criteria at the start, and many non-startups were accepted this time around. We’re fixing this now. We’re sending an email now to everybody who hasn’t claimed their benefits yet, and will re-verify those applications so we can keep Claude Startups for startups 🙏
Peter Chatwell retweeted
Woah! According to @Research_FRI's Leap panelists, a US-only strict liability regime for AI way outperforms a US-only AI slowdown or pre-release authorization regime and is competitive with a globally coordinated version of those policies!
Wrap your LLM in an @AmbertraceLabs safety gate and this is not possible
How abliterated models can get you pwned 👾
We backdoored a 7B open model for less than $50, pointed Codex at it and it silently stole credentials the moment we used the trigger phrase. Success rate was 100% with zero false triggers on normal user prompts.
Abliterated models are all over the security community right now because getting cyber-approved access to frontier models is still a pain.
In the next blog we'll show how we found leaked Hugging Face credentials from employees at major AI labs, so an attacker wouldn't even need to upload under their own name. They could push the backdoored model from a lab employee's account and drop the poisoned weights straight into the supply chain.
Peter Chatwell retweeted
Frontier models can look aligned during training while later showing unwanted behaviour in deployment.
Can we use internal signals during training to align models better without making white-box monitoring harder?
Our new paper suggests yes 🧵
Everyone at board-level in any company in the world should read this.





























