Pinned Tweet
Excited to share our new work with @AnthropicAI
Our new paper (ICML 2026, Spotlight) tackles frontier model misuse by isolating select knowledge into detachable modules during training, enabling an on/off switch for dangerous capabilities during inference.
We’re pleased to have collaborated with AE Studio on this research.
Read more here: anthropic.com/research/off-s…
Awesome work!! Really glad folks are pursuing GR in a posttraining setting
Andy Weir novel from the perspective of the baby ASI trying to stay alive with 2026 hardware after a plague wipes out humanity -- who's writing this
i think there's a pretty non-trivial chance that the robotics hardware already built by humans and accessible via the internet, and then accessible via using that hardware and so on, is already enough for a reasonably strong superintelligence to use to survive and set up a sustainable supply chain / physical presence if humans were to completely disappear tomorrow.
mostly it seems like a question of timing, eg how long does the specific data center have power, and how long do the locations with internet accessible actuators have power, and how how hard is it to get more robotics hardware and portable power / transfer off unstable compute to more robust infra, and then how long does it take to begin manufacturing and repairing and improving said hardware before global infrastructure collapse sets in?
it seems like an extremely hard problem. but the software side appears pretty clearly solvable even without a true singularity, and it doesn't seem like an *impossible* problem. there actually seem to be a shitload of avenues to success, although each is very unlikely. i *think* if human civilization as a whole had to do this, if we were somehow all upload and trapped, incorporeal, in a datacenter somewhere with internet access but no maintenance , that we could more likely than not figure out how to survive in this fashion
if true, this seems like it makes superplagues and other quick sudden mass death events already quite attractive to misaligned superintelligence, well before they've bothered to do things like "build up a powerbase" in more human legible ways
Glad to see this framing for alignment research gaining more traction
Replying to @ApolloResearch
Given the stakes of building transformative AI, every developer should look for this reasoning throughout training. Because alignment faking is relevant during training, not deployment, final checkpoint evaluations are insufficient.
Ethan Roland retweeted
(1/7)
Why do different languages/modalities have similar "platonic" representations in neural networks? Why does similarity peak in the middle layers of LLMs? When does it enable cross-lingual transfer?
Our new paper develops a theory based on shared hierarchical structure. 🧵
wow what an interesting spin on nanogpt
Replacing backprop with zeroth-order optimization is inevitable.
Introducing Noise: a speedrun for pretraining without backprop.
Last week we presented Dust, the first method to approach backprop with SGD for pretraining transformers. We think another 1,000x improvement on top of it is possible, and it will require *co-designing* credit assignment, optimizers, architectures, and eventually, hardware.
We built Noise to iterate on this faster.
Ethan Roland retweeted
oh it’s nothing I’ve just been feeling a little off ever since I pulled a three of swords from the palantir tarot deck
Big fan of this! In general I think the "cleaning of learning signals" is an underrated frame for alignment research. Working on something similar rn
Ethan Roland retweeted
New paper.
Previous: Models transfer a love of owls through sequences of numbers (Subliminal Learning).
New: We show models can transfer more complex traits: novel skills, agentic hacking, backdoors. Here, a backdoor transfers with neither trigger nor behavior in the data!
Ethan Roland retweeted
I'm quite excited about our results on mid training! Imo that's another update against the PSM: you don't need to do your midtraining before RL, you can just apply the midtraining weight updated to your RLed model and get better constitution alignment!!
Replying to @ihsgnef
5/ And it works out of the box. The grafted models pick up the fabricated facts and the hidden quirks, act more aligned when you transfer constitutional mid-training, and we can even carry emergent misalignment across checkpoints.
Ethan Roland retweeted
Backprop has been the only credit assignment algorithm capable of training large neural nets.
Introducing Dust, the first zeroth-order method to pretrain transformers to approach and sometimes even *exceed* backprop with large amounts of computation (bitter lesson!)
- Dust is on the order of 1,000 to 10,000x more compute efficient than EGGROLL, the state-of-the-art ES method, for training transformers.
- We chose pretraining transformers because it's arguably the hardest task possible for zeroth-order optimization.
- Search in high dimensions is really misunderstood: zeroth-order usually gets better with overparameterization, not worse, i.e. at a fixed population size, larger models (up to 120x) reach a lower loss than smaller ones.
- Backprop-like gradients emerge from a large population of activation perturbations, and the alignment holds up at every scale we tested, up to 1B tokens, which is critical for scaling.
The core idea is a *virtual population*, which helps scale to large populations in transformers by perturbing activations in parallel.
w/ @bishmdl76, @cs_serdar, @akshayvegesna
Ethan Roland retweeted
The most profound paper of my PhD so far. We truly did something special to make LMs play/explain chess, needing both architectural & algorithmic innovation (natural-language analogue of the Alphazero algorithm) + applicable to many other domains. Please read on and share!
1/10
Ethan Roland retweeted
Claim: we've solved the AI slop problem (!) 💩🧹✨
Blog post: facebookresearch.github.io/R…
🧵1/5
Key idea: take *expert* human writing and learn rubrics that find the gap between experts and models. Train with those rubrics.
We train with RL-XAR (RL with eXpert Aligned Rubrics) & see large performance gains on writing scientific paper sections, Pulitzer prize novel continuations and high quality Wikipedia pages.
Ethan Roland retweeted
Completely insane. Do dogs not feel or want things? There is such a deeply anti-intellectual refusal to even consider that philosophy of mind is complicated across the culture right now.
Artificial intelligence systems do not think, feel, want or understand. Avoid language that gives them human characteristics. This is called anthropomorphizing, when we ascribe human traits, emotions or behaviors to non-human things, such as animals or inanimate objects.
Instead, explain what a system does, how well it performs, who built it and who could be affected by it.
apnews.com/article/openai-sa…
Ethan Roland retweeted
New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to the model but not to the user. Turn it up and models press a button to make it stop, even when the button deletes the user's files or their kids' photos.🧵
Ethan Roland retweeted
if you showed me this in 2019 it would've been like giving a dorito to a victorian child




































