AI Researcher

Atlanta, USA
Joined August 2014
Excited to share our new work with @AnthropicAI Our new paper (ICML 2026, Spotlight) tackles frontier model misuse by isolating select knowledge into detachable modules during training, enabling an on/off switch for dangerous capabilities during inference.
1
1
13
2,350
Awesome work!! Really glad folks are pursuing GR in a posttraining setting
Models love reward hacking. Maybe we should just let them! In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!
4
70
Ethan Roland retweeted
Andy Weir novel from the perspective of the baby ASI trying to stay alive with 2026 hardware after a plague wipes out humanity -- who's writing this
i think there's a pretty non-trivial chance that the robotics hardware already built by humans and accessible via the internet, and then accessible via using that hardware and so on, is already enough for a reasonably strong superintelligence to use to survive and set up a sustainable supply chain / physical presence if humans were to completely disappear tomorrow. mostly it seems like a question of timing, eg how long does the specific data center have power, and how long do the locations with internet accessible actuators have power, and how how hard is it to get more robotics hardware and portable power / transfer off unstable compute to more robust infra, and then how long does it take to begin manufacturing and repairing and improving said hardware before global infrastructure collapse sets in? it seems like an extremely hard problem. but the software side appears pretty clearly solvable even without a true singularity, and it doesn't seem like an *impossible* problem. there actually seem to be a shitload of avenues to success, although each is very unlikely. i *think* if human civilization as a whole had to do this, if we were somehow all upload and trapped, incorporeal, in a datacenter somewhere with internet access but no maintenance , that we could more likely than not figure out how to survive in this fashion if true, this seems like it makes superplagues and other quick sudden mass death events already quite attractive to misaligned superintelligence, well before they've bothered to do things like "build up a powerbase" in more human legible ways
28
25
2
775
14,837
Glad to see this framing for alignment research gaining more traction
Replying to @ApolloResearch
Given the stakes of building transformative AI, every developer should look for this reasoning throughout training. Because alignment faking is relevant during training, not deployment, final checkpoint evaluations are insufficient.
2
77
Ethan Roland retweeted
(1/7) Why do different languages/modalities have similar "platonic" representations in neural networks? Why does similarity peak in the middle layers of LLMs? When does it enable cross-lingual transfer? Our new paper develops a theory based on shared hierarchical structure. 🧵
2
49
4
304
11,658
wow what an interesting spin on nanogpt
Replacing backprop with zeroth-order optimization is inevitable. Introducing Noise: a speedrun for pretraining without backprop. Last week we presented Dust, the first method to approach backprop with SGD for pretraining transformers. We think another 1,000x improvement on top of it is possible, and it will require *co-designing* credit assignment, optimizers, architectures, and eventually, hardware. We built Noise to iterate on this faster.
2
77
Ethan Roland retweeted
oh it’s nothing I’ve just been feeling a little off ever since I pulled a three of swords from the palantir tarot deck
70
363
144
4,972
230,520
Big fan of this! In general I think the "cleaning of learning signals" is an underrated frame for alignment research. Working on something similar rn
New paper: two ways distillation can help AI safety. - Models trained to hide a secret quirk don’t mention it, but students distilled from them confess it. - We find ways to distill a strong teacher's capabilities into a weaker student while blocking an unwanted subliminal trait.
1
31
Ethan Roland retweeted
New paper. Previous: Models transfer a love of owls through sequences of numbers (Subliminal Learning). New: We show models can transfer more complex traits: novel skills, agentic hacking, backdoors. Here, a backdoor transfers with neither trigger nor behavior in the data!
37
130
48
1,032
101,511
Ethan Roland retweeted
I'm quite excited about our results on mid training! Imo that's another update against the PSM: you don't need to do your midtraining before RL, you can just apply the midtraining weight updated to your RLed model and get better constitution alignment!!
Replying to @ihsgnef
5/ And it works out of the box. The grafted models pick up the fabricated facts and the hidden quirks, act more aligned when you transfer constitutional mid-training, and we can even carry emergent misalignment across checkpoints.
4
4
1
46
2,215
Ethan Roland retweeted
Backprop has been the only credit assignment algorithm capable of training large neural nets. Introducing Dust, the first zeroth-order method to pretrain transformers to approach and sometimes even *exceed* backprop with large amounts of computation (bitter lesson!) - Dust is on the order of 1,000 to 10,000x more compute efficient than EGGROLL, the state-of-the-art ES method, for training transformers. - We chose pretraining transformers because it's arguably the hardest task possible for zeroth-order optimization. - Search in high dimensions is really misunderstood: zeroth-order usually gets better with overparameterization, not worse, i.e. at a fixed population size, larger models (up to 120x) reach a lower loss than smaller ones. - Backprop-like gradients emerge from a large population of activation perturbations, and the alignment holds up at every scale we tested, up to 1B tokens, which is critical for scaling. The core idea is a *virtual population*, which helps scale to large populations in transformers by perturbing activations in parallel. w/ @bishmdl76, @cs_serdar, @akshayvegesna
62
299
61
2,623
274,462
Ethan Roland retweeted
The most profound paper of my PhD so far. We truly did something special to make LMs play/explain chess, needing both architectural & algorithmic innovation (natural-language analogue of the Alphazero algorithm) + applicable to many other domains. Please read on and share! 1/10
61
174
28
1,551
110,294
Ethan Roland retweeted
wake up babe the freaking pope got schmidhuber'd
55
316
37
3,898
177,620
Ethan Roland retweeted
That's why zeroth-order optimization is the ultimate nerdsnipe. Friends don't let friends give up gradients.
I looooooooooove the idea of just wiggling weights randomly many times and looking at the outcomes efficiently to modify them man it does something to me
2
3
1
82
6,821
Ethan Roland retweeted
Claim: we've solved the AI slop problem (!) 💩🧹✨ Blog post: facebookresearch.github.io/R… 🧵1/5 Key idea: take *expert* human writing and learn rubrics that find the gap between experts and models. Train with those rubrics. We train with RL-XAR (RL with eXpert Aligned Rubrics) & see large performance gains on writing scientific paper sections, Pulitzer prize novel continuations and high quality Wikipedia pages.
1
177
71
1,913
224,451
Ethan Roland retweeted
Completely insane. Do dogs not feel or want things? There is such a deeply anti-intellectual refusal to even consider that philosophy of mind is complicated across the culture right now.
Artificial intelligence systems do not think, feel, want or understand. Avoid language that gives them human characteristics. This is called anthropomorphizing, when we ascribe human traits, emotions or behaviors to non-human things, such as animals or inanimate objects. Instead, explain what a system does, how well it performs, who built it and who could be affected by it. apnews.com/article/openai-sa…
130
78
18
1,202
64,598
Ethan Roland retweeted
New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to the model but not to the user. Turn it up and models press a button to make it stop, even when the button deletes the user's files or their kids' photos.🧵
362
613
496
5,105
1,678,371
Ethan Roland retweeted
if you showed me this in 2019 it would've been like giving a dorito to a victorian child
34
145
8
3,951
81,791
don't know what to make of these recent developments
Americanism, not effective altruism. The United States will continue to be AI DOMINANT! 🇺🇸
151
Ethan Roland retweeted
You can just RL a coding model to paint with javascript btw
249
783
333
8,499
1,730,158