Kirill retweeted
1/7 Can AI debate reduce reward hacking in RLAIF?
Training against a weak LLM judge, RLAIF hacks: reward rises but judge and policy accuracy both collapse. Debate with an adversarial critic maintains judgments, allowing the policy to recover 45% of the performance gap to RLVR 🧵
Kirill retweeted
My MATS applications are open! Come do research mentored by me. Due Sept 4. All backgrounds welcome!
I've supervised 60 scholars and they get better each cohort, I'm excited to meet this round's!
The application: spend a weekend doing a safety research task
Scholar highlights:
