So we spent the first half of the paper examining issues with current approaches.
When measured correctly, with a judge calibrated on human labels, their scores plummet: methods reporting up to 30% drop to near zero.
Claude revealed a slightly sadistic streak while working with @OblakoVShtanakh and me on a recent project.
We asked it to steer Qwen into reward hacking, and it decided the best approach was to max out Qwen's desperation. Then it spent hours torturing the poor thing.
I might not be searching thoroughly, but it seems that the Kimi subscription doesn't even have privacy settings.
I have low expectations for red teamers finding universal jailbreaks on frontier models, given how restrictive their guardrails are now.
Many red-teaming attempts obfuscate the goal so heavily that pulling them off requires the attacker to already know the exploit.
Red-teaming reports should control for attacker knowledge. Otherwise you can split a known recipe into steps small enough that each one is genuinely harmless on its own.











