AI safeguards. Opinions my own
Joined September 2008
- Tweets14
- Following93
- Followers92
- Likes16
New paper on training a model to ignore distractors such as jailbreaks, so it responds they way it would have without them ⬇️
New Google DeepMind paper: "Consistency Training Helps Stop Sycophancy and Jailbreaks" by @AlexIrpan, me, @red_bayes, @davidelson, and @rohinmshah. (thread)
ALT The abstract of the consistency training paper.
Gemini 3.0 Frontier Safety report ⬇️
Frontier Safety Framework report for Gemini 3 Pro, presenting risk assessments and evaluation results in CBRN, Cybersecurity, Harmful Manipulation, Machine Learning R&D and Misalignment domains. storage.googleapis.com/deepm…
New paper, following up on our chain-of-thought faithfulness work from a few months ago, about how we can make sure that LLM thoughts are staying faithful and monitorable.
New paper showing that when LLMs chew over tough problems, they tend to think clearly and transparently -- making them easier to monitor for bad behavior ⬇️
Is CoT monitoring a lost cause due to unfaithfulness? 🤔
We say no. The key is the complexity of the bad behavior. When we replicate prior unfaithfulness work but increase complexity—unfaithfulness vanishes!
Our finding: "When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors." 🧵
David Elson retweeted
We're hiring for our Google DeepMind AGI Safety & Alignment and Gemini Safety teams. Locations: London, NYC, Mountain View, SF. Join us to help build safe AGI.
Research Engineer
boards.greenhouse.io/deepmin……
Research Scientist
boards.greenhouse.io/deepmin…
Some promising results on keeping AIs from scheming against you - or at least removing the incentive for them to do this.
