Studying human behaviour and brain function, and how to build safe and beneficial technology
Oxford University / UK AISI
Joined January 2013
- Tweets819
- Following901
- Followers8.6K
- Likes1.5K
new paper. During learning, why does it help to practice easy examples before hard ones? We provide a theory, based on deep learning models. TL;DR - curricula help with unlearning rather than learning. arxiv.org/abs/2611.04565. Led by @RubruckJirko and Alice Zhang, with @SaxeLab
AI research stays true for almost exactly the amount of time it takes an academic journal to publish it
How long does a finding about LLMs stay true?
A new paper re-tested 50 widely cited findings about LLM behavior (biases, hallucination, sycophancy, etc).
Within 15 months, most no longer held or had substantially reduced effects. The shelf life of LLM findings is short.
Almost half (44%) of the tested claims are now expired in today’s models, 34% are weakener, 16% persisted, and 6% have strengthened relative to the original claim
The only effects that got stronger were about sycophancy, LLM ingroup bias, and creativity.
This means we should be cautious when interpreting the long term durability of research on LLMs.
zou-group.github.io/llm-clai…
summerfieldlab @summerfieldlab.bsky.social retweeted
Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context.
Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster.
With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington
summerfieldlab @summerfieldlab.bsky.social retweeted
A partir de hoy, disponible online y en librerías mi primer libro, titulado "Tu Mente Te Miente", donde cuento 10 engaños que fabrica nuestra mente y qué herramientas (basadas en evidencia de las ciencias del comportamiento) tenemos para desarmarlos.
penguinlibros.com/ar/ciencia…
summerfieldlab @summerfieldlab.bsky.social retweeted
New in @PNASNews:
In two large experiments (~18k obs from ~14.8k people), we find that brief conversations with AI can get people to take real political action - incl. signing real petitions and giving money to charity.
Surprisingly, the persuasion strategy that worked best for changing people's attitudes - packing convos with info - was the least effective at changing behavior!
🧵
please do submit an essay for this!
AI agents could make organizations far more productive, but leave people more isolated and less autonomous.
We've teamed up with @inherent_labs on an essay prize exploring how organizations can harness AI while strengthening human autonomy and fulfillment.
We're thrilled that @lugaricano, @gleech, @divyasiddarth, and Chris Summerfield are joining us as judges.
blog.cosmos-institute.org/p/…
17
2,516
thank you! and of course everyone should read @MasudHusain 's book Our Brains, Our Selves, which is illuminating and entertaining in equal measure!
Fran Ashcroft and I discuss Chris Summerfield's insightful and very stimulating book on AI -- THESE STRANGE NEW MINDS -- on her podcast A Good Science Read podcasts.ox.ac.uk/good-scien…
3
729
summerfieldlab @summerfieldlab.bsky.social retweeted
Go work with Fazl!
We're recruiting PhD (DPhil) students at @UniofOxford to tackle some of the most important open problems in technical AI safety and governance!
In our Technical Safety & Governance (TSG) Lab, we're particularly interested in developing methods to better understand, evaluate and control advanced AI systems.
Some of the research directions we're excited about include:
Training models to explain themselves and other models: Can we train AI systems to provide explanations that help humans and other AI systems better understand, predict and oversee their behaviour?
Understanding and controlling how model behaviours emerge during training: Can we predict undesirable behaviours from early training checkpoints and intervene before they emerge?
These are just examples of the broader research questions we're excited about. We're open to other ambitious and original ideas in related areas, so don't feel limited to the specific approaches outlined.
There are also exciting research opportunities across AIGI, including projects led by fellow lab directors pro @maosbot on Bayesian governance and Prof. Amro Awad on hardware governance.
Who should apply?
If you have a strong technical background in machine learning, computer science, mathematics, or related fields, and are excited about pursuing ambitious research that could help make advanced AI systems safer, we'd love to hear from you.
I am committed to partnering with researchers from disadvantage and under represented backgrounds.
How to apply:
Explore the example projects, identify a research direction that interests you, and contact the relevant lab director to discuss your ideas. Your application should clearly address at least one of the AIGI labs.
📅 Deadline: 2 December 2026 (Direct DPhil route)
Please share with talented students and researchers who might be interested!
#PhD #AISafety #AIGovernance #Oxford #AIGI
summerfieldlab @summerfieldlab.bsky.social retweeted
[Hugging Face release] We’re releasing the dataset behind Reason to Play! I’m incredibly proud of our team for reaching this milestone.
@sreejan_kumar Austin Andrews @LHuntNeuro @summerfieldlab Josh Tenenbaum @somnirons @marcelomattar @momchiltomov
I have read Matt's book and you should too. It's a take that you won't find elsewhere - informed by a deep understanding of how tech works, and how the state works. recommend!
My book, AI and Political Freedom, is out today.
It argues that AI safety isn’t only a technical and regulatory problem. No matter what measures we take on those fronts, AI won’t be safe unless we also reinforce the foundations of constitutional democracy. The book lays out specific steps for ensuring that political freedom survives alongside powerful AI.
Link in the first reply.
9
1,426
AISI is an absolutely amazing place to work. come and join us
AISI IS HIRING!
It was less than a month ago that I became Director. I joined with the belief that AISI is a world-leading organisation, and living proof that government can build things that work.
One month in and I'm even more bullish about the vital role @AISecurityInst plays in frontier AI security. I'm astounded daily by the talent, focus and commitment of the brilliant team I get to work with.
There has never been a more urgent time to work on frontier AI security - independently, in the public interest. We have a lot of urgent work to do, our team is growing fast, and we need more brilliant people to fill those roles.
Our Red Team uses adversarial machine learning techniques to find failures in alignment measures, control monitors, and misuse guardrails. They are massively scaling up all parts of the team:
Alignment: job-boards.eu.greenhouse.io/…
Control: job-boards.eu.greenhouse.io/…
Misuse: job-boards.eu.greenhouse.io/…
AI capabilities in cybersecurity and autonomy are advancing faster than ever. Our Cyber & Autonomous Systems team assesses what frontier and open-weight models can really do - across cyber, autonomy and AI R&D. We're hiring a Cyber Security Engineer and Software Engineer to build the evaluations behind that work:
CSE: job-boards.eu.greenhouse.io/…
SWE: job-boards.eu.greenhouse.io/…
Our Human Influence team studies how AI can shift human decisions and behaviour, using everything from RCTs to multi-agent studies. We're hiring an Engineering Lead to help us grow the ambition and pace of that research:
job-boards.eu.greenhouse.io/…
We're also hiring exceptional Software Engineers at all seniority levels to join our Core Technology team, which works closely with researchers to build performant tools and infrastructure that enable and accelerate AISI's world-leading AI safety research:
job-boards.eu.greenhouse.io/…
28
3,201
thanks @NeuroCellPress for publishing our paper on the "recipe for intelligence" (with @neuro_kim). Now out cell.com/neuron/fulltext/S08…
what a great postdoc opportunity!
With Prof. @EmmaBrunskill , we're looking for a postdoc with expertise in LLMs and RL to work on building AI systems for long-term well-being! Please apply by 09/15: forms.gle/YwNrszeJGc1VucQj9
2
2,151
summerfieldlab @summerfieldlab.bsky.social retweeted
‼️ Hiring AI Safety Researchers ‼️
I am moving to Berlin to start a new research group on AI safety and societal impacts @HPI_DE. From September this year, I will be hiring fully funded PhDs, postdocs, and research interns.
My group will study the risks and benefits that emerge as we integrate highly capable AI systems into the social and political processes our societies depend on (e.g. communication, coordination, competition). We will measure these impacts through model evals, computational experiments, human studies, and develop technical mitigations. We will do important research that matters and have fun doing it.
Ideal candidates have a background somewhere between CS/NLP and the social sciences, are excited about working on AI in the current moment, and care deeply about making a positive impact through open research. If that sounds like you, PLEASE GET IN TOUCH using the form in the post below.
I am sad to be leaving @oiioxford and @AISecurityInst but will try my very best to bring the amazing energy of these places and their people to Germany. I am also very grateful to the DFG / Emmy Noether programme for funding my group for the next 6 (!) years.
Please spread the word, and I look forward to hearing from you 🤗
new preprint: how does curriculum learning affect representation formation in transformers, and why does it matter for AI safety? with Sebi Bruijns, Mia Whitefield, @RubruckJirko @akaijsa and @FazlBarez .
arxiv.org/abs/2607.04846
ten years ago, there was a lot of excitement about how neuroscience and cognitive science might help accelerate AI research. What has happened in the decade since? @neuro_kim and I provide answers in this new preprint: osf.io/preprints/psyarxiv/x9…
huge congrats to Jakob and the team!!!
Very excited to launch BOLD 🚀
BOLD is a national shot at ambitious bluesky fundamental AI research in an academic setting, fully committed to open-source and open-science.
BOLD's mission are research bets that would reshuffle the deck in AI if true. It will also be a launchpad for fast-tracking the British and European tech scene.
Oh - and @FLAIR_Ox is no more.
5
2,159
How is generative AI impacting us all? There are lots of opinions. But what does the evidence actually say? Find out here. doi.org/10.31234/osf.io/67eq…
with @hannahrosekirk @KobiHackenburg @AnnaGausen @LLuettgau @rebershak @KJuechems and lots of other great people.
summerfieldlab @summerfieldlab.bsky.social retweeted
New w/ @AISecurityInst & @UniofOxford:
Frontier AI can now out-persuade expert humans in conversation - incl. world-champ debaters and professional canvassers.
This held even when humans chose their topics, prepared in advance, and competed for £1,000 prizes 🧵
summerfieldlab @summerfieldlab.bsky.social retweeted
Anthropic now has a team dedicated to AI and the rule of law — and we've just opened our first role.
@AnthropicAI has studied what AI means for the economy. This team asks a different question: what will it mean for executive power, for courts and elections — and for the public deliberation that constitutional democracy ultimately rests on?
We're looking for someone with real depth in both AI and the law — a legal scholar, political scientist, or experienced government hand who can reason about frontier systems and the institutions they will affect.
If that's you, or someone you know: job-boards.greenhouse.io/ant…
