computational social psych + AI evals for safety / social intelligence @Gates_Cambridge @Cambridge_Uni | prev. @GoogleDeepMind @GoogleAI @nokia | he/him 🏳️🌈
Cambridge, England
Joined October 2016
- Tweets142
- Following557
- Followers241
- Likes3.3K
Pinned Tweet
Absolutely thrilled speak with @BBCWorld's @AnnitaBBC on 'The World Today' this evening about our personality testing framework for AI systems (now out in @NatMachIntell!) and why it matters for safety.
Catch the full interview below! 👇
@GoogleDeepMind @Cambridge_Uni @BBCNews – at London, England
Greg Serapio-García retweeted
Looking for an interesting read over the winter holidays? ❄️📚
Try this new paper by @gregserapio &Co 👇
🔗nature.com/articles/s42256-0…
Ask yourself, what is the personality of *your* favourite chat bot?🤔
#psychology #personality #ai #language #models #deepmind #cambridge
Absolutely thrilled speak with @BBCWorld's @AnnitaBBC on 'The World Today' this evening about our personality testing framework for AI systems (now out in @NatMachIntell!) and why it matters for safety.
Catch the full interview below! 👇
@GoogleDeepMind @Cambridge_Uni @BBCNews
Check out our paper here :)
nature.com/articles/s42256-0…
@mustafa_safdari @edabeau @peterromero @AleksandraFaust @CHIA_Cambridge – at London, England
Greg Serapio-García retweeted
Congratulations, @gregserapio! And on your interview on the BBC below.
🚨 Everyone's talking about 'personality' in their favorite chatbot 🤖💬—here’s our psychometrics-backed take on how to measure and steer it. 📊🎛️ SUPER proud to share our paper that's landed in @NatMachIntell 🧠(@GoogleDeepMind x @Cambridge_Uni)! 🎓🔬✨
nature.com/articles/s42256-0…
Greg Serapio-García retweeted
Can #AI chatbots have a measurable personality - and can it be shaped? A #CambridgeJudge & @GoogleDeepMind study unveils the first validated AI personality test framework.
Read more: loom.ly/d8XLGzg
#Psychometrics #Personality #LLMs #DeepMind #CambridgeJudge
Greg Serapio-García retweeted
Can AI models have a "personality traits"? And if so, can we control it?
After years of work with @GoogleDeepMind and @Cambridge_Uni, our new paper in @NatMachIntell provides the first scientifically validated framework to answer "Yes." 🧵👇
nature.com/articles/s42256-0…
🚨 Everyone's talking about 'personality' in their favorite chatbot 🤖💬—here’s our psychometrics-backed take on how to measure and steer it. 📊🎛️ SUPER proud to share our paper that's landed in @NatMachIntell 🧠(@GoogleDeepMind x @Cambridge_Uni)! 🎓🔬✨
nature.com/articles/s42256-0…
This work wouldn't have been possible without this brilliant team across three continents: @mustafa_safdari, Clément Crepy, @edabeau, Stephen Fitz, @peterromero, @marwaabdulhai, @AleksandraFaust, & Maja Matarić :)
@Gates_Cambridge @CambPsych @CamPsych @CambridgeJBS @stjohnscam
2) We can't just trust any old personality test at face-value as an eval, here! These tests were made for humans, not LLMs. We need explicit reliability/construct validation criteria: do these tests actually measure what they claim to measure??
Greg Serapio-García retweeted
Gregory Serapio-García is co-first author on a paper which outlines a new personality test for AI #chatbots and shows how 'personality' can be manipulated - gatescambridge.org/about/new…
@CambTweetPBS @Cambridge_Uni @GatesAlumni @gatesfoundation
Proud to see @GoogleAI’s latest blog post feature responsible AI + rater diversity research I’ve worked on w/ @laroyo, @AliciaVParrish, @blahtino, @justaturbo, @vinodkpg, @alxndrt, and others! #GoogleResearch
The Building Responsible AI Data & Solutions (BRAIDS) team within #GoogleResearch aims to simplify adoption of RAI practices through the utilization of scalable tools, high-quality data, streamlined processes and novel research. Learn about their work ↓ goo.gle/40HgSzs
Greg Serapio-García retweeted
My students and I are running a creative writing competition with a twist. You'll write three short pieces, but sometimes you'll work by yourself and sometimes with an AI. Winner gets £500 and 5 runners up get £100 each. Takes 40 mins, have a go & pls RT!
tinyurl.com/camcreativewriti…
How do we probe for particular moral stances mimicked by LLMs? What are the potential risks and downstream consequences of prompting models with moral stances? Check out this cool project I worked on at @GoogleAI with @marwaabdulhai + @natashajaques!
What moral stance do large language models inherit from their training data? We use Moral Foundation Theory to analyze LLMs, showing they exhibit moral foundations most similar to politically conservative humans.
sites.google.com/view/moral-…
Greg Serapio-García retweeted
A new APA report outlines the many obstacles faced by psychology faculty of color when pursuing tenure or promotion—including financial constraints, workplace complexities, and "inequity taxes" related to their engagement in social justice issues. apa.org/news/press/releases/…
Greg Serapio-García retweeted
So excited to have this paper be at @NeurIPSConf D&B track. This is the flagship paper (a few more to follow) out of a multi-year cross-org collaboration that was led by the amazing @laroyo. It adds to the recent line of papers interrogating disagreements in data annotations.
Very excited that our DICES - #Diversity in #ConversationalAI #Safety #Evaluation paper arxiv.org/abs/2306.11247 got accepted at the @NeurIPSConf Datasets and Benchmarks Track
with @vinodkpg @AliciaVParrish @alxndrt Mark Diaz, Christopher Homan, Greg Serapio-Garcia, Ding Wang
Greg Serapio-García retweeted
Very excited that our DICES - #Diversity in #ConversationalAI #Safety #Evaluation paper arxiv.org/abs/2306.11247 got accepted at the @NeurIPSConf Datasets and Benchmarks Track
with @vinodkpg @AliciaVParrish @alxndrt Mark Diaz, Christopher Homan, Greg Serapio-Garcia, Ding Wang
I’m excited to share this brilliant documentary from the @bbcworldservice, where I was interviewed by @ellielhouse on AI recommendation algorithms and their impact on LGBTQ people :) Out today! bbc.co.uk/programmes/w3ct5lc…
A sneak preview of my @bbcworldservice documentary, investigating how recommendation algorithms work, and what that means for LGBTQ people around the world 🏳️🌈
bbc.co.uk/news/technology-66…
Greg Serapio-García retweeted
Google is hosting the first "Machine Unlearning" challenge. Yes you heard it right - it's the art of forgetting, an emergent research field.
GPT-4 lobotomy is a type of machine unlearning. OpenAI tried for months to remove abilities it deems unethical or harmful, sometimes going a bit too far.
Unlike deleting data from disk, deleting knowledge from AI models (without crippling other abilities) is much harder than adding. But it is useful and sometimes necessary:
▸ Reduce toxic/biased/NSFW contents
▸ Comply with privacy, copyright, and regulatory laws
▸ Hand control back to content creators - people can request to remove their contribution to the dataset after a model is trained
▸ Update stale knowledge as new scientific discoveries arrive
Check out the machine unlearning challenge: ai.googleblog.com/2023/06/an…
