No single person or institution should define ideal AI behavior for everyone.
Today, we’re sharing early results from collective alignment, a research effort where we asked the public about how models should behave by default.
Blog here:
openai.com/index/collective-…
The changes we adopted through this process will soon be reflected in the Model Spec. We’re also sharing our dataset publicly with the research community: huggingface.co/datasets/open…
Huge congratulations to the team @mitchellgordon, @ezhang7423, @SandhiniAgarwal, and all of the incredible folks at OpenAI who supported this work!!
Tyna Eloundou retweeted
🚨 New paper alert! Excited to share our latest work with the amazing @actuallysoham @jaybaxter and @msaveski introducing Supernotes!
These are LLM-generated @CommunityNotes that synthesize multiple underlying "notes" and are selected by a simulated jury of diverse raters. See more info 👇
🚨🚨 New preprint alert! 🚨🚨
Our new preprint w/ @jaybaxter, @bakkermichiel @msaveski (dream team!) proposes a framework to generate ✨Supernotes✨ - AI-generated notes that synthesize information from several existing @communitynotes and are written to foster consensus among a diverse set of users.
A thread 🧵... (1/10)
Tyna Eloundou retweeted
🚨🚨🚨 VERY excited to share our new paper on how AI can facilitate democratic deliberation, published today in @ScienceMagazine! Together with @mhtessler, @summerfieldlab, and other amazing collaborators at @GoogleDeepMind we've been building the "Habermas Machine"
Tyna Eloundou retweeted
If you’re attending #chi2024 come check out the work being presented by @SocFuturesLab members and collaborators! See below for four upcoming talks and one workshop paper:
Tyna Eloundou retweeted
📖 we just shared the model spec, i.e. a “spec” for openai’s models. it’s a work in progress that we’re sharing for early feedback.
it also features profanity & cats, flat earth theory, and why the model says “sorry, i can’t help with that”.
from a product perspective, i’m personally excited about this concept of a model spec for three reasons:
1. there’ll be more clarity on whether something is a policy or rlhf bug
if a model says or does something that you disagree with, was that intended by openai or an rlhf bug? should you yell at @sama or researchers?? jk please don’t yell at researchers
2. principles are easier to debate and get feedback on, vs. hyper-specific screenshots or abstract feel-good statements
it’s easy for most people to agree on “models should be [adjective]”, but the more important questions lie deeper in thorny scenarios: how should the model engage with someone who claims that the earth is flat?
the model spec introduces language we can use to debate model behavior questions: objectives, rules, defaults, with difficult cases we’ve encountered through real-world use. i’m hoping that these new concepts will enable more nuanced and important discussions.
3. model spec feedback will help us steer our efforts in steerability
an explicit non-goal for the model spec is to reach consensus on a one-size-fits-all model. that will never happen. we want to give users and developers as much control as possible while staying within hard boundaries that people understand.
hearing feedback on where & how everyone wants to steer the model is helpful in (a) designing a more rigorous survey process and (b) informing the research and product roadmap.
a lot of people worked on this — both the model spec and all the thinking that led up to it — and they’re all genuinely excited to hear from you on the hardest questions. so please don’t be shy and tell us your thoughts!
To deepen the public conversation about how AI models should behave, we’re sharing our Model Spec — our approach to shaping desired model behavior. openai.com/index/introducing…
Tyna Eloundou retweeted
“What are human values, and how do we align to them?”
Very excited to release our new paper on values alignment, co-authored with @ryan_t_lowe and funded by @openai.
📝: meaningalignment.org/values-…
Tyna Eloundou retweeted
Thank you @CIFAR_News , it's official.
I am happy to announce that I have been awarded a Canada CIFAR AI Chair 🥳🎉
I am very grateful to CIFAR, @Mila_Quebec and @mcgillu
Replying to @CIFAR_News
David Ifeoluwa Adelani (@davlanade @mcgillu @Mila_Quebec) develops machine learning models for under-resourced languages, such as African, Latin-American and Indigenous languages, creating better automatic translations and text-to-speech services.
cifar.ca/cifarnews/2024/02/1…
Tyna Eloundou retweeted
Now on arXiv! 🌟
Should LLMs offer legal advice?⚖️ The risks of hallucinations and real-world impacts raise concerns. But is a ban the answer, or can LLMs offer some helpful info? We asked 20 legal experts to weigh in.
w/ King Xia, @kevinfeng, @cquanze, and @amyxzh [1/14]
Tyna Eloundou retweeted
10^18 searches where Exa is better than Google:
Just kidding, but here are 10: 🧵
Thrilled to share this update! Working with these 10 teams to me reaffirmed the promise of better tools and processes to better govern our present and our future.
Read about it all in our blog & their reports. Much work remains! I'm hiring researchers to join us on this mission
We provided grants to 10 teams who developed innovative prototypes for using democratic input to help define AI system behavior.
Read about our learnings and how we'll put this work into practice. openai.com/blog/democratic-i…
Tyna Eloundou retweeted
We’re releasing a new dataset for measuring discrimination across 70 different potential applications of language models, including loan applications, visa approvals, and security clearances.
We’ve used this dataset to measure discrimination in LMs and develop new mitigations.

ALT A title card for the paper. Top left: “Evaluating and Mitigating Discrimination in Language Model Decisions. Tamkin et al.” Bottom left: Anthropic logotype. Right: A monochrome image of a stone wall made up of rough, uneven rocks. The wall has fewer stones in the middle, suggesting one could almost climb over it. The wall stretches across a grassy field with hills and trees in the distance. The sky above is cloudy.
Tyna Eloundou retweeted
Pleased to announce the first event of my newly launched "Governing X" conference series 4 the #DemocraticInnovations program at @ISPSYale: 2 back-to-back conferences on "Governing Citizens' Assemblies" & "Governing (with) AI." Website (in progress!) here: campuspress.yale.edu/governi…

















