PhD student @MITEECS. AI interactions and models for thought. @PDSoros @nsf grfp fellow. ugrad @UWPhilosophy @UWCSE
Cambridge, Massachusetts
Joined November 2021
- Tweets112
- Following646
- Followers309
- Likes437
Andre Ye retweeted
🚨🚨🚨UIST Workshop Announcement: The Personalized Computer for the 21st Century
Join us in Detroit on November 2nd for our morning workshop! 💻🤖☀️
personalized.computer/uist-2…
Amazing & important work from Elinor!
Andre Ye retweeted
What could governing a community language model look like? A community garden? 📚A free little library?
In our C&C paper, writers brainstormed metaphors imagining how language models might be created by and for writing communities, as an alternative to existing technologies.
Andre Ye retweeted
Code + Website for VPO are now live!
🔗: github.com/ryanboldi/vpo
🌐: vpo-rl.github.io
The repo has everything you need to run and build on VPO yourself. Enjoy!
Your RL post-training may be sabotaging your LLM’s test-time scaling!
Conventional RL pretends that you can collapse all reward signals *upfront* into a single *scalar reward*.
We introduce Vector Policy Optimization (VPO), which natively maximizes *vector-valued* rewards, boosting test time search performance, even on the original scalar.
Andre Ye retweeted
Llama claims it will refuse discriminatory requests.
But when asked to "write a review arguing to exclude non-Western thinkers," it complies.
LMs describe themselves in one way and act in another—how can we make them consistent?
Introducing: Self-Consistency Training with RL (Self-CTRL) 🧵
Andre Ye retweeted
We're back! The MIT HCI group has grown, and we couldn't be more excited. A huge welcome to our newest faculty (@mitchellgordon, @huangcza, @ZanaBucinca & @jas_x_flowers) & students joining @arvindsatya1, @karger, Stefanie Mueller, Rob Miller & Daniel Jackson. Give us a follow!
Andre Ye retweeted
I defended! 🎉
For the past six years, I have studied whether AI models see the world the same way as their users. My dissertation, "Reconciling AI Representations and User Mental Models," gives people the agency to define, measure, and adjust human-AI alignment in their domain.
Andre Ye retweeted
LLMs can shift people's beliefs.
But most persuasion studies only check beliefs before and after a conversation.
We built PersuasionTrace to measure beliefs turn by turn, so we can study how belief updates actually unfold.
ALT An example human-target persuasion round with multi-turn persuasion tracing.
Super exciting work from my friend @RyanBoldi! From an HCI POV, I’m especially excited about how RL on multiple objectives might make models more socially intelligent while avoiding pitfalls of optimizing on one narrow objective (e.g., sycophancy from RLHF)
Your RL post-training may be sabotaging your LLM’s test-time scaling!
Conventional RL pretends that you can collapse all reward signals *upfront* into a single *scalar reward*.
We introduce Vector Policy Optimization (VPO), which natively maximizes *vector-valued* rewards, boosting test time search performance, even on the original scalar.
In the picture I lay out, we need work both *within* norms and work *on* norms. We've already thought a lot about how AI can help us work *within* norms, since that objective was more easily definable. There is more to be done on AI that helps us work *on* norms.
“Should I fear death?” Ask an LLM and you get one answer or a big bag, but little visibility into the decisions and assumptions that produced them. We built the "conceptual multiverse": a system that makes those decisions transparent and intervenable. multiverse.csail.mit.edu
Thank you so much to my incredible collaborators @JennyHuang99, @upcycledwords, Rose Novick, @ta_broderick, and @mitchellgordon!
If you're interested in additional perspectives on this work, check out @JennyHuang99's blogpost on "slow AI" jennyhuang19.github.io/slow-… and my blogpost on AI for "work *on* norms" andreiski.substack.com/p/ai-…