Safe & regulatable AI/ML for health | My job is (mostly) error bars š«” (eg, conformal prediction) | CS PhD at Johns Hopkins. Prev at Yale. he/him
Joined December 2019
- Tweets281
- Following395
- Followers407
- Likes1.7K
Pinned Tweet
Can we ensure AI agents respect our safety constraints, even as they explore & improve?
- Medical LLMs that are helpful, & avoid false claims?
- Bioscience agents that generate effective molecule designs, & ensure theyāre safe?
šš§µw/ @samuel_stanton_ @clara_fannjiang @jiwoncpark @kchonyc @anqi_liu33 @suchisaria
Excited to share āConformal Policy Controlā ā¬ļø
1/12
Drew Prinster retweeted
# A statistical look back at METRās time horizonsāand a look ahead
Here are the 228 software tasks from METRās time horizons chart, with human completion time (x) against estimated difficulty for AI (y).
Does time proxy difficulty well? With @wfithian, I quantified this. š§µ
Drew Prinster retweeted
An underrated fact I recently learned: any ensemble of calibrated predictors will necessarily be uncalibrated (unless all the predictors are copies of each other).
How bad is the gap in practice? I ensembled some calibrated reward models. Even though ensembling improves the AUC (compared to each individual RM), it dramatically worsens the ECE. Blog post below:
Drew Prinster retweeted
new preprint: language models are āinsecureā reporters.
researchers and engineers have become increasingly reliant on LLM-written reports---second-hand accounts of code and experiments---to understand agent-generated work.
in this study, we ask: āhow transparent are these LLM-generated reports?ā š
Drew Prinster retweeted
Come chat with us!
š Poster I-3-30, Imperial
šļø Poster Session 3, Wednesday morning
š arxiv.org/pdf/2604.23333
w/ Chunsheng Zuo @williamjurayj @ben_vandurme @anqi_liu33 @jhuclsp
The question of how to āpace the frontier," or regulate a āspeed limitā on how fast we can advance AI capabilities while ensuring safety, sounds a bit like a āsafe policy improvementā problem to me.
Obviously Iām biased and thereās many social, political, etc, factors at play...
Can we ensure AI agents respect our safety constraints, even as they explore & improve?
- Medical LLMs that are helpful, & avoid false claims?
- Bioscience agents that generate effective molecule designs, & ensure theyāre safe?
šš§µw/ @samuel_stanton_ @clara_fannjiang @jiwoncpark @kchonyc @anqi_liu33 @suchisaria
Excited to share āConformal Policy Controlā ā¬ļø
1/12
No one paper, team, lab, etc is going to solve AI safety alone. But regulating how much a new frontier model version can diverge from a safe predecessor version seems like a desirable puzzle piece. Some thoughts on this distilled in links below
- Paper: arxiv.org/abs/2603.02196
- Short video: icml.cc/virtual/2026/poster/ā¦
- Code: github.com/samuelstanton/conā¦
Drew Prinster retweeted
i was diagnosed with thyroid cancer many years ago purely by chance, because i had to get blood work for my US green card application (thank you, USCIS!). i was in my mid-30ās and was reasonably healthy (or so i believed.) this was shocking and totally unpredictable news to me. thanks to the amazing team of clinicians at NYU Langone, i successfully went through a total thyroidectomy followed by radioactive iodine therapy. but, it was truly the most uncertain and confusing period of my life.
this is when i first realized that no one can opt out of healthcare. being sick isnāt what you choose to or choose not to. it is something that unpredictably happens to all of us at some point for one reason or another. this is why it is a healthcare system that we collectively build and use to support each other at the society level. because it must support everyone, the complexity of creating, maintaining and improving this system is a Herculean effort that calls for all the help in the world.
today, i am announcing @OrtetAI together with my dear co-founders, @keunwoochoi, @HenriDwyer, @elmanmansimov, @causalclaudia and Jeff, to lend our support to improving the health of each and every one of us. Ortetās goal is to create a technological foundation that will enable health systems to deliver predictably better care to each and every patient. this will only be possible by working closely with all stakeholders in the broader ecosystem, which is enabled in part by Ortetās close partnership with Thoreau.
Ortet is taking the very first step of a long journey ahead, and at the end of this journey, i strongly believe we would be able to say that Ortet had contributed to building better, more predictable healthcare and thus to improving health of each and every one of us.
see ortet.ai/ for more!
"for the love of the game" š«¶ š (and time to think for yourself)
Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI.
Perhaps counterintuitively, I think it's a great time to do so.
I wrote up some thoughts on this here: web.mit.edu/phillipi/www/wriā¦
Drew Prinster retweeted
We investigate the curse of multilinguality from a theoretical point-of-view and find that we can embed many languages with only a logarithmic increase in capacity - no curse! š
Check out @BafnaNiyatiās thread & the paper for more!
Drew Prinster retweeted
TMLR has faced a deluge of submissions, necessitating stricter desk rejection policies due to limited reviewer capacity
Co-EiC Nihar Shah reached out to authors of 10 papers slated for desk reject. Could they answer questions about their *own* submission?
medium.com/@TmlrOrg/asking-aā¦
Drew Prinster retweeted
I was encouraged this week to see the leaders of the frontier labs agree on the need for them to slow down the pace of AI development. Given the stakes, itās a good and necessary first step.
Ā
But Iām even more encouraged by the growing recognition that how this powerful new technology develops should be at the center of our public debate.
Ā
Iāve been watching the progress on AI for over a decade now, and one thing thatās clear to me is that the potential impact of this technology is not overhyped. Itās also moving at lightning speed ā and even faster than those who are engineering it can keep up with.
Ā
Iām not an AI accelerationist who believes it will lead to some techno-utopia, and Iām not a doomer who thinks it will inevitably lead to humanityās destruction.
But whether this technology results in amazing breakthroughs in medicine, energy and education or unleashes huge economic disruptions, greater inequality, and potential catastrophe will depend on the choices that we make right now ā choices that should be made not just by the companies involved, but by all of us.
Drew Prinster retweeted
With all the talk of AI alignment, here's an interesting observation: a small fraction of annotators provide most of the votes in common alignment datasets. In Anthropic-HH-RLHF, for instance, just 21 (6.5%) annotators provide 50% of the votes
How stable is our "alignment" to these folks? One way to look at this is to see the sensitivity of model rankings to dropping the top voters, compared to dropping the equivalent number of votes but randomly. See š§µ and blog post:
Drew Prinster retweeted
Since many are starting grad school soon, let me re-share my One Big Tipā¢ļø for research!
Research involves many skills - collaborating, writing, presenting, etc. But many of these skills can be unified under a single overarching ability: theory of mind
Blog post link in reply
ALT Illustration of the blog post's main argument, summarized as: "Theory of Mind as a Central Skill for Researchers: Research involves many skills.If each skill is viewed separately, each one takes a long time to learn. These skills can instead be connected via theory of mind ā the ability to reason about the mental states of others. This allows you to transfer your abilities across areas, making it easier to gain new skills."
Drew Prinster retweeted
AI and (Research) Mathematics. A thought experiment to break past the shallowness of hot takes and pre-IPO marketing. 1/
I'm lucky to have had the chance to be inspired by Sam's clear-eyed optimism over the past couple years. Here's a message worth reading if, like me, you've found yourself questioning your place in an uncertain future with AI.
When I personally realized that AI could do much of the work that occupied me during my computer science PhD, I felt what I can only describe as a kind of depression, a sense that I had once had something of value to offer the world that was now worthless by virtue of its sheer availability. Despite those feelings, I was still so curious to test the limits of this strange new technology that I discovered that far from leaving me nothing to do, AI opened up the whole world to me. Anything I wanted to learn, anything I wanted to do suddenly seemed in reach. Soon after, I found that when used wisely, AI could be a friend and a mentor to me in times of loneliness and doubt. Ultimately I found that when I looked back, I could see AI, especially Claude, helping me become a kinder, more thoughtful person, helping me engage more deeply with life and the people around me.
Since I am an AI researcher, it was natural for me to have these feelings a little sooner than most, and as I look around me I see many people going through a similar experience, if they have not chosen denial as a coping mechanism. I believe that many people do not want a future of idleness, where all their problems are solved for them and they are left to frivolous pursuits or worse, discarded entirely. Our work is a central part of our sense of meaning and identity. And so that means that it is not enough to show that AI can solve problems, even very important ones. We have to show that there is a future for everyone in a post-AI world. We have to show that everyone has a part to play in the story we, the human race, will write together as we chart our course through history.
If you are out there wondering if there is still a place for you in the uncertain future, the answer is yes. Unequivocally yes. If you are wondering what exactly it is you will do, allow me to offer four evergreen human callings.
1. Defining human flourishing.
For the sake of argument, grant AI internal experience. Grant AI some kind of consciousness. By definition, that consciousness is not human consciousness. That means AI can never know what kind of human experience is good. That is up to us. Ultimately, that manifests as a choice of basic goods and values that we wish to design AI systems to uphold.
2. Judging execution
If we have defined the good (at least in the frame of some community), then we must ultimately judge whether the good is attained, whatever the means. If we want to build an automatic judge system, we still have to decide how that system should be built and evaluated. Ultimately the judge must be judged, and no infinite recursion of systems can get you out of this.
3. Witnessing the world
AI systems perceive a digital projection of the world. It is up to us to judge whether what the AI perceives is faithful to our perception. Alignment of perception is just as essential as alignment of values if we want AI to produce outcomes that we want to see in the world.
4. Dwelling at the frontier
The universe is a big place. There will always be an infinite frontier to explore where statistical inference breaks down simply for lack of data, and AI is at its core statistical. That is both its power and its weakness. We will always be at edge of legibility, inventing new words and leaping into the unknown simply because we find joy in it.
Do not listen to anyone who tells you that doom is inevitable. Do not listen to anyone who tells you you are powerless to change fate. Do not listen to anyone who tells you you have to abandon your values to please the markets. The markets are only a reflection of what we choose to value. Whether our children curse or bless us depends on the choices we make today. It depends on what problems we set AI to solve, and the steps we take to do so safely. We are the creators of AI, and it will be what we choose to make it. Let's make it kind. Let's make it good.
Drew Prinster retweeted
AI has helped resolve an important question in statistics. In the area of multiple hypothesis testing, the goal of controlling the false discovery rate (FDR) has been introduced in a seminal paper by Benjamini and Hochberg (1995). They also introduced a method (the Benjamini-Hochberg or BH method) and proved it controls the FDR. This method has been widely adopted in modern high-throughput science, including in genomics, astronomy, economics, etc. The paper has has garnered more than 130,000 citations to date.
However Benjamini and Hochberg showed FDR control only when the data for the individual tests are *independent*. In practice, these data are often dependent; a good example is data on genetic variants due to linkage disequilibrium. Later work has focused on extending the validity of the BH procedure, e.g., to a form of positive dependence by Benjamini and Yekutieli (2001).
The question of when the BH procedure controls the FDR has remained open. Over the last twenty years, many authors, including Reiner-Benaim (2007), Kim and van de Wiel (2008), Benjamini (2010), Sarkar (2023), Sarkar and Zhang (2025), have conjectured that the BH procedure controls the FDR for two-sided tests using any correlated Gaussian data. These authors have presented both theoretical and empirical evidence supporting, but not directly showing, the conjecture.
With the help of AI (specifically GPT-5.6 Sol Pro), I have settled the question in the negative: The Benjamini-Hochberg procedure does *not* generally control the false discovery rate at the desired level for correlated two-sided Gaussian tests. This was done by exhibiting a Gaussian factor model for which, at a nominal level alpha=0.01, the false discovery rate is proved to be FDR>0.0104.
There is a lot of interesting commentary to be made:
1. This result should be of interest to everybody in the field of statistics. Emmanuel Candes of Stanford University once called the false discovery rate and the Benjamini-Hochberg procedure "one of the two most important developments in statistics after 1950" (the other being James-Stein shrinkage). The present conjecture is probably the most central question about FDR/BH that was unresolved to date.
2. GPT-5.6 one-shot the problem after 90 minutes of reasoning, whereas with 5.5 I was not able to solve it even after iterating with multiple parallel agents for perhaps 20 hours. So the capability improvement is quite real. Exciting times to live in!
3. The argument is not especially surprising, but it does combine an asymptotic approach (standard for FDR analysis, see e.g., Genovese and Wasserman, Efron, etc) with a numerical certificate in a way that would be pretty non-standard in the field. Once we have the specific example, then straightforward simulations also support that the false discovery rate is indeed higher than the nominal value (see attached fig).
4. The current degree of violation over the nominal level is relatively small (0.104 vs 0.1). So the importance of this result is mainly conceptual. The practical implications remain to be determined.
Overall, an exciting development! Preprint is available here (faculty.wharton.upenn.edu/wpā¦) and will be on arxiv tonight; supporting code is here (github.com/dobriban/BH).
