Professor of Statistical Machine Learning at the University of Oxford, Head of RainML Research Lab (https://xitter.cf/t.co/uuBwQ2WdMN)
Oxford, England
Joined November 2016
- Tweets374
- Following292
- Followers7K
- Likes481
Tom Rainforth retweeted
Active testing enables label-efficient model evals but can be computationally expensive.
We show how to reduce costs and scale up to LLMs.
arxiv.org/abs/2508.09093
Work led by Gabrielle Berrada. Find her at EurIPS, or @janundnik and me at NeurIPS in San Diego.
Tom Rainforth retweeted
Apple and Oxford just made AI 6.5x better at problem-solving.
The secret: it teaches AI agents to ask perfect questions. This rockets success rates from 14% to 91%.
No need for fine-tuning or retraining. It runs on current models.
Here's how it works:
It's a strategic loop designed for multi-turn conversations. At every step, the agent works to find the shortest path to the right answer.
Hypothesize: The agent creates an internal list of all possible solutions to the problem.
Score Questions: It simulates asking various questions and scores each one on "Expected Information Gain" (EIG). This number represents how much a question is mathematically likely to shrink the list of possibilities.
Ask the Best Question: It asks the user only the single, highest-scoring question.
Update & Repeat: Based on the answer, it filters its list of hypotheses, getting smarter with each interaction, and then begins the loop again for the next turn.
Why this matters for your AI strategy:
This marks a shift from building passive "oracles" to proactive, question-asking agents
Business Leaders: A 6.5x multiplier on task success is a lever for efficiency. This translates to fewer failed customer interactions, faster diagnostics, and more accurate personalization, a clear ROI on smarter AI.
Practitioners: This is a deployment-time framework, not a new model. You can build this agent on top of existing LLMs today. It provides a principled way to overcome common multi-turn issues like inconsistency and context loss without fine-tuning or retraining.
Researchers: This paper is a victory for information theory. It proves that a full EIG calculation is superior to heuristics like predictive entropy. It sets a new standard for how to build intelligent information-seeking agents.
Tom Rainforth retweeted
Are you at ICLR?
Have you heard that In-Context Learning in LLMs does not learn label relationships?
Well that's not true.
Visit our poster TODAY to find out how LLMs incorporate label information.
Spoiler: it's not Bayesian inference.
Poster #129, May 7, 4.30 pm
Tom Rainforth retweeted
I will be presenting our work on "Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support" at AISTATS in Valencia tomorrow (details in thread below).
If you are interested in probabilistic programming, come and say hi at poster session 1!
Our new paper (arxiv.org/abs/2310.14888) shows that probabilistic programs with stochastic support are implicitly Bayesian model averages (BMA) which leads to issues if we assume our model is misspecified!
w/ @TimReichelt3 and Luke Ong
A thread (1/5)
Tom Rainforth retweeted
The current default recipe for Bayesian active learning doesn’t really work beyond MNIST scale.
We suggest why that is and identify a simple fix.
arxiv.org/abs/2404.17249
@aistats_conf with @adamefoster @tom_rainforth
1/5
Tom Rainforth retweeted
It is @NeurIPS time again! I am excited to present our trans-dimensional jump diffusion work with @AndrewC_ML @willarvey @ValentinDeBort1 @tom_rainforth and @ArnaudDoucet1 ! Come over on Thursday 2nd poster session, neurips.cc/virtual/2023/post…. arxiv.org/abs/2305.16261 #NeurIPS2023
Tom Rainforth retweeted
Interested in large language models? Worried about impacts of climate change? Come join us @oxcsml @NatureRecovery @UniofOxford in pushing the frontiers in #LLMs and at the same time help #NatureRecovery and address the impacts of #ClimateChange!
bit.ly/4750IBO
Tom Rainforth retweeted
We construct neural processes by iteratively transforming a simple stochastic process into an expressive one, similar to flow/diffusion-based models, but in function space!
Join us at our #NeurIPS2023 poster session: neurips.cc/virtual/2023/post… on Wednesday morning!
Tom Rainforth retweeted
⚡️I and Prof. Torr have a PhD opening @ Oxford Uni TVG (@OxfordTVG)
🥶 Sorry the deadline is pretty close 22Sept23
👉 details aims.robots.ox.ac.uk/study/i…
Topic is broad & flexible--foundation models & challenges therein
Oxford undoubtedly is a great place 👨🎓👩🎓. Please apply 🎓🎓
Tom Rainforth retweeted
Have you heard that In-Context Learning in LLMs does not learn label relationships?
Our new paper shows this is usually not true – even for smaller models with < 10B parameters.
We also study *how* LLMs incorporate label information in a variety of experiments.
🧵1/N
Tom Rainforth retweeted
I’ll be at ICML🌺 to present my @Microsoft Research internship project “CO-BED: Information-Theoretic Contextual Optimization via Bayesian Experimental Design” — work done with Joel Jennings, @tom_rainforth, Cheng Zhang and @AdamEFoster
1/7
Tom Rainforth retweeted
Conformal prediction provides valid prediction regions for any machine learning model, any data distribution and and any dataset size.
But what determines the size of prediction sets and can this itself size be predicted? I turns out that the answer is yes 🔥🔥🔥🔥 as shown in interesting new paper from University of Oxford by Guneet Singh Dhillon, George Deligiannidis and Tom Rainforth
arxiv.org/pdf/2306.07254.pdf
”While conformal predictors reap the benefits of rigorous statistical guarantees for their error frequency, the size of their corresponding prediction sets is critical to their practical utility. Unfortunately, there is currently a lack of finite-sample analysis and guarantees for their prediction set sizes. To address this shortfall, we theoretically quantify the expected size of the prediction set under the split conformal prediction framework. As this precise formulation cannot usually be calculated directly, we further derive point estimates and high probability intervals that can be easily computed, providing a practical method for characterizing the expected prediction set size across different possible realizations of the test and calibration data. Additionally, we corroborate the efficacy of our results with experiments on real-world datasets, for both regression and classification problems.”
#conformalprediction #machinelearning
Tom Rainforth retweeted
We introduce a new class of stochastic process models, which are constructed by stacking sequences of neural parameterised Markov transition operators in function space.
🗞️ arxiv.org/abs/2305.15574
w/ @emidup @kasparmartens @tom_rainforth
@yeewhye
A thread
Tom Rainforth retweeted
How can we apply diffusion models to data with varying dimensionality? We use jump diffusions to simultaneously generate the size and state values for varying size data e.g. molecules
arxiv.org/abs/2305.16261
w/ @willarvey @wh1lo @ValentinDeBort1 @tom_rainforth @ArnaudDoucet1
Tom Rainforth retweeted
Poster 27 at 4:30-7pm today:
BALD 👴🏼 vs 🐷 EPIG
Come and chat!
#AISTATS2023 @aistats_conf
Maximise expected info gain in model params (aka BALD) → effective active learning? Sadly not!
EPIG, our alternative, shares BALD’s 1950s foundations but targets predictions.
arxiv.org/abs/2304.08151
#AISTATS2023 w/ @blackhc @seb_far @yaringal @adamefoster @tom_rainforth
1/5
Tom Rainforth retweeted
Maximise expected info gain in model params (aka BALD) → effective active learning? Sadly not!
EPIG, our alternative, shares BALD’s 1950s foundations but targets predictions.
arxiv.org/abs/2304.08151
#AISTATS2023 w/ @blackhc @seb_far @yaringal @adamefoster @tom_rainforth
1/5
Tom Rainforth retweeted
Do Bayesian Neural Networks need to be fully stochastic? In our ✨ AISTATS Oral 🌼, we answer with a resounding "no".
Partially stochastic networks are
- just as expressive
- just as principled
- and often better performing
than more costly fully stochastic networks
details👇🏽
Tom Rainforth retweeted
Not all data is created equal!
How can we decide what to acquire?
Bayesian experimental design (BED) is a principled framework that has recently seen a surge of interest.
Our review of modern BED, building up from the basics to today’s frontier:
arxiv.org/abs/2302.14545
1/4
Tom Rainforth retweeted
Quick thread on a cool paper with surprising results I read this week: "Do Bayesian Neural Networks Need To Be Fully Stochastic?" by @MrinankSharma @seb_far @eric_nalisnick @tom_rainforth 🧵(1/7)
arxiv.org/pdf/2211.06291.pdf
