Associate Professor in Machine Learning at the University of Oxford. Interested in automatic inductive bias selection using Bayesian tools.
Oxford, UK
Joined November 2014
- Tweets399
- Following612
- Followers3.8K
- Likes1.8K
Mark van der Wilk retweeted
Here’s my first attempt to grapple with AI companies’ conquest of mathematics and our new mathematical condition.
argmin.net/p/applied-pure-ma…
With AI-generated research, authors are the first reviewers of the work, and must take responsibility for correctness.
There are already too many papers vying for attention, and wasting attention is a net negative for science.
We need more integrity, not simply more papers.
TMLR has faced a deluge of submissions, necessitating stricter desk rejection policies due to limited reviewer capacity
Co-EiC Nihar Shah reached out to authors of 10 papers slated for desk reject. Could they answer questions about their *own* submission?
medium.com/@TmlrOrg/asking-a…
🎓Interested in PhD positions in my group?🎓
We work on continual learning, causal inference and AI scientists, symmetry & generalization, generative models, dynamically growing neural networks, Bayes, (Deep) GPs, local learning rules, and much more!
Funding deadline: 2 Dec⬇️
There are lots of interesting problems in ML where conceptual and mathematical thinking can make a difference, both at small scale, and at large scale.
If this sounds interesting to you, do apply!
All applications will be considered, even if you haven't been in touch first.
See more about my group on mvdw.uk, where you will also find tips and guidelines for applying.
Apply at: ox.ac.uk/admissions/graduate…
I'm excited by the prospect of this book, and worth reading if @beenwrekt's blog posts are anything to go by.
"The core intellectual idea of the latest AI bubble is taking the initial ideas from the 1940s and riding them to their logical conclusion."
I’m excited to announce that my new book, The Irrational Decision, is available for pre-order from Princeton University Press.
press.princeton.edu/books/ha…
I totally agree that there are a very small number of principles that underpin all of modern-day ML. Expected utility (and Bayes as a closely related consequence) being a Big One. Both lead to strange conclusions if taken to the extreme!
More in our investigation of using Bayesian Model Selection for Causal Discovery: Multivariate Graphs.
As previously, the message is: Causal discovery requires assumptions, and Bayes enables soft, realistic assumptions. Good Bayesian inference then leads to good performance. 1/3
📢New #ICML2025 paper: "Continuous Bayesian Model Selection for Multivariate Causal Discovery".
We propose a Bayesian causal model that allows for scalable causal discovery without restrictive model assumptions.
Paper: arxiv.org/abs/2411.10154
Code: github.com/Anish144/Continuo…
🧵1/14
This does lead to the question, what models should we use, and how should we do inference?
We use a VAE with Gaussian Process mappings (GPLVM), but the idea applies equally to Bayesian NNs, if inference can be made to work!
2/3
In other work, we investigate metalearning as a way to implement these ideas. The advantage being that a generative model can directly learn the conditional distribution of interest, without a bottleneck of approximate inference!
For more on that, see xitter.cf/dhir_anish/status/1913…
3/3
Understanding causes is key to science. Finite observational data alone isn't enough. While Bayes offers a framework to deal with this, the calculations are often intractable. We introduce a method to accurately approximate the posterior over causal structures.
#ICLR2025 🧵1/15
Mark van der Wilk retweeted
Understanding causes is key to science. Finite observational data alone isn't enough. While Bayes offers a framework to deal with this, the calculations are often intractable. We introduce a method to accurately approximate the posterior over causal structures.
#ICLR2025 🧵1/15
Mark van der Wilk retweeted
Excited to be presenting this work at #ICLR2025. Please do reach out if you are interested in a similar space!
🗓️: Hall 3 + Hall 2B #471
🕐: Fri 25 Apr, 3 p.m.
📜: openreview.net/forum?id=eeJz…
This was a great collaboration w/ @_MatthewAshman, @jamesrequeima, @markvanderwilk
Mark van der Wilk retweeted
After using Deep Research for a while, I finally get the "it's just slop" complaint people have about AI art.
Because I don't care much about art, most AI art seems pretty good to me. But information is something where I'm much closer to a connoisseur, and Deep Research is just nowhere near a good human output. It's not useless, I think maybe ~20% of the time I get something I'm satisfied with. Even then, there's this kind of hall-of-mirrors quality to the output, I can't fully trust it, it's subtly distorted. I feel like I’m wading through epistemic pollution.
Obviously it’s going to improve, and probably quite rapidly. If it read 10x more sources, thought 100x longer, and had 1000x lower error rate, I think that would do it. So no huge leap required, just turning some knobs, it’s definitely going to get there. But at the same time, it’s quite jarring to me that a large fraction of people already find the outputs compelling.
Mark van der Wilk retweeted
I agree with @AmandaAskell. It is a fallacy. It can be appropriate to say both a) that LLMs are next-token predictors, at a mechanistic level, and b) that they have understanding, at another level. (a) is an engineering fact. But the negation of (b) does not follow. 1/2
I am genuinely astounded by this tweet, and from someone with philosophical training no less.
There is so much empirical evidence that LLMs stray from truth that the word “hallucinate” became the word of the year in 2023. People are desperately trying to find fixes for that problem. Amazon just set up a whole division to work on the problem.
And yet this person, Askell, an Anthropic employee, wants by some sort of verbal sleight of hand to deny both that LLMs are next-token predictors (which they obviously are) and to pretend that we haven’t seen years of evidence that they are factually challenged.
Good grief.
Mark van der Wilk retweeted
I'm recruiting a new PhD student on Gaussian Belief Propagation for Spatial AI. For this particular opportunity: 1. The funding I have is only available to a student who qualifies for "home fees" in England; 2. I'm looking for someone who can start soon, ideally April 2025.