Frontier AI @GoogleDeepMind. Generative modelling, Robotics and AI4Science.
London, England
Joined December 2010
- Tweets25
- Following169
- Followers90
- Likes155
Pinned Tweet
(1/6) Can Autoregressive Models (ARMs) go beyond fixed or random generation orders and learn the optimal order to generate new samples?
We believe so! We are excited to present our paper, “Learning-Order Autoregressive Models with Application to Molecular Graph Generation,” at ICML 2025 in Vancouver on July 18th. Feel free to drop by!
In our new paper, we introduce Learning-Order Autoregressive Models (LO-ARMs), which are trained to find a consistent, context-dependent generation order on their own. Our model matches and surpasses state-of-the-art results on two major molecular graph generation tasks.
Read our paper here: arxiv.org/abs/2503.05979
Joint work with @thjashin, Nicolas Heess, @ArthurGretton, Michalis K. Titsias
More in 🧵👇
Zhe Wang retweeted
💯 text diffusion is the future 🚀
new blog post! I'm convinced text diffusion is the future, and we'll start to see it everywhere very soon.
varunneal.github.io/essays/d…
Zhe Wang retweeted
Excellent talk by @thjashin:
- What separates autoregression and diffusion?
- Bridging the gap between continuous and masked discrete diffusion via loss reweighting
- Insertion-based sequence generation
Very clear and easy to follow! (via @mblondel_ml)
youtube.com/watch?v=kMimQxIJ…
Zhe Wang retweeted
AR models append tokens only at the end. But what if generation could insert tokens anywhere?
It's strictly more expressive, but much harder to train with standard max likelihood. We introduce a scalable variational learning method for insertion-based generative models.
arxiv.org/pdf/2606.02133
Zhe Wang retweeted
Variational Learning for Insertion-based Generation
✨Spotlight poster✨ #ICML2026
HALL A #2402
Tue, Jul 7, 2026 • 10:30 AM – 12:15 PM
Zhang, @zhexwang , Ying, @david_van_dijk Titsias, @thjashin
Zhe Wang retweeted
Qiang Liu, Chris Oates, and I are writing a monograph on Probabilistic Inference and Learning with Stein’s Method, and we’d love to get your feedback on the first draft
Zhe Wang retweeted
📢 Job alert
We are looking for a Postdoctoral Fellow to work with @ArthurGretton on creating statistically efficient causal and interaction models with the aim of elucidating cellular interactions.
⏰Deadline 27-Aug-2025
ℹ️ ucl.ac.uk/work-at-ucl/search…
Zhe Wang retweeted
Nice writeup on CTMC and discrete diffusions
Wrote up some notes providing an introduction to discrete diffusion models, going into the theory of time-inhomogeneous CTMCs via generators/time-evolution systems.
What motivated me was the sheer difficulty of finding a useful reference which laid out the theory (e.g. Kolmogorov equations etc.) in a single place. Hope these can be helpful to others.
Zhe Wang retweeted
Google DeepMind has been leading the revolution in Foundation Models for Robotics. Come join us. We are hiring!
It takes a village to build an awesome 🦾 🧠 . We are #hiring! Come transform robotics at Google DeepMind. I've received a lot of request for applied and engineering roles. Indeed we are hiring for those too!
Robotics engineering:
lnkd.in/gDt6CeAq
Applied ML SWEs:
lnkd.in/gCmY69XQ
Research Scientist:
lnkd.in/g47zS2Xz
Zhe Wang retweeted
Accelerated Diffusion Models via Speculative Sampling, at #icml25 !
at 16:30 Tuesday July 15 poster E-3012
arxiv.org/abs/2501.05370
@ValentinDeBort1 @agalashov @ArnaudDoucet1
Zhe Wang retweeted
Distributional diffusion models with scoring rules at #icml25
Fewer, larger denoising steps using distributional losses!
Wednesday 11am poster E-1910
arxiv.org/pdf/2502.02483
@agalashov
@ValentinDeBort1
Guntupalli
@zhouguangyao
@sirbayes
@ArnaudDoucet1
(5/6) Can LO-ARMs learn consistent and meaningful generation orders on their own?
Yes — and the strategy/generation order they discover is surprisingly intuitive.
On two major molecular generation tasks (QM9 and ZINC250k), our models learned a consistent, two-phase process:
1. Build the Skeleton: First, generate the chemical bonds.
2. Infill the Atoms: Then, add the atoms to the skeleton.
We find that 99% of the new samples follow this generation order.
Here’s an example of this in action for generating a ZINC250k molecule. We start with a blank slate where all tokens are masked, and in each step, the model adds one new piece — either an atom or a bond, until the molecule is complete.
Zhe Wang retweeted
📢 We have an opportunity for students to join our PhD programme in Theoretical Neuroscience and Machine Learning this September.
Application deadline is 27 May 2025.
Information & how to apply 👉 ucl.ac.uk/gatsby/study-and-w…
Zhe Wang retweeted
We are hiring a student researcher at Google DeepMind to work on fundamental problems in discrete generative modeling!
Examples of our recent work:
masked diffusion: arxiv.org/abs/2406.04329
learning-order AR: arxiv.org/abs/2503.05979
If you find this interesting, please send an email to: {jiaxins,mtitsias} AT google DOT com
Zhe Wang retweeted
Excited to share details of AlphaGeometry2 (AG2), part of the system that achieved silver-medal standard at IMO 2024 last July! AG2 now has surpassed the average gold-medalist in solving Olympiad geometry problems, achieving a solving rate of 84% for all IMO geometry problems over the last 25 years, compared to 54% previously! Figure is a crazy problem solved elegantly by AG2 (for the first time according to our knowledge!). Will follow up with more details later in a thread.
Paper: arxiv.org/abs/2502.03544
Nature article: nature.com/articles/d41586-0…
Authors: Yuri Chervonyi, Trieu H. Trinh, Miroslav Olsak, Xiaomeng Yang, Hoang Nguyen, Marcelo Menegali, Junehyuk Jung, Vikas Verma, Quoc V. Le, Thang Luong
Special thanks to Dawsen Hwang, Edward Lockhart, and Steven Creech for contributions to the development of AlphaGeometry2. We would also like to thank Yifeng Lu, Henryk Michalewski, Ed Chi, David Silver, Pushmeet Kohli, and Demis Hassabis for their thoughtful discussions, help and support.
Super thrilled to share that our AI has has now reached silver medalist level in Math at #imo2024 (1 point away from 🥇)! Since Jan, we now not only have a much stronger version of #AlphaGeometry, but also an entirely new system called #AlphaProof, capable of solving many more Olympiad problems. This is a large-scale project that I was fortunate to co-lead at @GoogleDeepMind! See our blog & NYT articel below!
Blog: dpmd.ai/imo-silver
NYT: nytimes.com/2024/07/25/scien…
Zhe Wang retweeted
Still a week to submit your work to ICLR 2025 workshop on world models!
🚀 Call for Papers: ICLR 2025 Workshop on World Models! 🌍🤖
📅 Submission Deadline: 10th Feb 2025 23:59 AOE
🌐 Website: sites.google.com/view/worldm…
We invite submissions on understanding, modeling, and scaling #WorldModels—from knowledge extraction to model-based RL, multimodal world models, and their applications in AI, robotics, and scientific discovery.
📩 Join us in shaping the future of AI-driven world modeling!
#ICLR2025 #WorldModels #AI #ML #RL
