Silver Professor @ NYU Courant & CDS AmiLabs Research in Machine Learning & AI, ex Director @ MetaFAIR, past in Quantum Comp. & Finance Posts my own.

Joined April 2024
Julia Kempe retweeted
Many thanks to our speakers for the uplifting discussions on the value and potential of small-scale research and models, and to everyone for a packed house! See you next year! 🌱
1
3
1
22
4,479
Julia Kempe retweeted
Replying to @KrzakalaF
Au contraire. C'est une nouvelle ère qui s'ouvre pour les mathématiques. Une ère où la démonstration formelle est largement automatisée et où l'accent sera reporté sur le développement de nouveaux concepts, nouvelles abstractions, nouvelles définitions, et nouvelles conjectures. L'invention du bateau a réduit l'importance de la nage, mais a permis la découverte de nouvelles terres.
145
641
142
4,508
244,931
Julia Kempe retweeted
This is outrageous framing. Quite frankly, given the current tensions around AI and mathematics, this kind of announcement does not help. After the recent Buckmaster/OpenAI dispute over scientific credit and provenance, the last thing we need is more promotional PR language that blurs the history of a research problem. Meta presents its work by asking: “Can AI contribute when a problem is genuinely open and without an existing solution path?” Seriously?! For the ellipsoid-fitting example, this is misleading and insulting. There was an established research programme, substantial rigorous progress, and by the time of their announcement, publicly available proofs of the exact threshold. Let us be specific: - In 2023, Antoine Maillard and Dmitriy Kunisky gave a statistical-physics derivation of the conjectured transition at n/d² = 1/4, using the replica method. Their analysis also predicted the geometry of the solutions and suggested directions for rigorous proofs. This was not a proof, but it was a detailed and successful theoretical picture, not an absence of a path. arxiv.org/abs/2310.01169 - Afonso Bandeira and Antoine Maillard then rigorously established the sharp threshold for approximate fitting, using methods inspired by the statistical physics of disordered systems, using actually spin glass mathematical technics. Their preprint appeared in 2023 and was published in the Electronic Journal of Probability in 2025. Their Gaussian-equivalence framework provided a foundation for further work (including many of mine!). At this point, the "exact" fitting and removing the spectral restriction were the outstanding gaps, but these are technical problems, it was not about not discovering the threshold from scratch. arxiv.org/abs/2310.05787 - On 10 August 2026, Theodor Misiakiewicz and Garrett Wen posted a proof of the exact Gaussian threshold, explicitly closing those two gaps. arxiv.org/abs/2608.10184 - On 12 August, Sofia de la Cerda, Aaron Potechin, Madhur Tulsiani and Jeff Xu posted another proof, establishing both sides of the transition up to a vanishing multiplicative factor. arxiv.org/abs/2608.12415 - On 27 August, Frederic Koehler and Youngtak Sohn posted a broader universality theorem that includes the Gaussian threshold. arxiv.org/abs/2608.27372 These were not researchers merely “on the verge” of solving the problem. THE PROOFS WERE ALREADY PUBLIC. Meta's announcement comes in October, and Section 3 of their paper begins: “We use the groundbreaking approximate-fitting theorem of Bandeira and Maillard…” Dear META: Those are your words. This is not simply a reference in the bibliography: your feasibility proof explicitly starts from their theorem. How is that consistent with advertising this example through the suggestion that there was “no existing solution path”? Yes, your social post mentions independent solutions, and your paper and blog identify the August results and describe your work as independently developed. That does not answer the objection to the framing. A generic acknowledgement of parallel work does not make clear that the exact threshold already had publicly available proofs before this announcement. An independently developed AI-assisted proof, released after other proofs, should be presented as exactly that. Readers should not have to reconstruct the chronology to understand what is, and is not, new. A new proof can contain valuable ideas. But a new proof and a newly solved problem are not the same claim. Nor does a problem being open when a project begins mean it is still open when the marketing campaign launches. You say the aim is to “empower researchers”? Nothing more “empowering” than seeing years of collective research turned into the backdrop for a cheap marketing stunt. Correct the announcement! Give the scientific history the same prominence as the product. Give Antoine, Afonso, Theo and all the other researchers involved the public credit they deserve.
AI at Meta
@AIatMetaOct 2
Replying to @AIatMeta
1️⃣ Probability: Mathematicians worked with Muse Spark to answer a question about fitting random points onto the surface of a stretched sphere. For the setting studied, they proved a sharp cutoff between when an exact fit is likely and when it is unlikely. Read the paper: ai.meta.com/research/publica…
25
122
14
701
81,501
Julia Kempe retweeted
We are having our first @SimonsFdn yearly meeting on the Physics of Learning and Neural Computation Sept 24th-25th and all are welcome to join on this zoom: simonsfoundation.zoom.us/j/9… We have an excellent set of speakers. Here is the agenda: simonsfoundation.org/event/s… Join our webinar starting Sept 24th 9:30am ET to learn how the structure of data, architectures, and learning dynamics conspire to empower AI to learn, scale, compute, imagine, reason and collaborate! See physicsoflearning.org/ for more about our collaboration.
2
58
11
286
42,369
An update on Lean (auto)-formalization. Speaking about my own research workflow, in 2026, we have gone from: March: Lean formalization is not possible (for my papers). May: Lean formalization is possible... but way too costly in time and tokens to be practical most of the time. July: Lean formalization is now practical. I can probably formalize most of my papers now before they hit arxiv. August: Lean formalization is now essential-- it increases efficiency of my workflow dramatically. That is, the papers are getting written *informally* and finished much faster because of the Lean formalization.
33
93
20
869
146,680
Julia Kempe retweeted
Daniel Litt
@littmathAug 11
I'm currently returning to Toronto from a summit on the future of mathematics, at OpenAI. @SebastienBubeck asked me to talk a bit about the future we'd all like to avoid, where humans are mathematically disempowered. @Jacob_Tsimerman advised us to try to prioritize detail over correctness, and I have no doubt that I succeeded in deprioritizing correctness. I tried to find a title that wasn't too bombastic:
64
230
44
1,410
334,745
Julia Kempe retweeted
I'll be at #ICML2026 to present our *spotlight* work! TLDR: LLMs can learn to self-generate curricula for problems they can't yet solve, using self-play with meta-RL. Please reach out to chat about self-improving agents, synthetic data & environments, curriculum learning, or anything else! We've updated the paper with some fun additions ⬇️
Can a model learn to break its own reasoning plateau? In our new paper, we show that LLMs can be taught with meta-RL to generate their own "stepping stones" that kickstart learning on hard math problems (0/128 success rate) where direct RL fails. Paper 📝: arxiv.org/abs/2601.18778 Blog post 🌐: ssundaram21.github.io/soar/ (1/n)
7
20
2
149
19,165
Julia Kempe retweeted
Math is starting to fall — so what's next? 🎙️ New episode of The Information Bottleneck is out! We've all seen the recent wave of Erdős problems being solved by frontier models, and the question now is what it actually means for the future of mathematics, and for AI research more broadly. We sit down with @KempeLab - Professor at NYU's Center for Data Science and researcher at Meta FAIR's Foundations of Reasoning team, to dig into exactly that. Julia makes the case that math is the next Go. With formal verification and LLM agents that can propose, formalize, and check proofs at scale, a new industry of automated mathematical discovery is closer than most mathematicians believe. We also get into: → Why physics is harder than math → Model collapse, synthetic data, and what's left to squeeze from the internet → Scaling limits, energy costs, and where academia still has the edge → How to advise PhD students when Claude can already do their first-year work → AI safety, agent security, and the Wild West of deployed agents → Why the Renaissance researcher is finally back One of our favorite conversations yet. Listen now 👇
🤖 Made with AI
6
4
34
4,342
The 2nd Sci4DL (@scifordl) workshop at #ICLR2026 in Rio drew a packed room. Organized by CDS PhD alumni @ZKadkhodaie & @LotfiSanae, CDS Instructor @FlorentinGuth, and CDS-associated Prof. Eero Simoncelli, with CDS Silver Prof. @KempeLab as a speaker. scienceofdlworkshop.github.i…
7
1
6
1,961
CDS Silver Prof @KempeLab working with @arnal_charles, @TacoCohen, Vivien Cabannes, and Remi Munos studied how LLMs can train more efficiently by reusing past experience instead of constantly generating new data. Accepted to the ICML '26 conference. arxiv.org/abs/2604.08706
2
6
1,536
"Insights into LLMs reasoning" by Julia Kempe @KempeLab
1
5
31
4,745
Julia Kempe retweeted
Excited to be at #ICLR2026 in Rio this week! Presenting “Soft Tokens, Hard Truths” Saturday 10.30am (Pavilion 3, P3-#1020). Feel free to DM me to chat about self-improvement, reasoning, code gen. I’m also on the job market for industry research positions.
🔥New preprint: Soft Tokens, Hard Truths Introduces the first scalable continuous-token RL method for LLMs - no reference CoTs needed; scales to hundreds of thought tokens. Best to train soft, infer hard! Pass@1 parity ⚖️, Pass@32 gains 📈& better robustness 🛡️ vs. hard CoT 1/🧵
6
22
3,619
Julia Kempe retweeted
love this replay buffer paper from Meta: arxiv.org/abs/2604.08706 "methods like PPO or GRPO typically operate as on-policy as possible, meaning rollouts are generated, used for a single gradient update, and immediately discarded." this is crazy and we shouldn't do this!
11
50
5
478
57,068
Efficient RL Training for LLMs with Experience Replay "Empirically, we show that a well-designed replay buffer can drastically reduce inference compute without degrading – and in some cases even improving – final model performance, while preserving policy entropy."
5
52
1
346
22,272
Julia Kempe retweeted
(1/9) Experience replay can cut LLM RL training compute by up to ~40% (without hurting final accuracy—and sometimes improving it). Paper: arxiv.org/abs/2604.08706
3
40
2
197
17,319
Julia Kempe retweeted
Experience replay of LLM RL. Hyperparameters for replay buffers, buffer sizes N, number of new rollouts R, minibatch size B depend on compute imbalance of rollout generation and training, and task-specific statistics. It could be hard to optimize this well.
4
8
2
76
13,249
Replying to @KempeLab
@KempeLab and I just vibe-coded a Lean 4 formalization of elliptic De Giorgi–Nash–Moser theory. This is a cornerstone of modern elliptic PDE: local boundedness, weak Harnack, Harnack, and Hölder regularity for weak solutions with merely bounded measurable coefficients.
5
41
5
146
19,164
AI is becoming incredibly powerful at solving complex math, but it still requires strict human supervision. CDS Silver Prof Julia Kempe (@KempeLab) tested top models on unpublished math questions, exploring their capabilities and limits in a new paper. nyudatascience.medium.com/te…
1
7
17
2,003