@osclsd

Building continuous diffusion LMs 🚀 Research Intern @Apple MLR; PhD ML @UniofOxford; previously at @MSFTResearch, @EPFL, @imperialcollege

 Paris, France
olsdavis.github.io/
Joined May 2024
  • Tweets224
  • Following437
  • Followers863
28Media
Continuous Diffusion LMs went from niche to contender this year, but the recipe is shaky: why self-conditioning? How to embed tokens? 🤨 Our new work takes your one-hot flow from 4% to 50% on TinyGSM without self-conditioning, and explores the design space of CDLMs. 🔥🚀 (1/6)
13
52
6
200
15,908
Oscar Davis retweeted
Joey Bose
@bose_joey4h
🚨 New Paper Alert, 🎤 Accepted for an oral presentation at the NeurIPS 2026 STODY workshop! 📣 Introducing: Normalizing Flow Maps (NFM): Combining few-step transport from Flow maps with tractable likelihoods from Normalizing Flows. This is my first work with my first crop of new PhD students, @francescoshox and @tim_hitge, who genuinely knocked this project out of the park. 📄 Paper: arxiv.org/abs/2610.22711 📝 Blog: the-bose-lab.github.io/norma… 💻 Code: github.com/francescoshox/Nor… 🧵 1/8
2
41
4
161
6,944
Oscar Davis retweeted
Sander Dieleman
@sedielem23h
More progress on CDLMs: self-conditioning has always been a bit of a crutch, DoubleCat/SemiCat seems to be a principled way to get rid of it🎉
Oscar Davis
@osclsdOct 14
Continuous Diffusion LMs went from niche to contender this year, but the recipe is shaky: why self-conditioning? How to embed tokens? 🤨 Our new work takes your one-hot flow from 4% to 50% on TinyGSM without self-conditioning, and explores the design space of CDLMs. 🔥🚀 (1/6)
1
8
50
4,794
Oscar Davis retweeted
Rohit Dilip
@rohit_dilip_22h
oscar leave some work for the rest of us 😅 very cool idea and exciting to see more principled explorations of self conditioning!
Oscar Davis
@osclsdOct 14
Continuous Diffusion LMs went from niche to contender this year, but the recipe is shaky: why self-conditioning? How to embed tokens? 🤨 Our new work takes your one-hot flow from 4% to 50% on TinyGSM without self-conditioning, and explores the design space of CDLMs. 🔥🚀 (1/6)
1
9
849
Oscar Davis retweeted
Ruixiang Zhang
@onloglognOct 16
Introducing RSI-Inference Bench from 🍎 Apple ML Research 🌐 rsi-inference.com 💻 github.com/apple-aiml-resear… Can AI agents improve the systems that run AI? We spent $100K API tokens, 50K GPU hours to find out. We built environments and benchmark from real-world #vLLM and #SGLang tasks, covering the full lifecycle of model inference engineering: 🚀 Day-0 Model Support ⚡ Inference Optimization 🔧 Bug Fix → Unsaturated: best agent at 26.7% pass@1 → Comprehensive coverage of the inference stack → Verifiable rewards: binary and continuous Watch the video to find out more!
57
96
21
852
68,514
Oscar Davis retweeted
Luca Ambrogioni
@LucaAmbOct 15
Replying to @Guanghan__Wang
Let's get it to sub 1 ;)
1
1
4
438
Oscar Davis retweeted
Niklas Rindtorff
@Niklas_TROct 16
Introducing Langevin Flow Maps TLDR: Neural Network Potentials that learn their own stochastic pathwise solution for faster simulation. - Transferable across molecular systems. - Transition dynamics are preserved. - No trajectory data are required. Work together with @sgmccallum, supervised by @AlexanderTong7, and @JamesFosterBath. Arxiv: arxiv.org/abs/2610.05998 Data: huggingface.co/datasets/nikl… Code: github.com/sammccallum/lfm
11
86
3
494
22,149
Introducing SCALLOP 💜 Our new paper accepted at NeurIPS 2026 🐨 🚀 A scalable, efficient way to train few-step likelihood-available flow maps, enabling effective Boltzmann generation! 📽️ project page: tonyauyeung.github.io/scallo… 🔗 paper: arxiv.org/pdf/2606.29110
2
22
3
177
18,299
Oscar Davis retweeted
Brandon Amos
@brandondamosOct 16
We've released the weights for Beam!! 🤗 huggingface.co/reflection 📄 Model card: reflection.ai/beam-model-car… 🧪 Try Beam in our playground & get an API key: platform.reflection.ai/playg… 📚 AND we take open model safety seriously and have posted our safety report: reflection.ai/beam-safety-re…

reflection (Reflection)

Org profile for Reflection on Hugging Face, the AI community building the future.

huggingface.co
Brandon Amos
@brandondamosOct 5
We built our first model!! Beam is a competitive text-only 501B-A23B MoE, openly released under the Apache 2.0 license It’s been fun scaling the production RL/OPD stack for this over the past year. I’m so proud of the team! We attained exactly what we set out to do and ended up with a stable run on 10K GB300s that produced more than 100M rollouts across ~1M tasks that significantly improved the final performance of the model 👇 many more details in our announcement below :)
8
32
2
255
16,080
mlx-lm now supports image inputs with Muse! I asked Muse to explain a screenshot of the profiler output from its own decode. Muse explaining Muse. All running locally on two M5 Ultras with mlx ❤️
Ronan Collobert
@trebollocOct 15
So proud of the MLX team! Muse Spark Open, on your desk: 816B params at 36+ tokens/sec on 4 Mac Studio computers. We've been hard at work on better M5 GPU Neural Accelerator support, a smoother Thunderbolt experience (including multi-cable meshes), and kernel optimizations for MoE architectures. Oh — and MLX LM (mlx_lm.generate, mlx_lm.server) now supports the vision modality for Muse Spark Open! More on using MLX for distributed inference of frontier-scale models on Mac Studio: machinelearning.apple.com/re…
4
1
29
3,392
Oscar Davis retweeted
Konstantin Rusch
@tk_ruschOct 15
Introducing Quotient Recurrent Reasoners (QRR) 🚀 Looped reasoning models (e.g., TRM, FPRM) spend a large amount of loops on oscillatory dynamics that don't contribute to reasoning, wasting a lot of computation before converging. Turns out gauge equivariance helps tremendously by defining convergence in a quotient space instead of state-space, where these oscillations are part of the same equivalence class. Super strong performance on reasoning benchmarks: Sudoku-extreme: 97.9% 🤯 Incredibly work done by my student Sumin Park @psmiziz Check out the fascinating thread below. Also such a cool visualization! 🧵👇 @ELLISInst_Tue @MPI_IS @liquidai
Sumin Park@psmizizOct 15
Can reasoning finish while latent states keep moving? Happy to share Quotient Recurrent Reasoner (QRR)! We let symmetry-generated rotations persist while task-relevant dynamics converge faster in quotient space, reaching 97.9% on Sudoku-Extreme with 78% fewer loops than FPRM.
3
24
4
179
7,141
Oscar Davis retweeted
Luca Ambrogioni
@LucaAmbOct 15
Bitstream language diffusion is cool, but what about bistream diffusion for everything? Happy to share: Multimodal Co-Generation With Continuous Bitstream Diffusion Highly reliable and high performance training on many modalities without any modality-specific calibration!
3
18
71
3,692
We need better model acceleration. This looks great 🙌
Jaeyeon (Jay) Kim
@Jaeyeon_Kim_0Oct 15
Today, we introduce Priming Models, a new paradigm for training one-step generative models from scratch. No diffusion teacher. No flow map. No CFG. No JVP. We believe this work will challenge how we think about building one-step models. Joint work with @sitanch, @du_yilun, @parkwbin03, Bohyung Han. 🔥 1.19 FID one-step on ImageNet 256×256 while using substantially less compute, e.g., 12X less than Drifting-B, 6X less than Drifting-L 🎨 Scales to text-to-image generation (GenEval 0.82) 🤖 World modeling on RT-1 / Bridge-V2 Priming Model is built on one core message: [Reconstruction pretraining --> Distributional post-training].
1
23
1,524
Oscar Davis retweeted
Louis@louistichelmanOct 15
Can one foundation model reason across different graphs, features, relation types and prediction tasks? Introducing 𝗪𝗮𝗻𝗱𝗲𝗿🚶 Our new preprint explores 𝗴𝗿𝗮𝗽𝗵 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹𝘀 through a shared prediction objective and a random-walk interface. 🧵 1/7
1
14
3
26
2,795
Why don't Graph Transformers attend to the same token? In LLMs, most sequences attend heavily to the first token, an "attention sink". Do Graph Transformers do it too? I am happy to publish my new paper, accepted at the GDDL workshop for the @NeurIPSConf 2026 🧵 @bold_lab_ai
Attention received per node on the same molecule, for three models: GRIT with its published gated score, GRIT-dotlogit where only the logit is a dot product, and GraphGPS (GINE plus dot product). GRIT spreads attention almost uniformly over the atoms. The two dot-product models concentrate it on a single atom, shown in dark red, and half of their heads form a sink.

ALT Attention received per node on the same molecule, for three models: GRIT with its published gated score, GRIT-dotlogit where only the logit is a dot product, and GraphGPS (GINE plus dot product). GRIT spreads attention almost uniformly over the atoms. The two dot-product models concentrate it on a single atom, shown in dark red, and half of their heads form a sink.

2
3
3
183
Oscar Davis retweeted
Jerry Huang
@jrrhuangOct 14
#NeurIPS2026 Spotlight RL for diffusion and flow models often requires expensive rollouts. We show that flow maps enable simulation-free RL finetuning. Introducing Wasserstein-Tilted Flow Maps (WTF)! 🤩 The key is a transport-based alternative to KL regularization. 🧵
7
47
5
304
22,491
We’re bringing natural-language control to any scientific generator. We are very excited to introduce LLM-to-Flow Distillation (LFD), a method to adapt scientific generators to high-level natural-language discovery goals. Starting from textual goals like “design peptides with promising anticancer activity” or “design small molecules binding to the D2 dopamine receptor”, LFD distills high-level goal-related knowledge from LLMs into scientific generators through iterative DPO on LLM-generated sub-goals. 🔥 LFD does not need task-specific proxy rewards, works with any frontier LLM via API-access, and outperforms state-of-the-art reward-guided methods such as FlowGRPO and DiffusionNFT on held-out computational evaluations. (1/6)
7
39
6
163
22,380
I'm excited that continuous dLLMs are picking up again. I often heard back in 2023-24 (when discrete/masked diffusion came out) that these were just better suited for discrete data. This never made sense to me since continuous diffusion in pseudo-time has more flexibility. #dLLM
Oscar Davis
@osclsdOct 14
Continuous Diffusion LMs went from niche to contender this year, but the recipe is shaky: why self-conditioning? How to embed tokens? 🤨 Our new work takes your one-hot flow from 4% to 50% on TinyGSM without self-conditioning, and explores the design space of CDLMs. 🔥🚀 (1/6)
3
1
11
1,489
🚀 🐈
Oscar Davis
@osclsdOct 14
Continuous Diffusion LMs went from niche to contender this year, but the recipe is shaky: why self-conditioning? How to embed tokens? 🤨 Our new work takes your one-hot flow from 4% to 50% on TinyGSM without self-conditioning, and explores the design space of CDLMs. 🔥🚀 (1/6)
1
7
797
Check out this really awesome work from Oscar! really exciting to see all the progress for continuous diffusion language models
Oscar Davis
@osclsdOct 14
Continuous Diffusion LMs went from niche to contender this year, but the recipe is shaky: why self-conditioning? How to embed tokens? 🤨 Our new work takes your one-hot flow from 4% to 50% on TinyGSM without self-conditioning, and explores the design space of CDLMs. 🔥🚀 (1/6)
1
1
8
596
PS: Please do check out the amazing analyses of CDLMs that are already out there: - Why Gaussian Diffusion Fails on Discrete Data (@a__shabalin @Viacheslav91112): arxiv.org/abs/2604.02028 - CANDI (@PatrickPyn35903 @thjashin @ruqi_zhang): arxiv.org/abs/2510.22510 - FMLM* (@wognsfjq96 @wonjung_ki13109 @chandavidlee @FEijkelboom @jw9730): arxiv.org/abs/2607.00714

Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?

Diffusion models have become a standard approach for generative modeling in continuous domains, yet their application to discrete data remains challenging. We investigate why Gaussian diffusion...

arxiv.org
1
1
14
735
Tagging more usual suspects (the list is growing!): @jdeschena @sedielem @ValentinDeBort1 @ArnaudDoucet1 @baaadas @msalbergo @nmboffi @bose_joey @dario_sha @yjelid @LucaAmb @mmbronstein @JCJesseLai @JChemseddine @IAzangulov @iNikitaGushchin @pengzhangzhi1
3
129
Load more