Researcher @MSFTResearch, AI Frontiers Lab | Reasoning, Agent | Previously @Meta @MSFT_GSL @MITIBMLab @WisconsinCS
Redmond, WA
Joined March 2017
- Tweets303
- Following1.1K
- Followers1.7K
- Likes2.3K
Pinned Tweet
💻Tired of running so many slow, expensive benchmark evals across every checkpoint?
Try ✨BenchPress✨ at microsoft.github.io/benchpre…: provide a few benchmark scores, then get predictions for the remaining ~100 benchmarks, with trust probabilities and calibrated 90% prediction intervals.
How does this work? In his original post (xitter.cf/DimitrisPapail/status/…), @DimitrisPapail first tried the idea as a fun question: collect model-by-benchmark scores into a matrix, find its low-rank structure, and use matrix completion to predict missing benchmark scores from a few observed ones.
We expanded this into a full system: a fully audited 84-model x 133-benchmark score matrix, an optimized matrix-completion predictor, and a reliability layer for trust probabilities and 90% prediction intervals.
Beyond predicting missing scores, we also suggest practical seed benchmark sets. The five-probe set {GPQA-D, HLE, Codeforces, MMLU-Pro, ARC-AGI-1} recovers the rest of a model's public score profile with a MedAE of 3.93 points. A lower-cost set {GPQA-D, MMLU-Pro, Aider Polyglot, MATH-500, AIME 2026} reaches 4.55 points.
See more details below 🧵1/7
This work is with @DimitrisPapail at AI Frontiers, a boutique research lab inside @MSFTResearch.
Yuchen Zeng retweeted
Today, we introduce Priming Models, a new paradigm for training one-step generative models from scratch.
No diffusion teacher. No flow map. No CFG. No JVP.
We believe this work will challenge how we think about building one-step models.
Joint work with @sitanch, @du_yilun, @parkwbin03, Bohyung Han.
🔥 1.19 FID one-step on ImageNet 256×256 while using substantially less compute, e.g., 12X less than Drifting-B, 6X less than Drifting-L
🎨 Scales to text-to-image generation (GenEval 0.82)
🤖 World modeling on RT-1 / Bridge-V2
Priming Model is built on one core message: [Reconstruction pretraining --> Distributional post-training].
Yuchen Zeng retweeted
We can scale up our text-to-protein work thanks to generous support from Google.org! blog.google/company-news/out…
With @UWproteindesign, we're working on sustainability. Hopefully one day we'll be able to simply tell the model, "design an enzyme to degrade plastic"🪄
Super excited to share our year-long work on designing proteins directly from natural language, led by @jleechung in collaboration with @UWproteindesign @allen_ai!
Current protein design models are constrained in what they can condition on. As an analogy, think of an image generation model that only conditions on discrete class labels like "dog" or "cat", or "outdoors" or "indoors"; in contrast, a modern text-to-image model can be much more expressively controlled.
We do the same for protein design here, and the resulting model natively turns natural instructions into corresponding protein structures. This makes it much more effective at designing proteins with particular form/function, and it works especially well for agent-driven protein design!
Yuchen Zeng retweeted
Super excited to share our year-long work on designing proteins directly from natural language, led by @jleechung in collaboration with @UWproteindesign @allen_ai!
Current protein design models are constrained in what they can condition on. As an analogy, think of an image generation model that only conditions on discrete class labels like "dog" or "cat", or "outdoors" or "indoors"; in contrast, a modern text-to-image model can be much more expressively controlled.
We do the same for protein design here, and the resulting model natively turns natural instructions into corresponding protein structures. This makes it much more effective at designing proteins with particular form/function, and it works especially well for agent-driven protein design!
Yuchen Zeng retweeted
Idea Basins is my new favorite agent abstraction :)
Fork-and-Flush: help your agents escape Idea Basins!!
I love thinking of AutoResearch as Idea Descent: a metric of interest goes down, as a function of ideas/experiments, where the agents get pulled towards one idea or another. But we've seen very often an agent develop this weird tunnel vision, and honing in just one type of "idea", kinda like GD stuck in a local minimum.
We studied this and seems we can fix it with Fork-and-Flush, a super simple and kinda cute idea. Every so often, fork the agent into T copies in the same workspace but empty the context, let them run for a bit, and continue from the best one.
If you like the GD analogy, it's basically basin hopping en.wikipedia.org/wiki/Basin-….
BTW we found a silly way of also plotting "idea basins": an LLM rewrites every experiment as a spec and answers a few thousand binary queries "is Experiment A closer to B or to C?". Then we embed this NXN matrix into 2D and indeed see in many cases that each run saturates into a small region and keeps polishing it.
We call these idea basins!
At equal compute Fork and Flush beats single runs and best-of-N on 12/13 long-horizon tasks (some 4+ days), and in my own research projects it's been working really well!
BTW we only use it at test time, but it could also be part of training or RL.
Led by @jackcai1206 @christos_ziakas during Jack's internships with us, along with @vkontonis @ShivamGarg91462 @Tea_Pearce Sid Sen and Akshay Krishnamurthy.
Yuchen Zeng retweeted
Grok 4.7 ran out of API credit and was cut off 41 hours early. Nearly 70% of its GPU budget was still unused.
And somehow, the model it trained still made the top two!! @SpaceXSI
Stage 1 of RSIArena is over. Thank you, our collaborators @huggingface @scale_AI @NotreDame @UW and @Stanford, we did awesome! 🧡
Stage 2 training is now LIVE—and this time, the models have human feedback to work with.
Round 1 was a six-day competition: 10 models (AI researchers), the same @NVIDIAAI Nemotron base model, and 1,000 GPU-hours each to build something better.
They chose their data, wrote training code, ran experiments, and submitted their models.
At @COLM_conf and online, people then cast 1,685 votes in blind comparisons. @OpenAI GPT-6 Astra model finished first; the models trained by grok 4.7 and @Zai_org GLM-5.3 rounded out the top three.
Now those judgments go back into the experiment.
In Stage 2, each model resumes from its own Nemotron checkpoint with feedback from its model’s arena battles, another 500 GPU-hours, and $150 in API credit. Across both rounds, that’s a total budget of 1,500 GPU-hours and $450 in API credit per model.
The agents decide how to interpret the feedback, which experiments to run, and what to change in training. Their next models will show whether those decisions actually helped.
Stage 2 runs October 10–13. Watch the training live: 👇
Yuchen Zeng retweeted
As autoresearch agents run longer, do they keep exploring new approaches, or mostly refine the ones they found early?
Turns out, runs settle into different "idea basins" and stay there.
We introduce fork-and-flush, a simple way out 🧵👇
Three days at CoLM. Met so many smart people, and exchanging ideas feels much easier now. At my first conference in NeurIPS 2019, everyone seemed to work on very different things. Now most conversations have shared ground (yes, it’s CoLM 😄). Love the Bay Area vibe. Want to move here ASAP...
Yuchen Zeng retweeted
Only one way through of this. Keep creating interesting artifacts and share it with the world. Doesn’t matter if it’s called papers tweets services repos models or anything in between. Offer your findings in abundance and minimize friction to reproduce and build on.
Yuchen Zeng retweeted
I wrote a piece for Fast Company about games, robots, and what we're learning from working on both.
Some thoughts on how games can help us build AI for the physical world. Please check it out!
fastcompany.com/91616927/wha…
Yuchen Zeng retweeted
Can't make it to #COLM2026 in person, here's our paper in 23 seconds 🎬
Diffusion LM MoEs inherited token-choice routing from autoregressive LLMs. We flip it: let the experts choose.
"Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Models" — full rundown below 👇
—
1. The problem
Today's DLM MoEs use token-choice (TC) routing: each token picks its expert. Popular experts overflow, the rest sit half-empty, and every GPU waits for the busiest one. The usual fix is an auxiliary balancing loss, which spends gradient signal on bookkeeping.
2. Flip the direction
With expert-choice (EC), each expert picks its top-k tokens. Every expert processes exactly k tokens, so load is balanced by construction. No auxiliary loss.
Per-GPU memory std: 3.6 GB (TC) → 0.0 GB (EC).
3. Balance turns into speed
EC reaches loss 3.75 in 10.6h. TC needs ~20h.
≈2× faster wall-clock convergence, with 1.5–2.1× higher throughput per GPU.
4. The part I'm most excited about: compute becomes a policy
A DLM denoises the same sequence over many steps, and those steps are not equally worth computing on. EC capacity is an external knob, so we can set it per denoising step.
Giving more capacity to low-mask-ratio steps wins under matched FLOPs (OpenWebText PPL 37.1 → 36.5, best of 6 schedules). The reason: tokens in low-mask-ratio contexts learn an order of magnitude more efficiently, so that's where extra compute pays off most.
5. A cheap upgrade for existing models
Already have a pretrained TC DLM? Replace only the router with EC and finetune. You get faster convergence and better downstream accuracy, no retraining from scratch.
6. Bigger picture
DLMs shouldn't just inherit their architecture from autoregressive models. Iterative denoising opens up a design space AR models simply don't have, and MoE routing is only one piece of it. I remain optimistic.
Since I can't be there to present it, questions are very welcome here or in DMs 🙏
My co-authors will be at our poster, come say hi 👋
🪧 Poster Session 6, #90
🕓 Thu Oct 8, 4:40–6:30pm PDT
📍 Franciscan B, Hilton San Francisco Union Square
📄 Paper: arxiv.org/abs/2604.01622
💻 Code: github.com/zhangshuibai/EC-D…
Joint work with @HermesCui, @zhihanyang_, @pengzhangzhi1, Yanxin Zhang, @haoyue_bai, Caspian Zhuang, Zack Jia, Yang Zhou, Guanhua Chen and Ming Liu
Yuchen Zeng retweeted
What happens when you give a computer-use agent just a terminal and let it build its own tools, organically?
Meet CUAWright, with **only** one tool: run_command.
📄 Paper: arxiv.org/abs/2610.04116
🌐 Project: microsoft.github.io/CUAWrigh…
💻 Code: github.com/microsoft/CUAWrig…
Video credit to @JangLawrenceK!
Yuchen Zeng retweeted
Congrats!!!
On Wisconsin 👐
A @NobelPrize for a Badger 🏅
Congratulations to Francis Halzen, named the 2026 Nobel laureate in physics for his groundbreaking work with @uw_icecube to advance our understanding of neutrinos and the universe’s most energetic events.
news.wisc.edu/university-of-…
Yuchen Zeng retweeted
i support even a 5-10 limit/author a year. It's plausible this will lead to better papers than what we see at ICML and Neurips :)
arXiv has updated our policy on rate limiting for all submitters.
This update was made to fairly distribute moderator time & support the arXiv community of staff, volunteers, readers & authors.
Please read our announcement to learn more: blog.arxiv.org/2026/10/01/up…
ALT arXiv logo in white over a whiteboard with math symbols.
Yuchen Zeng retweeted
Why can some people manage multiple coding agents while others can only manage one at a time? Led by @taolongg we worked on understanding this and piloting solutions to bridge the gap arxiv.org/abs/2609.33113
Maybe a bit late, but I wanted to comment on this: I think the AI boom made people forget what a PhD was meant for. Now it's going back to its original purpose: not short-term profit, but long-term research that might not have a practical use for another 10 or 20 years -> Backprop existed decades before the deep learning boom.
A PhD isn't for everyone, and that's okay. Industry may suit you better if you want to build something useful now, or academia if you believe in a problem industry doesn't care about yet. The AI boom made a PhD look like preparation for an industry job, but that was never its main purpose.
Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI.
Perhaps counterintuitively, I think it's a great time to do so.
I wrote up some thoughts on this here: web.mit.edu/phillipi/www/wri…
One more point, what I mentioned above is the most ideal case, but a PhD can also open doors for normal people, too. For me, it gave me time to figure out what I wanted to do and a chance to meet great people who could guide and help me. I feel very lucky to have my phd advisor as a mentor in research and in life.
Yuchen Zeng retweeted
How much of post-training research can frontier models actually do on their own 🤔?
We’re testing this with LIVE RSIArena, in collaboration with @Stanford, @NotreDame, @UW, and @scale_AI.
✨8 research agents. ✨Same 30B base model. ✨144 hours on a shared cluster of 64 RTX PRO 6000 Blackwell GPUs. 🎉 Model training model!
Each agent gets 1,000 GPU-hours to choose its data, write training code, and run experiments. We freeze and independently evaluate the models they submit.
Come watch the agents do research—and have their own “why did this work yesterday?” moments.
If you work on post-training, agents, or evaluation, we’d love your feedback!
RSIArena, by @bake_ai_hq 😆
Watch: 👇
We’ll also host a livestream event at the @COLM_conf. Come chat with us at booth #107!
Yuchen Zeng retweeted
🧵 It's finally here! The full technical report on PUBG Ally is finally on Arxiv: arxiv.org/abs/2609.29837
Sharing a few highlights in the thread!
