Researcher at Microsoft working in computer vision and machine learning

Seattle, WA, USA
Joined April 2009
My team at Microsoft Research, working in multimodal, AI is hiring! Please apply if you are interested in working at the cutting edge of multimodal generative AI. jobs.careers.microsoft.com/g…
8
25
2,701
🚀Stoked to announce the release of Rho, a family of open-weights 5B VLAs underlying @MSFTResearch work on Rho-alpha VLA+ (microsoft.com/en-us/research…) 🤖 Project page: microsoft.github.io/rhobotic… 📰 Tech report: arxiv.org/abs/2609.38164 ⌨️ Code: github.com/microsoft/rhoboti… 🤗 Models and data: huggingface.co/collections/m… Rho is a step in the direction of physical AI adept at adapting to new tasks and environments, motivated by a simple observation: offline supervised finetuning (SFT), the physical AI task adaptation stage "owned" by robot operators, is too data-hungry, because it tries to do too much besides learning the task per se: (1) mastering the control of the robot doing the task, and (2) exposing the model to many target task and environment variations. To address hurdle (1), we publish Rho checkpoints midtrained for @i2rt_robotics YAM Box, @Universal_Robot AI Trainer, and @FRANKAROBOTICS FR3 Duo, representative dual-arm robot setups across research labs and the industry. Rho itself is built on @Microsoft's own special Phi-series backbone and pretrained on robot data in the end-effector space, including UMI-style and @NVIDIARobotics Isaac Sim demonstrations, to facilitate its midtraining for other robots. Embodiment-midtrained Rho outperforms the strongest existing open-weights VLAs on physical robots and in simulation. And then there is hurdle (2), complicated by the tendency of offline data collection to oversample the same task situations while under-sampling those where the robot's policy actually needs help. We mitigate this by pairing with Rho with online adaptation methods, particularly FlowDAgger (microsoft.github.io/FlowDAgg…, a 🏅Spotlight🏅 at #CoRL2026), which kick in after offline SFT and use online corrections for model improvement, resulting in much more data-efficient task adaptation end-to-end. VAMs and WAMs, new zero-shot capabilities of physical AI, tactile sensing data, tactile-sensing humanoid hands, GPT-6 -- all of these hold the potential for making Rho and its successors so much more effective than we imagined just a year ago. Come to the 🤖8th Robot Learning Workshop at #NeurIPS2026 🤖 (robot-learning.ml/2026/) to geek out about these topics, and try Rho on your robots! 🙌 Congratulations to the team that has made Rho happen -- Simran Bagaria, Daphne Chen, Dean Fortier, Jianlong Fu, Michael Harrison, @TessHelleb88231, @neelsj, Dalton Moore, Galen Mullins, Michael Murray, Eduardo Salinas, and Reuben Tan! A huge shout-out to TPMs @Vivan_Amin and Ade Famoti, to @Universal_Robot for partnering with us, and to @ashleyllorens for his sponsorship and guidance in this work. Coming together from across @MSFTResearch's multiple organizations and locations in Redmond, Phoenix, New York, and Beijing, this team built more than a VLA family; it has laid a major foundation stone for @Microsoft's physical AI efforts. Truly impressive and inspiring. 🚀 #PhysicalAI #Robotics #MicrosoftResearch
6
12
6
108
10,213
TLDR: we integrate many computational and statistical improvements for muon-style updates here: github.com/microsoft/dion . Comp: Gram Newton Schulz, Row Selection, Megabatching, Cudagraph Capture, Symmetric Kernels Stat: Joint learning rate+fraction, row norm, fused rounding
Newton-Schulz is what makes Muon work, but it's also a massive pain compared to Adam. *Dion3* makes it ~6x faster and more parallelizable too. Combines our work on row-filtered gradients, Gram-NS alg, and GPU kernels in one package. Try pip install dion! arxiv.org/abs/2608.11612
1
25
152
11,990
MagenticLite's models are now fully open source. MagenticBrain and Fara 1.5, previously available on Microsoft Foundry, are live on Hugging Face with open weights. The app, the harness, and every model in the stack are now open.
4
22
7
89
19,482
Neel Joshi retweeted
Meet Phi‑4‑reasoning‑vision‑15B in Foundry: high‑fidelity vision reasoning + high‑quality perception for real‑world dev workflows. Learn more: msft.it/6011QgKvl
9
34
2
159
17,069
So proud of this work from my team and excited to share it with the world!
Vision-language models improve multimodal systems, but can make them slower, costlier, and harder to deploy. Learn how Phi-4-reasoning-vision-15B, a compact and fast multimodal reasoning model, blends strengths of different methods while reducing their limits: msft.it/6014Q5X0u
4
197
At NeurIPS, @KwangjunA is giving a semi-tutorial on distributed orthonormal updates Tuesday 4pm-5pm nips.cc/virtual/2025/loc/san… at Upper Level Ballroom 6CDEF . These updates are worthl +10% to +100% more data, and the new Dion2 approach makes them tractable at any scale.
1
4
48
5,509
📌 You can now find all the evaluation logs (and reasoning traces for common benchmarks!) from our inference-time scaling report and the Phi-4 reasoning report at  huggingface.co/datasets/micr…. The evaluation code can be found at Eureka ML Insights: github.com/microsoft/eureka-….
1
12
1
51
4,906
One of the secret weapons we had in doing the phi-4-resoning report is Eureka: github.com/microsoft/eureka-… Eureka and our eval team have been doing an amazing job with adding new benchmarks to doing deeper analysis of results beyond single-score statistics.
BTW when you test AIME or HMMT, or anything that has ~30 questions, PLEASE DO 50-100 runs, and report error bars!! The pass at 1 for 5 runs is super noisy... Look at the variance even for 50 runs
1
4
15
2,453
We’ve been cooking... a new open weights 14B Phi-4 reasoning model, SFT’d on ~1.4M carefully curated reasoning demonstrations from o3-mini and RL’d for a tiny bit. This model is a little beast.
35
230
47
1,427
440,819
I am thrilled to share our newest Phi models. This time we went all in on post-training to produce Phi-4-reasoning (SFT only) and Phi-4-reasoning-plus (SFT + a touch of RL) — both 14B models that pack a punch in a small size across reasoning and general purpose benchmarks🧵
3
20
3
73
23,485
A new post: Headroom for AI Development hunch.net/?p=13763046 . It's quite interesting to compare biological and silicon capabilities.
5
15
1,689
Announcing AutoGen 0.4, fully reimagined library for building advanced agentic AI systems, developed to improve code quality and robustness. Its asynchronous, event-driven architecture is designed to support dynamic, scalable workflows. Learn more: msft.it/6012ohgli
17
159
36
674
108,511
Excited to announce the release of Eureka, an open-source framework for evaluating and understanding large language and multimodal models! I’m really proud of the team. This is important work and crucial to have it be public and open.
How can we rigorously evaluate and understand state-of-the-art progress in AI? Eureka is an open-source framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings. Learn more about the extended findings. msft.it/6014mpZ8O
6
214
I am delighted to announce our HoloAssist challenges at the EgoVis (egovis.github.io/) #CVPR2024 workshop. HoloAssist is a large-scale egocentric human interaction dataset, where two people collaboratively complete physical manipulation tasks using HoloLens2.
7
2
6
516
Neel Joshi retweeted
Join us in shaping the future of AI! AI Frontiers is a lab inside Microsoft Research with a mission to unlock new AI capabilities and solve real-world problems. We are hiring researchers and engineers in our teams based in Redmond and NYC. Apply here: microsoft.com/en-us/research…
5
51
9
205
121,439
📣Spring hiring: AI Frontiers at Microsoft Research is looking for researchers passionate about in-depth understanding and rigorous evaluation of foundation models and multi-agent systems: jobs.careers.microsoft.com/g…. Learn more about the group here: microsoft.com/en-us/research…. This is an excellent opportunity to work with our AI&ML crowd: @ecekamar @neelsj @hmd_palangi @VibhavVineet Safoora Yousefi @vidhisha_b @SaleemaAmershi @julia_kiseleva @adamfourney @bansalg_ and many more. Examples of in-focus research topics include: mechanistic interpretability, training and instruction data understanding, measurement methodologies and sanity checks for evaluation practices, evaluation of multi-turn and multi-agent interactions.
23
1
73
14,334
HoloAssist is a new multimodal dataset consisting of 166 hours of interactive task executions with 222 participants. Discover how it offers invaluable data to advance the capabilities of next-gen AI copilots for real-world tasks: msft.it/60139Uv4d
1
9
2
39
26,123