Meta AR Research

Redmond, WA
Joined June 2009
a tiny AI 0.5B just turned a normal phone video into a fully labeled 3D room run phone. SpatialLM that understands walls, doors, furniture, and everything layout. it trained as a standard multimodal LLM. Most vision models see pixels, SpatialLM sees space. Open models : github.com/manycore-research…
27
204
8
2,071
67,545
Check out CoMind, an ECCV 2026 dataset exploring how people collaborate, from two participants; perspectives! Captured with Project Aria glasses, CoMind combines synchronized egocentric video, gaze, audio, and 3D scans to study shared attention, anticipate interactions, and predict handovers. A rich resource for building AI that understands and supports human collaboration. Fantastic work by the CoMind team! 📷 @xiwang1212
🚀 New from our team: CoMind (comind.ethz.ch/) - a dataset for AI that understands not just what I am doing, but what WE are doing together. Two people. Two egocentric views. Gaze. Speech. 3D scenes. Social cues. Accepted at #ECCV2026 🧵👇
2
6
729
MingfeiYan retweeted
I know teleoperation is old these days but it feels magical when you could teleoperate a humanoid just by wearing a pair of classes (bonus: the robot also sees from glasses too!) Of course we don't stop here --- we are working on a system to train robots with everyday human experience, without any teleop data. More to come! Kudos to the team Chuye Zhang @DanielZhenyang @ChuizhengKong
18
13
5
161
12,697
MingfeiYan retweeted
sEMG×一人称カメラでロボット学習 🔹Wetour Roboticsが8ch sEMGリストバンド+カメラを統合 🔹視覚で手の動き、sEMGで筋活動を取得 🔹視覚単体では手を21.8%のフレームで検出できず、最大4.32秒欠落 1秒分のsEMG処理は50.4ms。情報元はリプ欄。
5
46
411
26,063
MingfeiYan retweeted
I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: darioamodei.com/post/we-must…
5,145
7,161
4,384
67,431
17,173,311
MingfeiYan retweeted
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: darioamodei.com/post/we-must…
10,643
16,374
11,359
87,774
76,897,238
Heading to #ECCV2026? Join us for the Fifth Hands-On Egocentric Research Tutorial — a half-day, hands-on deep dive into egocentric data with Project Aria. 📍Palissad West, Floor 0 (must take the stairs) Tue, Sept 8 9AM–12:30PM Can’t make it to Sweden? We’ll post the whole session on YouTube after the conference. projectaria.com/project_aria…
1
2
3
813
MingfeiYan retweeted
FROM 0-5 TO 7-5 IN THE THIRD 🤯 ZHENG QINWEN WITH THE COMEBACK OF THE YEAR!!! 🇨🇳
188
821
356
6,284
254,718
MingfeiYan retweeted
Voice mode now runs on Claude's more capable models and reaches the tools you've connected mid-conversation. Talk through the hard problems out loud, in many more languages.
717
782
433
11,546
1,767,960
New Anthropic research: Project Fetch. We asked two teams of Anthropic researchers to program a robot dog. Neither team had any robotics expertise—but we let only one team use Claude. How did they do?
81
210
80
1,876
566,290
Introducing Depth Estimation with NVIDIA’s FoundationStereo—a new open tutorial and codebase for metric depth sensing using Project Aria Gen 2 devices. ⚡️ Real-time ready (20 FPS) 🤖 Zero-shot generalization ⚡️Stereo image rectification and disparity estimation 🔓 Open Source Check out the code and tutorial: github.com/facebookresearch/… 🔌@the_shweenz
7
32
3
223
35,871
Meta and Project Aria are headed to Denver for CVPR 2026. ✈️ The Project Aria team will be onsite, highlighting the latest from our program on the future of Contextual AI and Robotics research! 🤖 📍 Denver, Colorado 📅 June 3–7, 2026 🤝Booth 201 We'll see you tomorrow for presentations on the cutting edge of computer vision research across CVPR's workshops and tutorials. See our snapshot agenda below for details!
2
2
27
2,045
MingfeiYan retweeted
Introducing Claude Opus 4.8: it builds on Opus 4.7 with sharper judgment, more honesty about its own progress, and the ability to work independently for longer than its predecessors. Available today at the same price.
3,591
8,328
5,300
66,226
15,510,512
How do we train AI to be a real-world expert? 🛠️🧠 Introducing Ego-EXTRA, a new video-language dataset for Expert-Trainee assistance. Researchers at the University of Catania captured unscripted interactions where experts guide trainees through complex tasks in real-time using Aria Glasses. 👓 50 hours of data 💬 15k+ high-quality VQA sets ✅ RGB + Eye Gaze tracking ✅ Hand keypoints & SLAM ✅ IMU, Magnetometer, & Audio ✅ Natural expert-trainee dialogue 🚲 4 scenarios: Bike Workshop, Kitchen, Bakery, & Assembly Explore the benchmark: fpv-iplab.github.io/Ego-EXTR… Read the paper: arxiv.org/abs/2512.13238 👉@FrancRagusa, @GMFarinella
2
15
96
8,047
MingfeiYan retweeted
Introducing Claude Design by Anthropic Labs: make prototypes, slides, and one-pagers by talking to Claude. Powered by Claude Opus 4.7, our most capable vision model. Available in research preview on the Pro, Max, Team, and Enterprise plans, rolling out throughout the day.
4,055
14,698
10,168
146,183
64,570,785
MingfeiYan retweeted
Introducing Project Glasswing: an urgent initiative to help secure the world’s most critical software. It’s powered by our newest frontier model, Claude Mythos Preview, which can find software vulnerabilities better than all but the most skilled humans. anthropic.com/glasswing
1,935
6,533
4,417
43,437
31,958,769
Today, we’re releasing a feature that allows Claude to control your computer: Mouse, keyboard, and screen, giving it the ability to use any app. I believe this is especially useful if used with Dispatch, which allows you to remotely control Claude on your computer while you’re away.
878
1,433
897
18,421
4,796,225
MingfeiYan retweeted
Introducing EgoVerse: an ecosystem for robot learning from egocentric human data. Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling 1300+ hrs, 240 scenes, 2000+ tasks, and growing Dataset design, findings, and ecosystem 🧵
38
160
45
908
311,332
MingfeiYan retweeted
#NVIDIA just released a whole ecosystem for human(oid) motion and robot learning from human data. 🚀🦾 Data, as we all know, is the key to scaling AI models. To accelerate the field of Embodied AI, we have open-sourced a full stack of models and tools to capture, generate, retarget, and simulate human(oid) motion data at scale, along with a massive high-quality dataset and a standard human skeletal representation, SOMA, to make them all seamlessly communicate with each other. The entire suite is available under the Apache 2.0 license. 1️⃣ SOMA: A universal interface to unify all parametric human body models (SOMA-shape, SMPL, MHR, etc.) into a standard skeletal representation, eliminating the need for custom adapters or model-specific retargeting. 🔗 lnkd.in/gsxhiJnn 2️⃣ Kimodo: High-fidelity, controllable text-to-motion generation for both humans and humanoid robots. 🔗 lnkd.in/gCc84XnX 3️⃣ GEM: A global human pose estimation method from in-the-wild videos, natively compatible with SOMA. 🔗 lnkd.in/g_QAvRjn 4️⃣ Bones-SEED: A massive dataset of 150k+ motions in SOMA format, including data already retargeted for the Unitree G1, created with our partners at Bones Studio. 🔗 lnkd.in/gfx-QD-w 🔗 lnkd.in/gyNdTwQx 5️⃣ SOMA Retargeter: A dedicated tool for seamless motion retargeting from the SOMA skeleton to the Unitree G1. 🔗 lnkd.in/gqz9Na-H 6️⃣ ProtoMotions: Our high-performance simulation framework for training digital human(oid)s via RL, now with native SOMA support. 🔗 lnkd.in/gmvMikMU This is just the beginning, and we have much more in the pipeline. Excited to see what the community builds next! #NVIDIA #GTC #GTC2026 #Robotics #EmbodiedAI #PhysicalAI @NVIDIAAI
5
77
8
426
49,826