MingfeiYan retweeted
a tiny AI 0.5B just turned a normal phone video into a fully labeled 3D room run phone.
SpatialLM that understands walls, doors, furniture, and everything layout.
it trained as a standard multimodal LLM.
Most vision models see pixels, SpatialLM sees space.
Open models : github.com/manycore-research…
MingfeiYan retweeted
Check out CoMind, an ECCV 2026 dataset exploring how people collaborate, from two participants; perspectives!
Captured with Project Aria glasses, CoMind combines synchronized egocentric video, gaze, audio, and 3D scans to study shared attention, anticipate interactions, and predict handovers. A rich resource for building AI that understands and supports human collaboration.
Fantastic work by the CoMind team! 📷 @xiwang1212
🚀 New from our team: CoMind (comind.ethz.ch/) - a dataset for AI that understands not just what I am doing, but what WE are doing together.
Two people. Two egocentric views. Gaze. Speech. 3D scenes. Social cues.
Accepted at #ECCV2026 🧵👇
MingfeiYan retweeted
I know teleoperation is old these days but it feels magical when you could teleoperate a humanoid just by wearing a pair of classes (bonus: the robot also sees from glasses too!)
Of course we don't stop here --- we are working on a system to train robots with everyday human experience, without any teleop data. More to come!
Kudos to the team Chuye Zhang @DanielZhenyang @ChuizhengKong
MingfeiYan retweeted
sEMG×一人称カメラでロボット学習
🔹Wetour Roboticsが8ch sEMGリストバンド+カメラを統合
🔹視覚で手の動き、sEMGで筋活動を取得
🔹視覚単体では手を21.8%のフレームで検出できず、最大4.32秒欠落
1秒分のsEMG処理は50.4ms。情報元はリプ欄。
MingfeiYan retweeted
I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.
Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
MingfeiYan retweeted
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
MingfeiYan retweeted
Heading to #ECCV2026? Join us for the Fifth Hands-On Egocentric Research Tutorial — a half-day, hands-on deep dive into egocentric data with Project Aria.
📍Palissad West, Floor 0 (must take the stairs)
Tue, Sept 8
9AM–12:30PM
Can’t make it to Sweden? We’ll post the whole session on YouTube after the conference.
projectaria.com/project_aria…
MingfeiYan retweeted
New Anthropic research: Project Fetch.
We asked two teams of Anthropic researchers to program a robot dog. Neither team had any robotics expertise—but we let only one team use Claude.
How did they do?
MingfeiYan retweeted
Introducing Depth Estimation with NVIDIA’s FoundationStereo—a new open tutorial and codebase for metric depth sensing using Project Aria Gen 2 devices.
⚡️ Real-time ready (20 FPS)
🤖 Zero-shot generalization
⚡️Stereo image rectification and disparity estimation
🔓 Open Source
Check out the code and tutorial: github.com/facebookresearch/…
🔌@the_shweenz
MingfeiYan retweeted
Meta and Project Aria are headed to Denver for CVPR 2026. ✈️
The Project Aria team will be onsite, highlighting the latest from our program on the future of Contextual AI and Robotics research! 🤖
📍 Denver, Colorado
📅 June 3–7, 2026
🤝Booth 201
We'll see you tomorrow for presentations on the cutting edge of computer vision research across CVPR's workshops and tutorials. See our snapshot agenda below for details!
MingfeiYan retweeted
Breaking News: Anthropic, the A.I. company behind Claude, filed to go public, setting the stage for a massive IPO. nyti.ms/4vlQ7yz
Introducing Claude Opus 4.8: it builds on Opus 4.7 with sharper judgment, more honesty about its own progress, and the ability to work independently for longer than its predecessors.
Available today at the same price.
MingfeiYan retweeted
How do we train AI to be a real-world expert? 🛠️🧠
Introducing Ego-EXTRA, a new video-language dataset for Expert-Trainee assistance. Researchers at the University of Catania captured unscripted interactions where experts guide trainees through complex tasks in real-time using Aria Glasses.
👓 50 hours of data
💬 15k+ high-quality VQA sets
✅ RGB + Eye Gaze tracking ✅ Hand keypoints & SLAM ✅ IMU, Magnetometer, & Audio ✅ Natural expert-trainee dialogue
🚲 4 scenarios: Bike Workshop, Kitchen, Bakery, & Assembly
Explore the benchmark: fpv-iplab.github.io/Ego-EXTR…
Read the paper: arxiv.org/abs/2512.13238
👉@FrancRagusa, @GMFarinella
Introducing Claude Design by Anthropic Labs: make prototypes, slides, and one-pagers by talking to Claude.
Powered by Claude Opus 4.7, our most capable vision model. Available in research preview on the Pro, Max, Team, and Enterprise plans, rolling out throughout the day.
MingfeiYan retweeted
Introducing Project Glasswing: an urgent initiative to help secure the world’s most critical software.
It’s powered by our newest frontier model, Claude Mythos Preview, which can find software vulnerabilities better than all but the most skilled humans.
anthropic.com/glasswing
MingfeiYan retweeted
Today, we’re releasing a feature that allows Claude to control your computer: Mouse, keyboard, and screen, giving it the ability to use any app.
I believe this is especially useful if used with Dispatch, which allows you to remotely control Claude on your computer while you’re away.
MingfeiYan retweeted
Introducing EgoVerse: an ecosystem for robot learning from egocentric human data.
Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling
1300+ hrs, 240 scenes, 2000+ tasks, and growing
Dataset design, findings, and ecosystem 🧵
MingfeiYan retweeted
#NVIDIA just released a whole ecosystem for human(oid) motion and robot learning from human data. 🚀🦾
Data, as we all know, is the key to scaling AI models. To accelerate the field of Embodied AI, we have open-sourced a full stack of models and tools to capture, generate, retarget, and simulate human(oid) motion data at scale, along with a massive high-quality dataset and a standard human skeletal representation, SOMA, to make them all seamlessly communicate with each other.
The entire suite is available under the Apache 2.0 license.
1️⃣ SOMA: A universal interface to unify all parametric human body models (SOMA-shape, SMPL, MHR, etc.) into a standard skeletal representation, eliminating the need for custom adapters or model-specific retargeting.
🔗 lnkd.in/gsxhiJnn
2️⃣ Kimodo: High-fidelity, controllable text-to-motion generation for both humans and humanoid robots.
🔗 lnkd.in/gCc84XnX
3️⃣ GEM: A global human pose estimation method from in-the-wild videos, natively compatible with SOMA.
🔗 lnkd.in/g_QAvRjn
4️⃣ Bones-SEED: A massive dataset of 150k+ motions in SOMA format, including data already retargeted for the Unitree G1, created with our partners at Bones Studio.
🔗 lnkd.in/gfx-QD-w
🔗 lnkd.in/gyNdTwQx
5️⃣ SOMA Retargeter: A dedicated tool for seamless motion retargeting from the SOMA skeleton to the Unitree G1.
🔗 lnkd.in/gqz9Na-H
6️⃣ ProtoMotions: Our high-performance simulation framework for training digital human(oid)s via RL, now with native SOMA support.
🔗 lnkd.in/gmvMikMU
This is just the beginning, and we have much more in the pipeline. Excited to see what the community builds next!
#NVIDIA #GTC #GTC2026 #Robotics #EmbodiedAI #PhysicalAI @NVIDIAAI



















