Researcher at @Waymo Research; ML/Embodied AI; PhD @Princeton ECE
NYC
Joined August 2018
- Tweets49
- Following287
- Followers666
- Likes117
If you're a PhD student looking for an impactful research internship for Summer 2027, our team would like to hear from you! Topics include, but are not limited to, world models, RL, post-training, and planning: careers.withwaymo.com/jobs/2…
I rarely repost content, but this is a step-change for robotics research. As an embodied AI researcher, I find this to be a truly special moment for the field
Jimmy (Tsung-Yen) Yang retweeted
📢 Accepted to TMLR, with reproducibility certification 🏅
v2 of our JEPA-WM study (arXiv:2512.24497) is out, with new data-scaling experiments, a Lipschitz analysis of multistep rollout training, and extended discussions.
Recap + what's new 👇
w/ @JimmyTYYang1, Jean Ponce, @AdrienBardes, @ylecun
My first PhD paper is out! 🎓
"What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?"
tl:dr: JEPA-WMs for robotics: learn dynamics on top of visual encoders, optimize actions towards goal 👇
w/ @JimmyTYYang1, Jean Ponce, @AdrienBardes, @ylecun
It was great to work with @BasileTerv987 on this project! The JEPA world model works well on unseen robot tasks. Thanks Basile for the amazing work!
My first PhD paper is out! 🎓
"What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?"
tl:dr: JEPA-WMs for robotics: learn dynamics on top of visual encoders, optimize actions towards goal 👇
w/ @JimmyTYYang1, Jean Ponce, @AdrienBardes, @ylecun
It is my pleasure to work with Raktim! A concrete example of how to use human videos to learn robot policies
Introducing DexWM: Dexterous Manipulation World Model
tl;dr: train on human videos; fine-grained actions; hand-consistency loss; DexWM+MPC -> zero-shot dexterous manipulation
w/ @_amirbar, @DavidJFan, @JimmyTYYang1, @GaoyueZhou, P. Krishnamurthy, M. Rabbat, F. Khorrami, @ylecun
It was so much fun to work on this: llama3 + sim2real skills, all are open-sourced!
FAIR researchers (@AIatMeta) presented SegmentAnything and our robotics work at the White House correspondents’ weekend.
Llama3 + Sim2Real skills (trained with @ai_habitat) = a robot assistant
Jimmy (Tsung-Yen) Yang retweeted
FAIR researchers (@AIatMeta) presented SegmentAnything and our robotics work at the White House correspondents’ weekend.
Llama3 + Sim2Real skills (trained with @ai_habitat) = a robot assistant
Washingtonians delved into the world of artificial intelligence (AI) at the Washington AI Network’s inaugural weekend TGAIFriday Lunch for White House correspondents. trib.al/FwHF9Um
Very excited to see that Habitat 3.0 is finally out! Please also check out our paper here: ai.meta.com/static-resource/…
Please check out our Spot Sim2Real repo! github.com/facebookresearch/…
It aims to reproduce the demos of ASC and LSC. We open-source the weights of the navigation, picking, and place skills, and the infrastructure to use Large Language Models (LLMs) to interact with those skills!
Habitat stable version v0.2.5 released!
-- 🤖 Hardware bridge for BD Spot
-- 📷 Improved PBR rendering for better synthetic scene visuals
-- 🤗 ai-habitat on HuggingFace
-- 🏠 Habitat Synthetic Scenes Dataset (HSSD)
and more...
github.com/facebookresearch/…
github.com/facebookresearch/…
Jimmy (Tsung-Yen) Yang retweeted
“We were very interested in testing out high-level reasoning and planning. We wanted to see...if the robot could figure out the steps it needed to take to solve the problem.” - Discover why @MetaAI chose Spot for their research. bit.ly/44ILlhG
This is our recent work building toward "reproducible" robotic research. Thanks @chris_j_paxton for leading such an amazing project!
The future of robot butlers starts with mobile manipulation.
We’re announcing the NeurIPS 2023 Open-Vocabulary Mobile Manipulation Challenge!
- Full robot stack ✅
- Parallel sim and real evaluation ✅
- No robot required ✅👀
aihabitat.org/challenge/2023…
our demo @CVPR!
FAIR and @gtcomputing researchers demoing our sim2real work on @BostonDynamics Spot at @MetaAI booth at #CVPR23.
Check our work at Meta AI #CVPR23 ! Spot is able to understand natural language instructions and complete the task for you!
We’ll also be showcasing a handful of demos on the showfloor. We brought Spotty to demo some of our exciting recent work towards general-purpose embodied AI agents.
More on this research ➡️ bit.ly/3N8LeVg
5/6
Jimmy (Tsung-Yen) Yang retweeted
We’ll also be showcasing a handful of demos on the showfloor. We brought Spotty to demo some of our exciting recent work towards general-purpose embodied AI agents.
More on this research ➡️ bit.ly/3N8LeVg
5/6
Come join us for an instruction-following robot assistant demo at #CVPR2023 Meta Booth by the FAIR Embodied AI team.
Give Spot a free-form natural language instruction for object fetching.
Details: languageguidedskillcoordinat…
🧵
We will be running the LSC demo at two locations at #CVPR2023:
Meta AI booth: Tuesday to Thursday 10:00 AM to 6:00PM.
CVPR Demo track booth: Thursday 10:00 AM to 6:00PM
Details: languageguidedskillcoordinat…
Come join us!
This is the joint work of my amazing colleague: Sergio Arnaud, Kavit Shah, @naokiyokoyama0 , Alex Clegg, @joannetruong , Eric Undersander, Oleksandr Maksymets, @sehoonha , Mrinal Kalakrishnan, Roozbeh Mottaghi, @DhruvBatraDB , and Akshara Rai