@kvuongdev

PhD-ing @CMU_Robotics | interning @MetaAI, ex-intern @Apple | Vision, Robotics

 Pittsburgh, PA
khiemvuong.com/
Joined February 2018
  • Tweets85
  • Following372
  • Followers341
18Media
[1/6] Can one model fix rendering artifacts from any 3D representation? And what does it even mean to "fix" a rendering? At #ECCV2026, we present FixAnything (fix-anything.github.io): a single generalist video model that refines 3DGS/mesh/sparse point-cloud renderings into photorealistic, 3D-consistent videos.
7
45
4
263
16,776
Khiem Vuong retweeted
Keenan Crane
@keenanisaliveOct 9
With everyone busily using LLMs + Blender/Three.js to make 3D models, I thought I'd try a different geometric representation. This dragon isn't a mesh, NeRF, 3DGS, or video model: it's 27kb of GLSL code (generated via LLM), defining a closed-form implicit surface. 🧵 [1/n]
139
476
71
6,243
683,981
Current image-to-3D-object methods are either not pixel-aligned (resulting in wrong colours, missing details, and inability to match the input), or bad at generalizing beyond what it is seen. With @StefanoEsp we fix this, by using rendering-guidance during denoising to pixel-align an otherwise not-aligned image-to-3D-object model, resulting in state-of-the-art results for both seen and unseen parts! We also extend this to work with multiple input images and video too!
Stefano Esposito@StefanoEspOct 9
One photo in, a complete 3D object out, faithful in shape, color & pose. GenIA grounds a frozen 3D foundation model (SAM3D) in the input images at test time. Works from single images, multi-view, even monocular video 🧵 🌐 facebookresearch.github.io/G…
2
11
1
91
6,662
Khiem Vuong retweeted
Zihan Wang
@Z1hanWSep 30
Humans generalize from rich interaction experience. We bring this to humanoid loco-manipulation through scalable real-to-sim-to-real. Introducing PRISM, we expand 4 real videos into 256 counterfactual variants, recover robot-object trajectories, and train one policy that generalizes across diverse objects, scales, and spatial layouts. CoRL’26 🚀 Website for details 👉 prism-real2sim2real.github.i… Work done at Amazon FAR w/ @zhenkirito123 @pabbeel @rocky_duan @JitendraMalikCV @carlo_sferrazza @ckarenliu @GuanyaShi @akanazawa
12
75
15
373
153,962
Introducing PointZero—a 3D world model pre-trained without robots. Dexterous manipulation requires understanding diverse 3D dynamics, but current approaches rely on expensive robot data. So how can we pre-train a 3D dynamics model without it? PointZero introduces a simple idea: learning to complete 3D point tracks yields transferable 3D dynamics! Our model achieves SOTA results on: 🔥 Zero-shot 3D dynamics 🔥 Action-conditioned 3D dynamics (post-training) 🔥 Imitation learning (post-training) Details and links 👇 1/6
8
46
5
260
28,926
Khiem Vuong retweeted
Adam Hung
@AdamjhungSep 16
World-action models typically imagine the future in RGB – but are pixels really the right representation for robotics? Our bet is no: RGB spends capacity on fine-grained details and variation that are often irrelevant for robot policies. DINO features, point tracks, and depth capture more useful features like semantics, motion, and geometry. But no single modality captures everything — how can we effectively combine them? We introduce ✨ModAR✨, which predicts the future one modality at a time, with each prediction informing the next. We train from scratch and find that this formulation performs best. Our 30.1M scratch-trained model even outperforms a 6B video-model-initialized model finetuned on the same data! 🧵 [1/8]
12
61
5
410
38,880
Khiem Vuong retweeted
Minsik Jeon@Minsik_Je0nSep 10
[1/N] Introducing Point4D: dense 4D reconstruction across hundreds of frames from a monocular video. Unlike existing feed-forward 4D methods that operate over short windows, Point4D keeps tracking in 3D, even across occlusions and out-of-frame motion.
6
47
3
331
35,098
Curious how little finetuning it takes for a video model to clean up rendering artifacts — 3DGS, NeRF, mesh, sparse point clouds — while those same renders give you camera control capabilities? Come check out our poster at #ECCV2026 (ExHall #283) tomorrow!! Sadly I can't be at ECCV this year, but please catch @PeterHedman3 who is helping me present the poster — come say hi! 📍 ExHall #283 at 10:30 AM CEST / Thu, Sep 10, 2026 📰 eccv.ecva.net/virtual/2026/p… 🏠 Project Page: fix-anything.github.io/
Khiem Vuong
@kvuongdevAug 26
[1/6] Can one model fix rendering artifacts from any 3D representation? And what does it even mean to "fix" a rendering? At #ECCV2026, we present FixAnything (fix-anything.github.io): a single generalist video model that refines 3DGS/mesh/sparse point-cloud renderings into photorealistic, 3D-consistent videos.
1
28
2,383
If you are curious about how to turn a video model into a multiview novel view synthesis model, come see our poster at #ECCV2026 (ExHall #275) tomorrow!! Sadly I will have to miss ECCV this year, but @qi_wu57 will be there presenting — go say hi! ExHall # 275 at 10:30 AM CEST / Thu, Sep 10, 2026 eccv.ecva.net/virtual/2026/p… arXiv: arxiv.org/abs/2604.08500
Khiem Vuong
@kvuongdevApr 22
[1/7] Video diffusion has come a long way, generating more & more realistic videos. Can we revisit sparse-view novel view synthesis through these video priors? Meet FrameCrafter: a permutation-invariant multi-view model built on video diffusion 🧵 🌐 frame-crafter.github.io
10
1,265
My group at Johns Hopkins now has a name and a website Introducing the Brains, Bots, and Behavior Lab b3.cs.jhu.edu We're always looking for creative researchers at all levels to join us!
15
63
5
567
46,803
Thanks @orlitany for organizing, had lots of fun discussion! If interested, please also check out FixAnything, the paper I most talked about, here xitter.cf/kvuongdev/status/20926….
Or Litany@orlitanyAug 26
Had the pleasure of (virtually) hosting @kvuongdev today at my lab and hear about his awesome work! Thanks for sharing your insights from working on 3D and video models.
2
420
Thanks @kwangmoo_yi for the post! If interested, please also see our thread here: xitter.cf/kvuongdev/status/20926….
Kwang Moo Yi@kwangmoo_yiAug 26
Vuong et al., "FixAnything: 3D-Consistent Rendering Refinement via Video Generative Priors" Fix intermediate render representations via a fine-tuned video model. Uses LoRA and DPO (with geometric consistency reward)
1
1,033
[5/6] Can we tell where the model is hallucinating? Yes! As preliminary experiment, we run inference 5× with different seeds → per-pixel std = a training-free uncertainty map. Quantitatively, we found that it closely correlates with reconstruction error!
1
3
437
[6/6] Check out the project page + paper + code for more results and details! 🌐 fix-anything.github.io Sadly I can't be at #ECCV2026 in person, but do catch @PeterHedman3 at the poster session (#283, Session 1) if you'd like to chat (thanks Peter!). Work done w/ @RamananDeva and Srinivasa Narasimhan at CMU.
6
432
Introducing MotionForesight: A very simple approach re-purposing video models for forecasting future 3D scene flow given a short observed video context Trained with just 40k RGB human videos, it generalizes well to completely new home/office scenes! 1/n
6
33
4
120
16,652
This is super cool, congrats on the amazing launch @ethanjohnweber!
Ethan Weber@ethanjohnweberJul 8
Today we launched Muse Image and are previewing Muse Video! 🚀 ai.meta.com/blog/introducing… 👈 Muse Image does agentic image generation! It's great at tool use and search. I highly recommend trying it out on meta.ai! Excited to continue! 🧑‍🍳🔥
2
222
Load more