NVIDIA Research Scientist - Generative Graphics

Zurich
Joined November 2021
Congrats to @LukasHollein for his successful PhD defense! Lukasโ€™s thesis is on ๐†๐ž๐ง๐ž๐ซ๐š๐ญ๐ข๐ง๐  ๐Ÿ‘๐ƒ ๐–๐จ๐ซ๐ฅ๐๐ฌ ๐Ÿ๐ซ๐จ๐ฆ ๐Ÿ๐ƒ ๐†๐ž๐ง๐ž๐ซ๐š๐ญ๐ข๐ฏ๐ž ๐Œ๐จ๐๐ž๐ฅ๐ฌ. His most impactful work so far, Text2Room (ICCVโ€™23 Oral), was the first method to generate complete room-scale 3D scenes with textured geometry from text alone. The key idea is to lift pre-trained 2D text-to-image models into 3D: iteratively generate new views, estimate depth, and fuse everything into a seamless textured mesh, with a carefully designed viewpoint selection to complete the scene. This inspired many others on text-to-3D scene generation and led to a long line of follow-up works, including Ctrl-Room, LucidDreamer, WonderJourney, WorldGen, and many more. Today, the Text2Room paradigm of leveraging 2D priors has become the primary approach for both image and video diffusion models to generate 3D scenes. However, Lukas has done much more cutting-edge research. ViewDiff turns text-to-image models into 3D-consistent multi-view generators, WorldExplorer generates fully navigable 3D scenes from video diffusion models, and 3DGS-LM speeds up Gaussian Splatting optimization with Levenberg-Marquardt, and most recently World Reconstruction From Inconsistent Views. In summary, his PhD has helped define how we generate 3D worlds with generative AI. Lukas also keeps pushing the boundaries of 3D world generation. Heโ€™s now continuing his research at Apple and I'm sure we can expect more great research very soon! Check his great research: lukashoel.github.io Congrats Dr. Hรถllein, you will be deeply missed in the lab!
6
10
3
148
8,009
๐Ÿ“ข 3D world models from video diffusion suffer from inconsistent frames -> blurry output. Our fix: instead of naรฏve 3D reconstruction, we non-rigidly align each frame into a globally-consistent 3DGS representation. ->sharp visuals on top of any VDM! lukashoel.github.io/video_toโ€ฆ
6
81
3
519
53,514
๐Ÿ“ข๐Ÿ“ข๐Ÿ“ขData release: high-res, multi-view, OLAT face recordings ๐Ÿ“ข๐Ÿ“ข๐Ÿ“ข We captured individuals in our custom light stage with 16 high-end, global shutter cameras (72 fps) and 40 LED modules, totaling 2.8M precisely calibrated frames. We us the data for BecomingLit (#NeurIPS2025): intrinsically decomposed Gaussian avatars, enabling photorealistic and real-time relighting via hybrid neural shading. Code & Data: jonathsch.github.io/becomingโ€ฆ Great work by @jnthnschmdt, @SGiebenhain
5
40
1
236
19,550
๐Ÿ“ขPix2NPHM: Learning to Regress NPHM Reconstructions From a Single Image๐Ÿ“ข We directly regress neural parametric head models (NPHMs) from a single image โ€” fast, stable, and significantly more expressive than classical 3DMMs such as FLAME. Face tracking & 3D reconstruction are often limited by the representational capacity of PCA-based face models. By lifting NPHMs to a first-class reconstruction primitive, we enable more accurate geometry, richer expressions, and finer animation control. Pix2NPHM obtains fast and reliable NPHM reconstructions on real-world data. Inference-time optimization against surface normals and canonical point maps can further increase fidelity. Key to successful and generalized training of our ViT-based network are: (1) large-scale registration of existing 3D head datasets, and (2) self-supervised training on vast in-the-wild 2D video datasets using pseudo ground-truth surface normals. Finally, we show that geometry-aware pretraining on pixel-aligned reconstruction tasks significantly outperforms generic visual pretraining (e.g., DINO-style features) in terms of generalization. ๐ŸŒsimongiebenhain.github.io/Piโ€ฆ ๐ŸŽฅyoutu.be/MgpEJC5p1Ts Great work by @SGiebenhain, @TobiasKirschst1, @liamschoneveld, Davide Davoli, Zhe Chen
15
79
6
536
38,030
๐Ÿ“ข๐Ÿ“ข๐Ÿ“ข๐Œ๐ž๐ฌ๐ก๐‘๐ข๐ฉ๐ฉ๐ฅ๐ž: Structured Autoregressive Generation of Artist-Meshes High-fidelity, topologically complete 3D assets that expand naturally like a ripple on a surface! ๐ŸŒŠ Existing AR models often rely on sliding-window inference over truncated segments. However, this limitation breaks long-range geometric dependencies, causing holes and fragmentation. Instead, MeshRipple uses frontier-aware BFS and sparse-attention global memory to ensure coherent growth with an unbounded receptive field. -> Highly detailed-mesh generations -> Artist-like meshing quality -> Works on room-scale environments ๐ŸŒmaymhappy.github.io/MeshRippโ€ฆ ๐ŸŽฅyoutu.be/AHvaLslzXQU Great work by Junkai Lin, Hang Long, Huipeng Guo, Jielei Zhang, Jiayi Yang, Tianle Guo, Yang Yang, Jianwen Li, Wenxiao Zhang, Wei Yang
5
100
1
649
34,172
Releasing Echo today is incredibly exciting for me โ€” because it is a critical step for generative AI, enabling the creation of virtual worlds. Echo is our first world model at SpAItial AI. It turns text or images into explorable 3D environments โ€” spaces you can move through, inspect, and build on. Seeing this work in real time still feels a bit surreal. My fascination with this goes back a long way: video games, virtual environments, and the idea of capturing the real world in 3D. As a researcher, I spent years working on 3D reconstruction, neural rendering, and scene understanding โ€” all driven by the same question: how do we teach machines to understand the world? One thing became clear over time: the biggest bottleneck isnโ€™t compute or rendering โ€” itโ€™s 3D worlds themselves. High-quality, consistent environments are expensive to create by hand and donโ€™t scale to the experiences we want to build. In particular, I believe that the ability to generate virtual worlds is ultimately key towards understanding the real world. Thatโ€™s why we founded SpAItial AI. Weโ€™re building spatial world models that combine geometric understanding with creative generation โ€” models that can generate, edit, and eventually reason about 3D environments. Echo is just the beginning. For me, this feels like the moment when decades of research finally meet the imagination that got many of us into graphics, games, 3D understanding in the first place.๐ŸŒ spaitial.ai/
๐Ÿš€ Announcing Echo โ€” our new frontier model for 3D world generation. Echo turns a simple text prompt or image into a fully explorable, 3D-consistent world. Instead of disconnected views, the result is a single, coherent spatial representation you can move through freely. This is part of a bigger shift in AI: from generating pixels and tokens to generating spaces. Echo predicts a geometry-grounded 3D scene at metric scale, meaning every novel view, depth map, and interaction comes from the same underlying world โ€” not independent hallucinations. Once generated, the world is interactive in real time. You control the camera, explore from any angle, and render instantly โ€” even on low-end hardware, directly in the browser. High-quality 3D world exploration is no longer gated by expensive equipment. Under the hood, Echo infers a physically grounded 3D representation and converts it into a renderable format. For our web demo, we use 3D Gaussian Splatting (3DGS) for fast, GPU-friendly rendering โ€” but the representation itself is flexible and can be easily adapted. Why this matters: consistent 3D worlds unlock real workflows โ€” digital twins, 3D design, game environments, robotics simulation, and more. From a single photo or a line of text, Echo builds worlds that are reliable, editable, and spatially faithful. Echo also enables scene editing and restyling. Change materials, remove or add objects, explore design variations โ€” all while preserving global 3D consistency. Editing no longer breaks the world. This is only the beginning. Echo is the foundation for future world models with dynamics, physical reasoning, and richer interaction โ€” environments that donโ€™t just look right, but behave right. Explore the generated worlds on our website and sign up for the closed beta. The era of spatial intelligence starts here. ๐ŸŒ #Echo #WorldModels #SpatialAI #3DFoundationModels Check it out: spaitial.ai/
24
100
5
626
71,358
Peter Kocsis retweeted
Radiance Meshes for Volumetric Reconstruction ๐ŸŽ‰ We made a thing! A Radiance Field that is composited out of a triangle meshes that renders accurately the underlying volumetric field, with no popping AND faster than approximate methods like Gaussian Splatting.
5
37
2
306
26,523
FLUX.2 is here - our most capable image generation & editing model to date. Multi-reference. 4MP. Production-ready. Open weights. Into the new.
161
458
245
2,925
485,242
Peter Kocsis retweeted
๐Ÿ“ขProcGen3D: Learning Neural Procedural Graphs for Image-to-3D Reconstruction @xinyi092298 learns neural procedural graphs to generate high-fidelity 3D - MCTS-guided sampling maintains consistency with the input image, even from real images! Check it out: xzhang-t.github.io/project/Pโ€ฆ
8
52
1
237
14,715
๐Ÿ“ข๐Ÿ“ข ๐๐ž๐ซ๐œ๐‡๐ž๐š๐: ๐๐ž๐ซ๐œ๐ž๐ฉ๐ญ๐ฎ๐š๐ฅ ๐‡๐ž๐š๐ ๐Œ๐จ๐๐ž๐ฅ ๐Ÿ๐จ๐ซ ๐’๐ข๐ง๐ ๐ฅ๐ž-๐ˆ๐ฆ๐š๐ ๐ž ๐Ÿ‘๐ƒ ๐‡๐ž๐š๐ ๐‘๐ž๐œ๐จ๐ง๐ฌ๐ญ๐ซ๐ฎ๐œ๐ญ๐ข๐จ๐ง & ๐„๐๐ข๐ญ๐ข๐ง๐ ๐Ÿ“ข๐Ÿ“ข PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text. At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS. Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output. In addition, our architecture seamlessly adapts to downstream tasks: by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt. We also provide an interactive GUI to enable the exploration of our editing pipeline. ๐ŸŒ antoniooroz.github.io/PercHeโ€ฆ ๐Ÿ“ฝ๏ธ youtu.be/4hFybgTk4kE Great work by @antonio_oroz and @TobiasKirschst1
5
37
5
257
18,936
Peter Kocsis retweeted
๐Ÿ“ขNew in ScanNet++: High-Res 360ยฐ Panos! @chandan__yes & @liuyuehcheng have added pano captures for 956 ScanNet++ scenes, fully aligned with the 3D meshes, DSLR, and iPhone data - multiple panos per scene Check it out: Docs kaldir.vc.in.tum.de/scannetpโ€ฆ Code github.com/scannetpp/scannetโ€ฆ
44
221
12,937
Fantastic retreat this weekend by our research groups! Internal reviews, ideas brainstorming, paper reading, and much more! Of course also many social activities -- the highlight being our kayaking trip - lots of fun :)
1
7
87
10,300
So You Want to Be an Academic? A couple of years into your PhD, but wondering: "Am I doing this right?" Most of the advice is aimed at graduating students. But there's far less for junior folks who are still finding their academic path. My candid takes: anandbhattad.github.io/blogsโ€ฆ
16
95
17
737
92,031
Can we use video diffusion to generate 3D scenes? ๐–๐จ๐ซ๐ฅ๐๐„๐ฑ๐ฉ๐ฅ๐จ๐ซ๐ž๐ซ (#SIGGRAPHAsia25) creates fully-navigable scenes via autoregressive video generation. Text input -> 3DGS scene output & interactive rendering! ๐ŸŒmschneider456.github.io/worlโ€ฆ ๐Ÿ“ฝ๏ธyoutu.be/N6NJsNyiv6I
7
74
5
373
30,931
HairGS: Hair Strand Reconstruction based on 3D Gaussian Splatting (#BMVC2025) ๐Ÿ“ข We reconstruct realistic 3D hair strands from multi-view images in ~1 hour. Results show robustness to a wide range of hair styles such as challenging curly hair. Project: yimin-pan.github.io/hair-gs/
4
48
1
225
15,613
๐Ÿ“ขCode release for 3DGS-LM (#ICCV2025)๐Ÿ“ข Gaussian-Splatting made faster by changing the underlying optimizer. It's compatible with other changes that accelerate the optimization along different axes. lukashoel.github.io/3DGS-LM/
(1/2) How to accelerate the reconstruction of 3D Gaussian Splatting? 3DGS-LM replaces the commonly used ADAM optimizer with a tailored Levenberg-Marquardt (LM). => We are ๐Ÿ‘๐ŸŽ% ๐Ÿ๐š๐ฌ๐ญ๐ž๐ซ ๐ญ๐ก๐š๐ง ๐Ÿ‘๐ƒ๐†๐’ for the same quality. lukashoel.github.io/3DGS-LM/ youtu.be/tDiGuGMssg8
3
26
1
127
15,703
ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions (#SIGGRAPH) We reconstruct ultra-high fidelity photorealistic 3D avatars capable of generating realistic and high-quality animations including freckles and other fine facial details. We operate on patch-based local expression features and increase the representation capacity by synthesizing 3D Gaussians dynamically by leveraging tiny scaffold MLPs conditioned on localized expressions. We further propose a color-based densification and progressive training scheme for improved quality and faster convergence. Project: shivangi-aneja.github.io/proโ€ฆ Video: youtu.be/VyWkgsGdbkk Great work by @shivangi2201, Sebastian Weiss, Irene Baeza, Prashanth Chandran, Gaspard Zoss, Derek Bradley
2
36
1
175
17,338
๐Ÿ“ข LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans๐Ÿ โœจ -> converts RGB-D scans into compact, realistic, and interactive 3D scenes โ€” featuring high-quality meshes, PBR materials, and articulated objects. ๐Ÿ“ทyoutu.be/ecK9m3LXg2c ๐ŸŒlitereality.github.io/
5
64
3
319
23,917
Peter Kocsis retweeted
Check out our #ICCV2025 work on functional 3d scan editing, learning to optimize, multi-level 3d captioning, interactive mesh editing, audio-driven avatars, & shape matching! Congrats @ElBoudjogh24002, @liuyuehcheng, @chandan__yes, @hcxrli, @shivangi2201, Emery for amazing work!
2
27
118
9,582
Presenting PrEditor3D at #CVPR2025 ๐Ÿ“ข๐Ÿ“ข If you'd like to learn more about our work and discuss 3D generation/editing, come visit our poster on Friday, June 13th, in ExHall D between 10:30-12:30 (Poster #44). Project Page: ziyaerkoc.com/preditor3d
11
3
34
3,949