Peter Kocsis retweeted
Congrats to @LukasHollein for his successful PhD defense!
Lukasโs thesis is on ๐๐๐ง๐๐ซ๐๐ญ๐ข๐ง๐ ๐๐ ๐๐จ๐ซ๐ฅ๐๐ฌ ๐๐ซ๐จ๐ฆ ๐๐ ๐๐๐ง๐๐ซ๐๐ญ๐ข๐ฏ๐ ๐๐จ๐๐๐ฅ๐ฌ. His most impactful work so far, Text2Room (ICCVโ23 Oral), was the first method to generate complete room-scale 3D scenes with textured geometry from text alone.
The key idea is to lift pre-trained 2D text-to-image models into 3D: iteratively generate new views, estimate depth, and fuse everything into a seamless textured mesh, with a carefully designed viewpoint selection to complete the scene. This inspired many others on text-to-3D scene generation and led to a long line of follow-up works, including Ctrl-Room, LucidDreamer, WonderJourney, WorldGen, and many more. Today, the Text2Room paradigm of leveraging 2D priors has become the primary approach for both image and video diffusion models to generate 3D scenes.
However, Lukas has done much more cutting-edge research. ViewDiff turns text-to-image models into 3D-consistent multi-view generators, WorldExplorer generates fully navigable 3D scenes from video diffusion models, and 3DGS-LM speeds up Gaussian Splatting optimization with Levenberg-Marquardt, and most recently World Reconstruction From Inconsistent Views. In summary, his PhD has helped define how we generate 3D worlds with generative AI.
Lukas also keeps pushing the boundaries of 3D world generation. Heโs now continuing his research at Apple and I'm sure we can expect more great research very soon!
Check his great research: lukashoel.github.io
Congrats Dr. Hรถllein, you will be deeply missed in the lab!
Happy to share that Iโve successfully defended my PhD! ๐
Excited to continue this research at NVIDIA and explore what comes next in generative graphics!
Congrats to @Peter4AI for his successful PhD defense!
Peterโs thesis is on ๐๐๐ง๐๐ซ๐๐ญ๐ข๐ฏ๐ ๐๐ซ๐ข๐จ๐ซ๐ฌ ๐๐จ๐ซ ๐๐๐ญ๐๐ซ๐ข๐๐ฅ ๐๐ฌ๐ญ๐ข๐ฆ๐๐ญ๐ข๐จ๐ง ๐๐ง๐ ๐๐ข๐ ๐ก๐ญ๐ข๐ง๐ ๐๐จ๐ง๐ญ๐ซ๐จ๐ฅ. His arguably most impactful work, Intrinsic Image Diffusion, was among the first to bring diffusion models to material estimation.
The key idea is that intrinsic decomposition is inherently ambiguous, so rather than regressing a single answer, it is formulated probabilistically and samples plausible decompositions, leveraging the strong image priors of diffusion models.
This opened up a new line of diffusion-based inverse rendering work, including IntrinsicAnything, Marigold-IID, DiffusionRenderer, and many more. Today, generative priors are the standard approach for material estimation and relightable reconstruction.
However, Peter has done much more cutting-edge research: LightIt brought explicit lighting control to diffusion models, IntrinsiX enabled direct PBR generation from text, and Intrinsic Image Fusion lifts these priors to high-quality multi-view 3D material reconstruction. In summary, his PhD has shaped how we think about materials and lighting in the age of generative AI.
Iโm also particularly proud that Peter is continuing this line of research as a Research Scientist at NVIDIA, e.g., his new work on Texture Space Material Diffusion and NeuMatEx.
Also thanks to Ruediger Westermann and George Drettakis for being on the committee.
Check out Peter's fantastic work: peter-kocsis.github.io
Congrats Dr. Kocsis, it was an honor to have you in the lab!
Thanks! Unfortunately, we don't have a recording, but you can find our works on my webpage: peter-kocsis.github.io/
Dissertation will also be public!
๐ข Texture Space Material Diffusion
We generate relightable materials from photos or text using video diffusion priors directly in texture space, scaling to 8K textures.
Great project with Jacob Munkberg (@yoslber) and Jon Hasselgren!
nvlabs.github.io/texdiffusioโฆ
๐ขNeuMatEx - Material Reconstruction beyond PBR
We extract neural materials from multi-view images using a Large Material Reconstruction Model combined with differentiable rendering,
Great project from @kim_youwang!
nvlabs.github.io/neumatex/
Excited to share that Iโve joined NVIDIA as a Research Scientist in Generative Graphics! ๐
Looking forward to working on generative rendering and the next steps in graphics with an incredible team. Grateful to everyone who helped along the way!
#NVIDIAlife
We present Intrinsic Image Fusion in Poster Session 4 (106) on Saturday at #CVPR ๐ฅณ
Come by to learn more about room-scale material estimation!
PS: Code is released ๐
๐ข Intrinsic Image Fusion for Multi-View 3D Material Reconstruction ๐ข
We combine generative material priors with inverse path tracing: 1) define a parametric texture space 2) fuse monocular predictions across views into consistent textures 3) optimize low-dimensional parameters for physically-grounded reconstructions.
The results are relightable PBR textures for 3D scenes: check out the result on a real-world 3D scan from the ScanNet++ dataset!
๐peter-kocsis.github.io/Intriโฆ
๐ฅyoutu.be/-Vs3tR1Xl7k
Great work by @Peter4AI @LukasHollein!
Peter Kocsis retweeted
๐ข 3D world models from video diffusion suffer from inconsistent frames -> blurry output.
Our fix: instead of naรฏve 3D reconstruction, we non-rigidly align each frame into a globally-consistent 3DGS representation.
->sharp visuals on top of any VDM!
lukashoel.github.io/video_toโฆ
Peter Kocsis retweeted
๐ข๐ข๐ขData release: high-res, multi-view, OLAT face recordings ๐ข๐ข๐ข
We captured individuals in our custom light stage with 16 high-end, global shutter cameras (72 fps) and 40 LED modules, totaling 2.8M precisely calibrated frames.
We us the data for BecomingLit (#NeurIPS2025): intrinsically decomposed Gaussian avatars, enabling photorealistic and real-time relighting via hybrid neural shading.
Code & Data: jonathsch.github.io/becomingโฆ
Great work by @jnthnschmdt, @SGiebenhain
Peter Kocsis retweeted
๐ขPix2NPHM: Learning to Regress NPHM Reconstructions From a Single Image๐ข
We directly regress neural parametric head models (NPHMs) from a single image โ fast, stable, and significantly more expressive than classical 3DMMs such as FLAME.
Face tracking & 3D reconstruction are often limited by the representational capacity of PCA-based face models. By lifting NPHMs to a first-class reconstruction primitive, we enable more accurate geometry, richer expressions, and finer animation control.
Pix2NPHM obtains fast and reliable NPHM reconstructions on real-world data. Inference-time optimization against surface normals and canonical point maps can further increase fidelity.
Key to successful and generalized training of our ViT-based network are:
(1) large-scale registration of existing 3D head datasets, and
(2) self-supervised training on vast in-the-wild 2D video datasets using pseudo ground-truth surface normals.
Finally, we show that geometry-aware pretraining on pixel-aligned reconstruction tasks significantly outperforms generic visual pretraining (e.g., DINO-style features) in terms of generalization.
๐simongiebenhain.github.io/Piโฆ
๐ฅyoutu.be/MgpEJC5p1Ts
Great work by @SGiebenhain, @TobiasKirschst1, @liamschoneveld, Davide Davoli, Zhe Chen
Peter Kocsis retweeted
๐ข๐ข๐ข๐๐๐ฌ๐ก๐๐ข๐ฉ๐ฉ๐ฅ๐: Structured Autoregressive Generation of Artist-Meshes
High-fidelity, topologically complete 3D assets that expand naturally like a ripple on a surface! ๐
Existing AR models often rely on sliding-window inference over truncated segments. However, this limitation breaks long-range geometric dependencies, causing holes and fragmentation.
Instead, MeshRipple uses frontier-aware BFS and sparse-attention global memory to ensure coherent growth with an unbounded receptive field.
-> Highly detailed-mesh generations
-> Artist-like meshing quality
-> Works on room-scale environments
๐maymhappy.github.io/MeshRippโฆ
๐ฅyoutu.be/AHvaLslzXQU
Great work by Junkai Lin, Hang Long, Huipeng Guo, Jielei Zhang, Jiayi Yang, Tianle Guo, Yang Yang, Jianwen Li, Wenxiao Zhang, Wei Yang
๐ข Intrinsic Image Fusion for Multi-View 3D Material Reconstruction
We reconstruct clean and sharp relightable textures using inverse path tracing and monocular priors.
Check out our project page for more results:
peter-kocsis.github.io/Intriโฆ
๐ข Intrinsic Image Fusion for Multi-View 3D Material Reconstruction ๐ข
We combine generative material priors with inverse path tracing: 1) define a parametric texture space 2) fuse monocular predictions across views into consistent textures 3) optimize low-dimensional parameters for physically-grounded reconstructions.
The results are relightable PBR textures for 3D scenes: check out the result on a real-world 3D scan from the ScanNet++ dataset!
๐peter-kocsis.github.io/Intriโฆ
๐ฅyoutu.be/-Vs3tR1Xl7k
Great work by @Peter4AI @LukasHollein!
Peter Kocsis retweeted
Releasing Echo today is incredibly exciting for me โ because it is a critical step for generative AI, enabling the creation of virtual worlds.
Echo is our first world model at SpAItial AI. It turns text or images into explorable 3D environments โ spaces you can move through, inspect, and build on. Seeing this work in real time still feels a bit surreal.
My fascination with this goes back a long way: video games, virtual environments, and the idea of capturing the real world in 3D. As a researcher, I spent years working on 3D reconstruction, neural rendering, and scene understanding โ all driven by the same question: how do we teach machines to understand the world?
One thing became clear over time: the biggest bottleneck isnโt compute or rendering โ itโs 3D worlds themselves. High-quality, consistent environments are expensive to create by hand and donโt scale to the experiences we want to build. In particular, I believe that the ability to generate virtual worlds is ultimately key towards understanding the real world.
Thatโs why we founded SpAItial AI. Weโre building spatial world models that combine geometric understanding with creative generation โ models that can generate, edit, and eventually reason about 3D environments.
Echo is just the beginning. For me, this feels like the moment when decades of research finally meet the imagination that got many of us into graphics, games, 3D understanding in the first place.๐
spaitial.ai/
๐ Announcing Echo โ our new frontier model for 3D world generation.
Echo turns a simple text prompt or image into a fully explorable, 3D-consistent world. Instead of disconnected views, the result is a single, coherent spatial representation you can move through freely.
This is part of a bigger shift in AI: from generating pixels and tokens to generating spaces. Echo predicts a geometry-grounded 3D scene at metric scale, meaning every novel view, depth map, and interaction comes from the same underlying world โ not independent hallucinations.
Once generated, the world is interactive in real time. You control the camera, explore from any angle, and render instantly โ even on low-end hardware, directly in the browser. High-quality 3D world exploration is no longer gated by expensive equipment.
Under the hood, Echo infers a physically grounded 3D representation and converts it into a renderable format. For our web demo, we use 3D Gaussian Splatting (3DGS) for fast, GPU-friendly rendering โ but the representation itself is flexible and can be easily adapted.
Why this matters: consistent 3D worlds unlock real workflows โ digital twins, 3D design, game environments, robotics simulation, and more. From a single photo or a line of text, Echo builds worlds that are reliable, editable, and spatially faithful.
Echo also enables scene editing and restyling. Change materials, remove or add objects, explore design variations โ all while preserving global 3D consistency. Editing no longer breaks the world.
This is only the beginning. Echo is the foundation for future world models with dynamics, physical reasoning, and richer interaction โ environments that donโt just look right, but behave right.
Explore the generated worlds on our website and sign up for the closed beta. The era of spatial intelligence starts here. ๐
#Echo #WorldModels #SpatialAI #3DFoundationModels
Check it out: spaitial.ai/
Peter Kocsis retweeted
Radiance Meshes for Volumetric Reconstruction
๐ We made a thing!
A Radiance Field that is composited out of a triangle meshes that renders accurately the underlying volumetric field, with no popping AND faster than approximate methods like Gaussian Splatting.
๐ข ๐๐ฃ๐ฉ๐ง๐๐ฃ๐จ๐๐ @NeurIPSConf
If you are excited about PBR materials, drop by at Wednesday from 4:30pm to 7:30pm (poster 4308), or feel free to dm me.
PS: We have just released the training code, so you can also train your own models now!
peter-kocsis.github.io/Intriโฆ
๐ข IntrinsiX: High-Quality PBR Generation using Image Priors ๐ข
From text input, we generate renderable PBR maps! Next to editable image generation, our predictions can be distilled into room-scale scenes using SDS for large-scale PBR texture generation.
We first train separate LoRA modules for the intrinsic properties of albedo, rough/metal, normal. Then, we introduce cross-intrinsic attention using a rerendering loss with importance-weighted light sampling to enable coherent PBR generation.
Our method outperforms text -> image -> PBR methods both in generalization and quality, since directly generating PBR maps does not suffer from the inherent ambiguity of intrinsic image decomposition. In addition, our design choice facilitates SDS-based PBR texture distillation.
๐ peter-kocsis.github.io/Intriโฆ
๐ฅ youtu.be/b0wVA44R93Y
Great work by @Peter4AI, @LukasHollein
Peter Kocsis retweeted
FLUX.2 is here - our most capable image generation & editing model to date.
Multi-reference. 4MP. Production-ready. Open weights.
Into the new.
Congrats @yawarnihal! Amazing work, really well-deserved!
Congrats to @yawarnihal for winning the @MdsiTum best paper award for his amazing ๐๐๐ฌ๐ก๐๐๐ work๐
MeshGPT autoregressively generates compact, artist-style triangle meshes by tokenizing faces into a learned discrete vocabulary (VQ-style codebook) and training a decoder-only transformer to predict those face tokens โ because discrete tokenization + attention lets GPT-style models learn long-range geometric & topological patterns and produce coherent, high-fidelity 3D assets.
MeshGPT's use cases go far beyond traditional content creation applications in computer graphics. For instance, the method was developed in collaboration with @Audi to help rapid prototyping of car designs, where explicit and precise mesh design are essential.
In the research community, there have already been many follow ups such as MeshAnything, MeshXL, Meshtron, and many more - finally, we can use AI to generate high-fidelity 3D content :)
Project: nihalsid.github.io/mesh-gpt/
Video: youtu.be/UV90O1_69_o
Peter Kocsis retweeted
๐ขProcGen3D: Learning Neural Procedural Graphs for Image-to-3D Reconstruction
@xinyi092298 learns neural procedural graphs to generate high-fidelity 3D - MCTS-guided sampling maintains consistency with the input image, even from real images!
Check it out: xzhang-t.github.io/project/Pโฆ
Peter Kocsis retweeted
๐ข๐ข ๐๐๐ซ๐๐๐๐๐: ๐๐๐ซ๐๐๐ฉ๐ญ๐ฎ๐๐ฅ ๐๐๐๐ ๐๐จ๐๐๐ฅ ๐๐จ๐ซ ๐๐ข๐ง๐ ๐ฅ๐-๐๐ฆ๐๐ ๐ ๐๐ ๐๐๐๐ ๐๐๐๐จ๐ง๐ฌ๐ญ๐ซ๐ฎ๐๐ญ๐ข๐จ๐ง & ๐๐๐ข๐ญ๐ข๐ง๐ ๐ข๐ข
PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text.
At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS.
Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output.
In addition, our architecture seamlessly adapts to downstream tasks: by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt. We also provide an interactive GUI to enable the exploration of our editing pipeline.
๐ antoniooroz.github.io/PercHeโฆ
๐ฝ๏ธ youtu.be/4hFybgTk4kE
Great work by @antonio_oroz and @TobiasKirschst1
Peter Kocsis retweeted
๐ขNew in ScanNet++: High-Res 360ยฐ Panos!
@chandan__yes & @liuyuehcheng have added pano captures for 956 ScanNet++ scenes, fully aligned with the 3D meshes, DSLR, and iPhone data - multiple panos per scene
Check it out:
Docs kaldir.vc.in.tum.de/scannetpโฆ
Code github.com/scannetpp/scannetโฆ













