Peter Kocsis retweeted
Congrats to @LukasHollein for his successful PhD defense!
Lukasโs thesis is on ๐๐๐ง๐๐ซ๐๐ญ๐ข๐ง๐ ๐๐ ๐๐จ๐ซ๐ฅ๐๐ฌ ๐๐ซ๐จ๐ฆ ๐๐ ๐๐๐ง๐๐ซ๐๐ญ๐ข๐ฏ๐ ๐๐จ๐๐๐ฅ๐ฌ. His most impactful work so far, Text2Room (ICCVโ23 Oral), was the first method to generate complete room-scale 3D scenes with textured geometry from text alone.
The key idea is to lift pre-trained 2D text-to-image models into 3D: iteratively generate new views, estimate depth, and fuse everything into a seamless textured mesh, with a carefully designed viewpoint selection to complete the scene. This inspired many others on text-to-3D scene generation and led to a long line of follow-up works, including Ctrl-Room, LucidDreamer, WonderJourney, WorldGen, and many more. Today, the Text2Room paradigm of leveraging 2D priors has become the primary approach for both image and video diffusion models to generate 3D scenes.
However, Lukas has done much more cutting-edge research. ViewDiff turns text-to-image models into 3D-consistent multi-view generators, WorldExplorer generates fully navigable 3D scenes from video diffusion models, and 3DGS-LM speeds up Gaussian Splatting optimization with Levenberg-Marquardt, and most recently World Reconstruction From Inconsistent Views. In summary, his PhD has helped define how we generate 3D worlds with generative AI.
Lukas also keeps pushing the boundaries of 3D world generation. Heโs now continuing his research at Apple and I'm sure we can expect more great research very soon!
Check his great research: lukashoel.github.io
Congrats Dr. Hรถllein, you will be deeply missed in the lab!
Peter Kocsis retweeted
๐ข 3D world models from video diffusion suffer from inconsistent frames -> blurry output.
Our fix: instead of naรฏve 3D reconstruction, we non-rigidly align each frame into a globally-consistent 3DGS representation.
->sharp visuals on top of any VDM!
lukashoel.github.io/video_toโฆ
Peter Kocsis retweeted
๐ข๐ข๐ขData release: high-res, multi-view, OLAT face recordings ๐ข๐ข๐ข
We captured individuals in our custom light stage with 16 high-end, global shutter cameras (72 fps) and 40 LED modules, totaling 2.8M precisely calibrated frames.
We us the data for BecomingLit (#NeurIPS2025): intrinsically decomposed Gaussian avatars, enabling photorealistic and real-time relighting via hybrid neural shading.
Code & Data: jonathsch.github.io/becomingโฆ
Great work by @jnthnschmdt, @SGiebenhain
Peter Kocsis retweeted
๐ขPix2NPHM: Learning to Regress NPHM Reconstructions From a Single Image๐ข
We directly regress neural parametric head models (NPHMs) from a single image โ fast, stable, and significantly more expressive than classical 3DMMs such as FLAME.
Face tracking & 3D reconstruction are often limited by the representational capacity of PCA-based face models. By lifting NPHMs to a first-class reconstruction primitive, we enable more accurate geometry, richer expressions, and finer animation control.
Pix2NPHM obtains fast and reliable NPHM reconstructions on real-world data. Inference-time optimization against surface normals and canonical point maps can further increase fidelity.
Key to successful and generalized training of our ViT-based network are:
(1) large-scale registration of existing 3D head datasets, and
(2) self-supervised training on vast in-the-wild 2D video datasets using pseudo ground-truth surface normals.
Finally, we show that geometry-aware pretraining on pixel-aligned reconstruction tasks significantly outperforms generic visual pretraining (e.g., DINO-style features) in terms of generalization.
๐simongiebenhain.github.io/Piโฆ
๐ฅyoutu.be/MgpEJC5p1Ts
Great work by @SGiebenhain, @TobiasKirschst1, @liamschoneveld, Davide Davoli, Zhe Chen
Peter Kocsis retweeted
๐ข๐ข๐ข๐๐๐ฌ๐ก๐๐ข๐ฉ๐ฉ๐ฅ๐: Structured Autoregressive Generation of Artist-Meshes
High-fidelity, topologically complete 3D assets that expand naturally like a ripple on a surface! ๐
Existing AR models often rely on sliding-window inference over truncated segments. However, this limitation breaks long-range geometric dependencies, causing holes and fragmentation.
Instead, MeshRipple uses frontier-aware BFS and sparse-attention global memory to ensure coherent growth with an unbounded receptive field.
-> Highly detailed-mesh generations
-> Artist-like meshing quality
-> Works on room-scale environments
๐maymhappy.github.io/MeshRippโฆ
๐ฅyoutu.be/AHvaLslzXQU
Great work by Junkai Lin, Hang Long, Huipeng Guo, Jielei Zhang, Jiayi Yang, Tianle Guo, Yang Yang, Jianwen Li, Wenxiao Zhang, Wei Yang
Peter Kocsis retweeted
Releasing Echo today is incredibly exciting for me โ because it is a critical step for generative AI, enabling the creation of virtual worlds.
Echo is our first world model at SpAItial AI. It turns text or images into explorable 3D environments โ spaces you can move through, inspect, and build on. Seeing this work in real time still feels a bit surreal.
My fascination with this goes back a long way: video games, virtual environments, and the idea of capturing the real world in 3D. As a researcher, I spent years working on 3D reconstruction, neural rendering, and scene understanding โ all driven by the same question: how do we teach machines to understand the world?
One thing became clear over time: the biggest bottleneck isnโt compute or rendering โ itโs 3D worlds themselves. High-quality, consistent environments are expensive to create by hand and donโt scale to the experiences we want to build. In particular, I believe that the ability to generate virtual worlds is ultimately key towards understanding the real world.
Thatโs why we founded SpAItial AI. Weโre building spatial world models that combine geometric understanding with creative generation โ models that can generate, edit, and eventually reason about 3D environments.
Echo is just the beginning. For me, this feels like the moment when decades of research finally meet the imagination that got many of us into graphics, games, 3D understanding in the first place.๐
spaitial.ai/
๐ Announcing Echo โ our new frontier model for 3D world generation.
Echo turns a simple text prompt or image into a fully explorable, 3D-consistent world. Instead of disconnected views, the result is a single, coherent spatial representation you can move through freely.
This is part of a bigger shift in AI: from generating pixels and tokens to generating spaces. Echo predicts a geometry-grounded 3D scene at metric scale, meaning every novel view, depth map, and interaction comes from the same underlying world โ not independent hallucinations.
Once generated, the world is interactive in real time. You control the camera, explore from any angle, and render instantly โ even on low-end hardware, directly in the browser. High-quality 3D world exploration is no longer gated by expensive equipment.
Under the hood, Echo infers a physically grounded 3D representation and converts it into a renderable format. For our web demo, we use 3D Gaussian Splatting (3DGS) for fast, GPU-friendly rendering โ but the representation itself is flexible and can be easily adapted.
Why this matters: consistent 3D worlds unlock real workflows โ digital twins, 3D design, game environments, robotics simulation, and more. From a single photo or a line of text, Echo builds worlds that are reliable, editable, and spatially faithful.
Echo also enables scene editing and restyling. Change materials, remove or add objects, explore design variations โ all while preserving global 3D consistency. Editing no longer breaks the world.
This is only the beginning. Echo is the foundation for future world models with dynamics, physical reasoning, and richer interaction โ environments that donโt just look right, but behave right.
Explore the generated worlds on our website and sign up for the closed beta. The era of spatial intelligence starts here. ๐
#Echo #WorldModels #SpatialAI #3DFoundationModels
Check it out: spaitial.ai/
Peter Kocsis retweeted
Radiance Meshes for Volumetric Reconstruction
๐ We made a thing!
A Radiance Field that is composited out of a triangle meshes that renders accurately the underlying volumetric field, with no popping AND faster than approximate methods like Gaussian Splatting.
Peter Kocsis retweeted
FLUX.2 is here - our most capable image generation & editing model to date.
Multi-reference. 4MP. Production-ready. Open weights.
Into the new.
Peter Kocsis retweeted
๐ขProcGen3D: Learning Neural Procedural Graphs for Image-to-3D Reconstruction
@xinyi092298 learns neural procedural graphs to generate high-fidelity 3D - MCTS-guided sampling maintains consistency with the input image, even from real images!
Check it out: xzhang-t.github.io/project/Pโฆ
Peter Kocsis retweeted
๐ข๐ข ๐๐๐ซ๐๐๐๐๐: ๐๐๐ซ๐๐๐ฉ๐ญ๐ฎ๐๐ฅ ๐๐๐๐ ๐๐จ๐๐๐ฅ ๐๐จ๐ซ ๐๐ข๐ง๐ ๐ฅ๐-๐๐ฆ๐๐ ๐ ๐๐ ๐๐๐๐ ๐๐๐๐จ๐ง๐ฌ๐ญ๐ซ๐ฎ๐๐ญ๐ข๐จ๐ง & ๐๐๐ข๐ญ๐ข๐ง๐ ๐ข๐ข
PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text.
At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS.
Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output.
In addition, our architecture seamlessly adapts to downstream tasks: by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt. We also provide an interactive GUI to enable the exploration of our editing pipeline.
๐ antoniooroz.github.io/PercHeโฆ
๐ฝ๏ธ youtu.be/4hFybgTk4kE
Great work by @antonio_oroz and @TobiasKirschst1
Peter Kocsis retweeted
๐ขNew in ScanNet++: High-Res 360ยฐ Panos!
@chandan__yes & @liuyuehcheng have added pano captures for 956 ScanNet++ scenes, fully aligned with the 3D meshes, DSLR, and iPhone data - multiple panos per scene
Check it out:
Docs kaldir.vc.in.tum.de/scannetpโฆ
Code github.com/scannetpp/scannetโฆ
Peter Kocsis retweeted
Fantastic retreat this weekend by our research groups!
Internal reviews, ideas brainstorming, paper reading, and much more! Of course also many social activities -- the highlight being our kayaking trip - lots of fun :)
Peter Kocsis retweeted
So You Want to Be an Academic? A couple of years into your PhD, but wondering: "Am I doing this right?" Most of the advice is aimed at graduating students. But there's far less for junior folks who are still finding their academic path.
My candid takes: anandbhattad.github.io/blogsโฆ
Peter Kocsis retweeted
Can we use video diffusion to generate 3D scenes?
๐๐จ๐ซ๐ฅ๐๐๐ฑ๐ฉ๐ฅ๐จ๐ซ๐๐ซ (#SIGGRAPHAsia25) creates fully-navigable scenes via autoregressive video generation.
Text input -> 3DGS scene output & interactive rendering!
๐mschneider456.github.io/worlโฆ
๐ฝ๏ธyoutu.be/N6NJsNyiv6I
Peter Kocsis retweeted
HairGS: Hair Strand Reconstruction based on 3D Gaussian Splatting (#BMVC2025) ๐ข
We reconstruct realistic 3D hair strands from multi-view images in ~1 hour. Results show robustness to a wide range of hair styles such as challenging curly hair.
Project: yimin-pan.github.io/hair-gs/
Peter Kocsis retweeted
๐ขCode release for 3DGS-LM (#ICCV2025)๐ข
Gaussian-Splatting made faster by changing the underlying optimizer. It's compatible with other changes that accelerate the optimization along different axes.
lukashoel.github.io/3DGS-LM/
(1/2)
How to accelerate the reconstruction of 3D Gaussian Splatting?
3DGS-LM replaces the commonly used ADAM optimizer with a tailored Levenberg-Marquardt (LM).
=> We are ๐๐% ๐๐๐ฌ๐ญ๐๐ซ ๐ญ๐ก๐๐ง ๐๐๐๐ for the same quality.
lukashoel.github.io/3DGS-LM/
youtu.be/tDiGuGMssg8
Peter Kocsis retweeted
ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions (#SIGGRAPH)
We reconstruct ultra-high fidelity photorealistic 3D avatars capable of generating realistic and high-quality animations including freckles and other fine facial details.
We operate on patch-based local expression features and increase the representation capacity by synthesizing 3D Gaussians dynamically by leveraging tiny scaffold MLPs conditioned on localized expressions.
We further propose a color-based densification and progressive training scheme for improved quality and faster convergence.
Project: shivangi-aneja.github.io/proโฆ
Video: youtu.be/VyWkgsGdbkk
Great work by @shivangi2201, Sebastian Weiss, Irene Baeza, Prashanth Chandran, Gaspard Zoss, Derek Bradley
Peter Kocsis retweeted
๐ข LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans๐ โจ
-> converts RGB-D scans into compact, realistic, and interactive 3D scenes โ featuring high-quality meshes, PBR materials, and articulated objects.
๐ทyoutu.be/ecK9m3LXg2c
๐litereality.github.io/
Peter Kocsis retweeted
Check out our #ICCV2025 work on functional 3d scan editing, learning to optimize, multi-level 3d captioning, interactive mesh editing, audio-driven avatars, & shape matching!
Congrats @ElBoudjogh24002, @liuyuehcheng, @chandan__yes, @hcxrli, @shivangi2201, Emery for amazing work!
Peter Kocsis retweeted
Presenting PrEditor3D at #CVPR2025 ๐ข๐ข
If you'd like to learn more about our work and discuss 3D generation/editing, come visit our poster on Friday, June 13th, in ExHall D between 10:30-12:30 (Poster #44).
Project Page: ziyaerkoc.com/preditor3d



















