It was a pleasure working with @howard_xhc and the team. Check out the cool results we got on integrating powerful perception models into active imaging via foveation.
The era of ultra-high-resolution imaging has arrived. Modern image sensors exceeding 200 MP resolution are common in smartphones, with over 400 MP sensors under development. However, the large number of pixels poses significant challenges for acquisition and processing, especially on edge devices.
Which pixels should be acquired, and when, for bandwidth-efficient imaging and perception?
We introduce Policy-based Foveated Imaging and Perception, an on-device, real-time, predictive, and task-aware framework that dynamically allocates sensor resolution to prioritize important regions under specific perception objectives. This paper will be presented at #SIGGRAPH2026!
[1/6]
Boyang Deng retweeted
The era of ultra-high-resolution imaging has arrived. Modern image sensors exceeding 200 MP resolution are common in smartphones, with over 400 MP sensors under development. However, the large number of pixels poses significant challenges for acquisition and processing, especially on edge devices.
Which pixels should be acquired, and when, for bandwidth-efficient imaging and perception?
We introduce Policy-based Foveated Imaging and Perception, an on-device, real-time, predictive, and task-aware framework that dynamically allocates sensor resolution to prioritize important regions under specific perception objectives. This paper will be presented at #SIGGRAPH2026!
[1/6]
Boyang Deng retweeted
📢 Stop by our #CVPR2026 workshop next week: GeoFreeNVS: Geometry-Free Novel View Synthesis and Controllable Video Models
geofreenvs.github.io/
Especially excited to discuss how large generative models can solve 3D tasks like novel view synthesis!
Boyang Deng retweeted
New paper: AsymFlow🔥
JiT x0-prediction is not enough for pixel generation. Better keep velocity in a low-rank subspace:
- 1.57 FID on ImageNet (best pixel flow model)
- Finetunes FLUX.2 klein into pixel space, beats the original on HPSv3/DPG/GenEval (#1 overall on HPSv3)
1/7
Boyang Deng retweeted
Yay, finally! Introducing Vision Banana🍌 from @GoogleDeepMind, our unified model that outperforms SoTA specialist models on various vision tasks!
By treating 2D/3D vision tasks as image generation, we unlock a new foundation for CV.
Project page: vision-banana.github.io
(1/5)
Boyang Deng retweeted
Introducing CityRAG!
We wanted video generative models to be grounded in the real world — if I’m in London, I want to look around and actually see Big Ben.
CityRAG generates videos of cities featuring real buildings and roads, with arbitrary weather, people, and cars. 1/N
page: cityrag.github.io
paper: arxiv.org/abs/2604.19741
Boyang Deng retweeted
Thrilled to announced that at #ICCV2025 we will host the first workshop on 𝐆𝐞𝐨𝐦𝐞𝐭𝐫𝐲-𝐅𝐫𝐞𝐞 𝐍𝐨𝐯𝐞𝐥 𝐕𝐢𝐞𝐰 𝐒𝐲𝐧𝐭𝐡𝐞𝐬𝐢𝐬 𝐚𝐧𝐝 𝐂𝐨𝐧𝐭𝐫𝐨𝐥𝐥𝐚𝐛𝐥𝐞 𝐕𝐢𝐝𝐞𝐨 𝐌𝐨𝐝𝐞𝐥𝐬
geofreenvs.github.io
a.k.a. "3D Computer Vision in the era of Video Models" 😅
Curious about how cities have changed in the past decade? We use MLLMs to analyse 40 million Street View images to answer this. Do you know that "juice shops became a thing in NYC" and "miles of overpasses were painted BLUE in SF"? More at→boyangdeng.com/visual-chroni… (vid ↓ w/ 🔊)
Work done w/ @songyoupeng, Kyle Genova, @GordonWetzstein, @Jimantha, Leonidas Guibas, and Thomas Funkhouser. This is a collaboration between @StanfordAILab and @GoogleDeepMind.
Boyang Deng retweeted
Introduce CoT-VLA – Visual Chain-of-Thought reasoning for Robot Foundation Models! 🤖
By leveraging next-frame prediction as visual chain-of-thought reasoning, CoT-VLA uses future prediction to guide action generation and unlock large-scale video data for training. #CVPR2025
Boyang Deng retweeted
Introducing 👀Stereo4D👀
A method for mining 4D from internet stereo videos. It enables large-scale, high-quality, dynamic, *metric* 3D reconstructions, with camera poses and long-term 3D motion trajectories.
We used Stereo4D to make a dataset of over 100k real-world 4D scenes.
Boyang Deng retweeted
🚀 Introducing CAT4D! 🚀
CAT4D transforms any real or generated video into dynamic 3D scenes with a multi-view video diffusion model.
The outputs are dynamic 3D models that we can freeze and look at from novel viewpoints, in real-time!
Be sure to try our interactive viewer!
Boyang Deng retweeted
Symmetries are everywhere — from butterfly’s wings to Greek temples. But detecting them in noisy data? That’s a challenge. 🦋🏛
Our #SIGGRAPHAsia2024 paper, Robust Symmetry Detection via Riemannian Langevin Dynamics, tackles this: symmetry-langevin.github.io/
🧵(1/n)
Thought about generating realistic 3D urban neighbourhoods from maps, dawn to dusk, rain or shine? Putting heavy snow on the streets of Barcelona? Or making Paris look like NYC? We built a Streetscapes system that does all these. See boyangdeng.com/streetscapes. (Showreel w/ 🔊 ↓)
Music in video: grandbrothers.bandcamp.com/t… (please support the artist at bandcamp).
Meet us at SIGGRAPH 2024 (2pm July 29 at Mile High 1A). Arxiv: arxiv.org/abs/2407.13759. HF: huggingface.co/papers/2407.1….
Boyang Deng retweeted
Introducing Proteus 0.1, REAL-TIME video generation that brings life to your AI.
Proteus can laugh, rap, sing, blink, smile, talk, and more. From a single image!
Come meet Proteus on Twitch in real-time. ↓
Sign up for API waitlist: apparate.ai/early-access.htm…
1/11