Try / Share / Iterate — Creative Technologist / Chief of Innovation @bonjourdpt
Montreal, QC, Canada
Joined February 2009
- Tweets5.1K
- Following1.6K
- Followers8.7K
- Likes5.9K
Pinned Tweet
Mixed Reality + Diffusion prototype as a tool for exploring concepts, styles, and moods by transforming real-world surroundings into alternate realities.
#MixedReality #MR #AI #StableDiffusion #Quest3
Hugues Bruyère retweeted
Vitre Experiment #4.
Winter is coming, so I brought a bit of spring into the hallway by adding synthetic 3DGS assets to the scene.
Still running smoothly.
#3DGS #GaussianSplatting #ComputerVision
Vitre Experiment #4.
Winter is coming, so I brought a bit of spring into the hallway by adding synthetic 3DGS assets to the scene.
Still running smoothly.
#3DGS #GaussianSplatting #ComputerVision
Hugues Bruyère retweeted
Finally got around to trying Spirula Studio, and I'm really impressed!
It simplifies the process of preparing datasets from 360° footage enormously.
Trained this 3D Gaussian Splatting scene from footage I captured back in 2022 with an Insta360 ONE RS 1-Inch 360.
#3DGS #GaussianSplatting #SpirulaStudio
Finally got around to trying Spirula Studio, and I'm really impressed!
It simplifies the process of preparing datasets from 360° footage enormously.
Trained this 3D Gaussian Splatting scene from footage I captured back in 2022 with an Insta360 ONE RS 1-Inch 360.
#3DGS #GaussianSplatting #SpirulaStudio
Hugues Bruyère retweeted
I added a ToF sensor to the equation... an Femto Bolt.
Both the captured Gaussian Splat and the Femto Bolt depth data are registered to the physical space and rendered with the same off-axis projection… so now people passing through the hallway are captured and inserted into the 3DGS scene.
Still rough around the edges… but I like it.
This time I needed some help with the demo. Someone had to be the tracked POV while I strangely walked around the hallway :)
#3DGS #GaussianSplatting #ComputerVision
Hugues Bruyère retweeted
anyone who was around for the early 2010s kinect scene out was dreaming about this sorta thing. we just didn't have the fidelity and framerate yet. sooo satisfying to see this for real finally.
I added a ToF sensor to the equation... an Femto Bolt.
Both the captured Gaussian Splat and the Femto Bolt depth data are registered to the physical space and rendered with the same off-axis projection… so now people passing through the hallway are captured and inserted into the 3DGS scene.
Still rough around the edges… but I like it.
This time I needed some help with the demo. Someone had to be the tracked POV while I strangely walked around the hallway :)
#3DGS #GaussianSplatting #ComputerVision
I added a ToF sensor to the equation... an Femto Bolt.
Both the captured Gaussian Splat and the Femto Bolt depth data are registered to the physical space and rendered with the same off-axis projection… so now people passing through the hallway are captured and inserted into the 3DGS scene.
Still rough around the edges… but I like it.
This time I needed some help with the demo. Someone had to be the tracked POV while I strangely walked around the hallway :)
#3DGS #GaussianSplatting #ComputerVision
Hugues Bruyère retweeted
This time the tracking runs with a 120 fps global-shutter camera.
Getting twice as many position updates, with much fewer gaps when tracking faster movements… It feels much better.
The window is still rendering from a position roughly 20–24 ms in the past, but I compensate for some of that by predicting the position slightly forward.
It still works best when tracking the phone, though.
#3DGS #GaussianSplatting #ComputerVision
This time the tracking runs with a 120 fps global-shutter camera.
Getting twice as many position updates, with much fewer gaps when tracking faster movements… It feels much better.
The window is still rendering from a position roughly 20–24 ms in the past, but I compensate for some of that by predicting the position slightly forward.
It still works best when tracking the phone, though.
#3DGS #GaussianSplatting #ComputerVision
When I need a break from my screen, I make it disappear.
Vitre, experiment #2.
This time I’m using one of the monitors on my desk. It happens to sit right in front of a glass wall looking into the hallway and the rest of the office, which makes for a much more interesting view.
Press the button and Vitre takes over the monitor, rendering a 3D Gaussian Splat I previously captured of the space behind it.
The 3DGS is registered to the physical space, and the view is rendered in real time from the tracked viewer’s position... making the monitor vanish.
I’ve been iterating on the tracking and motion-to-photon latency since the first experiment. Still a few things I want to explore, but the illusion is getting better.
#3DGS #GaussianSplatting #ComputerVision
Hugues Bruyère retweeted
When I need a break from my screen, I make it disappear.
Vitre, experiment #2.
This time I’m using one of the monitors on my desk. It happens to sit right in front of a glass wall looking into the hallway and the rest of the office, which makes for a much more interesting view.
Press the button and Vitre takes over the monitor, rendering a 3D Gaussian Splat I previously captured of the space behind it.
The 3DGS is registered to the physical space, and the view is rendered in real time from the tracked viewer’s position... making the monitor vanish.
I’ve been iterating on the tracking and motion-to-photon latency since the first experiment. Still a few things I want to explore, but the illusion is getting better.
#3DGS #GaussianSplatting #ComputerVision
Hugues Bruyère retweeted
constantly watching @smallfly for gaussian splat inspiration. he’s always been a step ahead on interaction design for more than a decade.
A few months ago, @kcimc sent me a message with a simple idea: capture my workspace as a Gaussian Splat, put a monitor back into that same space, track the viewer and make the monitor disappear.
It’s been on my to-do list ever since... I finally built a POC…
Working name: Vitre. It turns a screen into a window onto a 3D capture of the space it is actually in.
My office space was captured as a 3D Gaussian Splat and registered back into the physical space. A camera tracks the viewer’s position, and the scene is rendered from that point of view in real time. Move around the screen and the perspective changes with you, as if you were looking through a window.
Still a POC. The calibration of the whole system needs to be more precise, and there’s still some noticeable latency between the tracking and the render.
#3DGS #GaussianSplatting #ComputerVision
Hugues Bruyère retweeted
A few months ago, @kcimc sent me a message with a simple idea: capture my workspace as a Gaussian Splat, put a monitor back into that same space, track the viewer and make the monitor disappear.
It’s been on my to-do list ever since... I finally built a POC…
Working name: Vitre. It turns a screen into a window onto a 3D capture of the space it is actually in.
My office space was captured as a 3D Gaussian Splat and registered back into the physical space. A camera tracks the viewer’s position, and the scene is rendered from that point of view in real time. Move around the screen and the perspective changes with you, as if you were looking through a window.
Still a POC. The calibration of the whole system needs to be more precise, and there’s still some noticeable latency between the tracking and the render.
#3DGS #GaussianSplatting #ComputerVision
A few months ago, @kcimc sent me a message with a simple idea: capture my workspace as a Gaussian Splat, put a monitor back into that same space, track the viewer and make the monitor disappear.
It’s been on my to-do list ever since... I finally built a POC…
Working name: Vitre. It turns a screen into a window onto a 3D capture of the space it is actually in.
My office space was captured as a 3D Gaussian Splat and registered back into the physical space. A camera tracks the viewer’s position, and the scene is rendered from that point of view in real time. Move around the screen and the perspective changes with you, as if you were looking through a window.
Still a POC. The calibration of the whole system needs to be more precise, and there’s still some noticeable latency between the tracking and the render.
#3DGS #GaussianSplatting #ComputerVision
Hugues Bruyère retweeted
Quick look at The Great Eye, our in-house tool for handling and merging sensor data to make spaces interactive.
Here it’s running with 6 Orbbec Femto Mega cameras and 3 Hokuyo 2D LiDARs, all feeding into the same system.
We’ve been developing and evolving this tool for a while now, mostly as the sensor layer behind our interactive installations.
Built with @openframeworks
#openFrameworks #ComputerVision #LiDAR #DepthCamera
Quick look at The Great Eye, our in-house tool for handling and merging sensor data to make spaces interactive.
Here it’s running with 6 Orbbec Femto Mega cameras and 3 Hokuyo 2D LiDARs, all feeding into the same system.
We’ve been developing and evolving this tool for a while now, mostly as the sensor layer behind our interactive installations.
Built with @openframeworks
#openFrameworks #ComputerVision #LiDAR #DepthCamera