Jason Devitt

Follow me so I can DM you his email.

Part of the genius of that paper is that it doesn’t require the NeRF geometry to be accurate to train stereo; it only requires accurate view synthesis.

I'm at CES giving demos of @compoundeyeinc real-time 3D perception tech. We leverage automotive-grade cameras for on road and off road perception tasks. Find me at booth #61401 in Tech West at the Venetian Hotel Expo January 7th and January 8th. #CES2023 #AutonomousVehicles

Yes, but the diffusion model alone is used to inpaint the occlusion zones that are created when the frame is warped according to the depth map.

This beautiful work by @cut_pow suggests diffusion models have at minimum learned to reproduce static monocular depth cues. Not clear if that's the same as learning 3D geometry. But if not, MiDaS has no 3D knowledge either; it uses the same cues. FYI @jon_barron

OK. I am really surprised that diffusion models usually produce images that follow rules of perspective. In some hand-wavey sense StableDiffusion has learned more about 3D geometry than it has about … hands.

This is amazing. Diffusion networks clearly learn 3D representations of many object classes, else they could only generate trivial 2D views. Is it fair to say that your machinery coaxes the diffusion network to generate many mutually consistent views of a single scene?

#stablediffusion "a swarm of robots in a meadow, cyberpunk"

#stablediffusion is remarkable. The failure cases are as interesting as the successes. Prompt "photograph of a young woman looking through a telescope, professional" @karpathy

Congratulations @kvogt !

The term "self-driving" is problematic but this is a really good summary of what's available today.

You appear to be claiming that (say) highway-only L2 is impossible without lidar and "imaging" radar, or that it is useless. Many people thought so 10 years ago but it is a very lonely claim now.

Agreed, but snow-covered is very different than “whiteout”. @comma_ai does well on snow-covered streets with a single camera (forward facing)

The state of the art for commercially deployed systems I should say. It is possible to drive in a wide range of degraded conditions with passive sensors, but you specifically asked about a whiteout.

If you have a solution I am sure they and we would be interested. The state of the art for fully autonomous vehicles with all available sensors is driving in light rain.

You cannot drive safely on conventional roads in a true whiteout with any combination of sensors.

I don't see how you can get that out of the interview.

The podcast was about primary sensors for ADAS, so I’m assuming you didn’t listen to it.

There are lots of arguments for adding radar to a car, but since radar cannot read signs or see lane markings it cannot be the primary sensor on a vehicle. We just didn’t get to it.

Thanks for having us! We really enjoyed the conversation. @TheAutonocast