Say less, do more. All in on immersive
Toronto
Joined December 2014
- Tweets4.4K
- Following1.6K
- Followers677
- Likes19.4K
Pinned Tweet
A request to allow 3rd Party apps to use Visual Intelligence to create experiences like the "Will this fit in my bag?" Siri AI demo. Please show this post some love if you agree that this is important! #WWDC26 developer.apple.com/forums/t…
Mike Dopsa retweeted
Today I am open-sourcing my custom SwiftUI animation sequencing package that I developed to drive the animations in Letters. It can do all sorts of things!
github.com/jazzychad/Animati…
Half Life Alyx, M2 Apple Vision Pro, PSVR2 controllers. No streaming.
I’ve never seen this game look this good!
Not my project, you see the setup info here! reddit.com/r/VisionPro/comme…
Mike Dopsa retweeted
📢OneCanvas (NeurIPS'26 Spotlight) Code Release📢
What if a 2D VLM could see the whole scene in one view?
We reproject features from every frame into a single occlusion-free canvas to make a stock VLM reason in 3D!
🔗 github.com/baranowskibrt/one…
🌐 baranowskibrt.github.io/onec…
📢 OneCanvas: 3D Scene Understanding via Panoramic Reprojection
We extract features from video frames and reproject them into one occlusion-free view of the whole scene that a 2D VLM reads just like a normal image. We can center this view on any viewpoint, including an agent's own pose for situated reasoning.
The same projection lets us create spatial training tasks with no human annotation, solvable only by reasoning over the 3D positions of real object features placed on an otherwise empty canvas.
The result is a stock 2D VLM that reasons in 3D, setting a new state of the art across spatial benchmarks at far less compute.
🌐 baranowskibrt.github.io/onec…
▶️ youtu.be/NIaHLB9gA7s
Great work by @baranowskibrt & @davech2y
Justin is awesome.
If you are looking for XR or dev support, he is very seasoned, kind and hard working. Hit him up!
Mike Dopsa retweeted
🦏 RHINO: Reconstructing Human Interactions with Novel Objects from Monocular Videos
From one moving-camera RGB video, we reconstruct a 3D human, an unseen manipulated 3D object, and the static 3D scene in one world frame
Video input, then 3D output. 👉 Code and data are public!
Mike Dopsa retweeted
Ok this is pretty insane. Astra side-loaded a custom app for my DJI RC2 controller that takes a live video stream from my Mini 5 Pro drone, streams it to a @roboflow Workflow running RF-DETR via our Serverless Video API, & overlays the predictions back on the controller’s screen.
Mike Dopsa retweeted
Preview: LichtFeld is slowly pivoting towards an end-to-end pipeline.
Where traditional SfM pipelines fail, our SLAM-based dense reconstruction runs in real time at 30 FPS and 1920 px.
Four minutes of capture condensed into one.
Dense point cloud and camera poses from a single monocular walk with a DJI Osmo Action 4.
Mike Dopsa retweeted
Apple is so freaking weird. They just casually drop 3D authoring into the preview app on macOS. I can sign annoying PDFs and render a scene with ray tracing for 3D graphics in the same app.
Location aware VPS in a Vision Pro would be incredible. I hope there are a few more goodies in the release visionOS 27
Dug deep into visionOS 27 Release Candidate
There is a very WIP “VisualLocalization” system that lets VisionOS devices determine exactly where it is outdoors by recognizing the surrounding buildings and storefronts
1. visionOS device gets a rough location from Wi‑Fi/Bluetooth
2. Its cameras observe nearby buildings, signs, and visual landmarks
3. VisualLocalization compares those images against Apple Maps’ prebuilt visual-location tiles
4. It returns a more precise position, direction, and confidence score
Functionality that this could support (speculation based on datamine):
- Correcting the Maps blue dot when the rough location is inaccurate
- Determining which direction you’re facing.
- Recognizing a particular storefront or nearby Maps place
- Future headset/glasses-based walking directions or location-aware AR
- Visual Intelligence experiences such as looking at a business and identifying it
This is all pretty notable because current generation visionOS devices are clearly not designated for outdoor use (even if you can use it that way)
There is also a paired WIP “AONSense” that appears to be the lower-power background system that could automatically wake this process whenever an app requests very accurate location. However, that connection appears even more unfinished/disabled on the current VisionOS builds
Mike Dopsa retweeted
THIS IS WHAT ALL VR HEADSETS SHOULD DO TO ALL YOUR FLAT GAMES
Holy shit IT WORKS!! One shotted with GPT6 Astra
I now have a fork of UEVR where I can play any UE game within a volumetric window!
This actually renders the game in 3D - meaning, two real game cameras, not some depth map bullshit. Everything looks right
This is an alternative (and more comfortable IMO) way to play most 2D->VR hacked games as there’s no motion sickness, and no need to redesign the UI and interaction systems
Needs a lot more polish before I publish it but OMG IT WORKS!
This is how I want to play every 3D game made to date. The art and world is already dimensional, we’ve just been printing it onto a flat picture this whole time!
We should see into it like a perfect, magic window and play with regular controllers.
Holy shit IT WORKS!! One shotted with GPT6 Astra
I now have a fork of UEVR where I can play any UE game within a volumetric window!
This actually renders the game in 3D - meaning, two real game cameras, not some depth map bullshit. Everything looks right
This is an alternative (and more comfortable IMO) way to play most 2D->VR hacked games as there’s no motion sickness, and no need to redesign the UI and interaction systems
Needs a lot more polish before I publish it but OMG IT WORKS!
Mike Dopsa retweeted
For robotic simulation, Atlas reconstructs a space from just a few photos and generates the photorealistic RGB and depth data any robot's sensors would observe on any trajectory.
Robots can now be trained and tested in far more spaces. Until now, scanning spaces like these required expensive equipment and time-consuming capture.
