VR Camera Access @cixliv has been agitating for developer access to the cameras on Quest for mixed reality applications. I think he is right, and I’m glad to hear @boztank warming to the idea. I still don’t think MR is the best place to focus VR efforts for near term returns, but it will eventually result in important applications. It was always obvious to me that developers should have access to the camera images (just like a phone!), but there are forces pushing back against it: Defining and implementing new APIs is work. I suggest using Quest’s dominant position to just quickly make something simple and effective. Going through Khronos or some other industry group will make it 10x harder and less convenient to use. Plan on revving it multiple times instead of trying to get it perfect the first time. It isn’t hard to imagine PR nightmare scenarios – a malicious developer captures your significant other walking by in their underwear, etc. Many in the media salivate at the prospect of something to hammer Meta with, so it isn’t irrational, but be brave! Related, there is a cohort of people that just don’t trust third party developers, and feel that it is Meta’s duty to protect the user. Some of this is over privacy issues, but some of it is also just the understanding that a lot of developers will do a bad job. They will! Let the market sort it out. Not one “AR developer” in a hundred is going to have the computer vision expertise that internal Meta teams have, but a diversity of approaches is actually more important. I think this is easy, low hanging fruit: Like the red dot for recording, put a green dot up in the corner of the view when an app is receiving external camera frames, so the user is informed. Allow this to be disabled in settings. Start off with just CPU buffers of the camera images. GPU / DSP buffers can be added later, and devs are going to be doing a lot of clunky, low performance exploration anyway. Ideally you would offer all of the following, along with metadata and timestamps: Raw fisheye camera images. Rectified images from camera positions. Reprojected images from eye positions, which will have depth estimation distortions. Maybe also include the depth buffer. Meta could ship a basic implementation of this in a month if they wanted to.