Design and Engineering @EvenRealities. Experiment with #spatialcomputing. Build creative tools.
London, UK
Joined December 2009
- Tweets8K
- Following1.5K
- Followers7.1K
- Likes8.7K
Pinned Tweet
Started experimenting with hand-tracking-based interactions for my little voxel editor – BoxelXR. There are a lot of things to tune, but overall it feels good. Though it would require some decent work in sound design to compensate for the lack of haptic feedback.
Assembled the arm. Calibrated. Worked smooth. Thinking about starting a band.
Robot: SO-101
Interface: LeRobot
Synth: Orchid
Played with rotational movements and explored the relationship between circular and sinusoidal motion.
The latter can be treated as a dimensional reduction: projecting 2D circular motion onto 1D.
Microcontroller: ESP32
Language: C++
Device: StackChan
Randomly looking around Stackchan by @M5Stack.
One of my favorite things to do when I explore new tech is to implement some ideas from The Nature of Code by Daniel Shiffman @thecodingtrain. Used the Random Walker algorithm this time (always a good starter). The cute robot doesn’t walk for now, but she is looking around in the same manner. This will be useful when I add the object detection model.
Added ERASING and COLORING modes to the WIP VisionOS version of BoxelXR.
On Quest, I put all operations on the controllers to speed up interactions (like shortcuts). But hand-tracking is less precise compared to controllers, so I am trying to figure out the best way to handle frequently used UI.
For now, I use a floating UI panel. It works fine, btw. Since hand-tracking input is quite ambiguous in this context, having a persistent UI layer reduces the chance of accidental triggering when you work with voxels.
Made a few base voxel creation interactions for the BoxelXR VisionOS version.
Unlike the Quest version of the app, which I built with Unity, this one will be developed natively. RealityKit, with its ECS architecture, is quite fun to work with.
The AVP version will be hand-tracking first, but I plan to support accessories and make a few custom controllers (ESP32 is a nice base for this).
more toys (=^･ω･^=)
thanks @M5Stack
Made a few base voxel creation interactions for the BoxelXR VisionOS version.
Unlike the Quest version of the app, which I built with Unity, this one will be developed natively. RealityKit, with its ECS architecture, is quite fun to work with.
The AVP version will be hand-tracking first, but I plan to support accessories and make a few custom controllers (ESP32 is a nice base for this).
🕹️ Connected M5Atom (ESP32 + IMU) to my little VisionOS game 🐍
Made a custom controller for VisionOS. Uses IMU to drive discrete directional events and BLE to connect to the device.
Apple Vision Pro doesn’t have native controllers, but as an heir to a much more mature OS and ecosystem, it has everything you need to build your own!
Interesting research about hybrid intelligence by @lining_yao. Love the fusion of mechanical/digital/bio design. So inspiring!
Meshbots are 🔥
youtube.com/watch?v=wfJpKjMw…
Made a variant of my previous experiment. It uses 4 directions to drive BRIGHTNESS and HUE states. Tap the screen to switch between these two modes.
💡 4 directions are enough to encode 4 main brightness states: ⬆️ Bright | ➡️ Less Bright | ⬇️ Dim | ⬅️ No light. For Hue, I used the most common colors Warn/Cool White and Red/Blue, which are used in domestic agriculture and skincare.
🔬 The state picker/slider follows the physical affordance of the device itself. This is a lovely way to simplify the interaction pattern by leveraging one of the Fundamental Interaction forces.
🎛️ The hardware is M5AtomS3R (ESP32). Rotational data is provided by the built-in IMU sensor. It’s hacked to the bulb directly via BLE. M5Canvas is for UI staff.
Ported the previous experiment with EPS32 and BLE bulb to M5Atom. The mode selector is IMU-driven: UP increases brightness, DOWN decreases it.
Atom has a nice balanced physical affordance for defining 4 directions. It’s squarish. I guess the next iteration would be to leverage them: stationary orientation triggers the brightness level or hue (if you treat it as a parallelogram, which it is, you get 6 directions to map!). Generalizing this idea: clockwise directional update: i++, anticlockwise one: i--.
Many of you are familiar with this situation: you found the best-looking lamp, but sadly it doesn’t have a brightness control. Fortunately, there is no problem that can not be fixed with tape and a bit of hacking.
M5Stick is directly connected to the bulb via BLE. The potentiometer controls the brightness range.
Made another experiment using IMU data to drive UI interactions: A bulb brightness controller.
This time I used only three directions. When UP is detected, nothing happens; it’s the idle state. FORWARD/BACKWARD activate increasing/decreasing modes, respectively. Clicking the button applies changes.
On the technical side, the M5Stick directly controls the bulb by broadcasting commands via BLE.
Solving a system of linear equations is very similar to UX Design. It's an overlooked way to hone UX skills and sharpen your eye for system simplification.
Oleg Frolov retweeted
After 4+ years at Meta Reality Labs building Ray-Ban Meta, Orion, and shipping AR to millions…
I’m starting my own spatial computing studio.
Building Realms is live.
For the last 12 years I’ve turned ideas that felt like science fiction into working products — at Amazon, Meta, and through public prototypes on Apple Vision Pro.
Now we’re taking that craft and applying it where it counts: operating rooms, construction sites, factory floors, classrooms, and brands that want to create real wonder.
Not every problem needs spatial. Knowing when it does (and when it doesn’t) is the part most people get wrong.
We’ll be building in public. More details 👇
If you have a spatial problem worth solving — or know someone who does — DM me or visit buildingrealms.com
What’s one real-world problem you wish spatial computing could actually solve?