I made a computer vision tool for rock climbing holds analysis! As someone who is newer to climbing and wants to improve, I've noticed more and more recently how everyone solves the route differently. As I said to my friends the other day, "Everyone has to find their own way." 🤣 So I extended my last rock climbing demo with new analytics that I'm really excited about. I added what percentage of the total time your hands and feet are on a hold, as well as which holds each hand and foot is using. This data can be used in a few ways. For example, I know that I should use my feet and legs more when climbing. This tool can show me how much I use my feet, so I can use that as a baseline to improve. What I'm most excited about is how it enables comparing climbing sequences on the same route. You can compare against your friends' sequences: maybe you use 1, 3, 5, 7, and another friend uses 1, 2, 6, 7. Or you can compare against your own historical data, like I show here at the end. I used ViTPose+ Large for pose estimation and SAM 3.1 to segment the holds, both models accessed through the @vlmrun Gateway. I also use SAM 3.1 to segment the gray floor, so we know when the feet leave the ground to help with starting the route timer. Would love to hear what you think! The code is open-source on GitHub: github.com/jeremyipark/visio…

I made a computer vision tool for evaluating deadlift form! I've shared a few deadlift demos in the past. This new demo includes some new models, accessed through the @vlmrun Gateway: - SAM 3.1 to segment the barbell weight plate - ViTPose+ Large to measure the hip hinge angle, which serves as a backup for segmenting the reps. The pose data can definitely be used more later. - Gemma 4 26B through the Gateway's TypeSafe-compatible API to output the probability of the back being rounded or not. The predictions line up well with how I intentionally performed each rep. Of course, there are many caveats to what is good form, as it depends on the individual. All said, this has the pieces in place to quickly make adjustments to fit the individual better. In short, a tool like this gives people data to assess their form and improve over time. Let me know what you think! The code is open-source on GitHub: github.com/jeremyipark/visio…

I made a computer vision tool for rock climbing analysis in 3D using iPhone LiDAR! Having learned a lot from sharing my previous rock climbing demos, I realized that a lot of rock climbing analysis is well-suited for 3D. Even something as simple as supporting videos where the person filming moves with the climber requires 3D information. To get the depth information, I used my iPhone 15 Pro’s LiDAR depth sensor through my local iPhone app. I found that the depth measurements help a lot. I think the holds activation is better, and I like the final view of all of the holds in 3D. It’s also interesting to see the distance traveled in meters. Plus, it looks cool and it feels like a video game 🙂 In short, I think this new demo is an improvement in that climbers can see the real-world distance traveled and a visualization of which hand and foot activated which hold. I recorded the video and depth measurements from my iPhone app, and I ran the rest of the analysis on my computer. I used ViTPose+ Large for pose estimation and SAM 3.1 to segment the holds, both models accessed through the @vlmrun Gateway. Let me know what you think! The analysis code is open-source on GitHub: github.com/jeremyipark/visio…