@vibiumdev co-founder. also @tapsterbot, @saucelabs, @seleniumhq, @appiumdevs, @pinthing go irish ☘️
Chicago (Oak Park)
Joined September 2007
- Tweets27K
- Following7.2K
- Followers11.1K
- Likes43.3K
fun weekend project: roadie boards! 🎸🤘
what is roadie? it's a "kvm" (keyboard, video/voice, mouse/multitouch) hardware-based approach to mobile automation. roadie works out of the box with vibium. it captures video directly from a phone over hdmi (or a camera when direct video isn't available), and can act as a keyboard, mouse, touch stylus, and now microphone and headset.
the two smaller boards are Adafruit QT Py RP2040s. one connects to the phone, the other connects back to a computer running an ai agent and vibium. this board i designed simply connects the two boards over UART (RX/TX), with a common ground.
this is arguably one of the simplest projects you can make in KiCad. still takes some thinking and planning, though.
our ai friends aren't particularly good at circuit design yet, but chatgpt has been surprisingly useful for talking through design choices.
next step: get some boards made and see if they actually work.
ALT kicad pcb screenshot
ALT kicad schematic screenshot
i've been getting more inbound interest about using vibium with mobile phones lately, specifically getting them wired up to be driven by coding harnesses for testing/verification. a while back, i released this project (called Roadie) that bridges real devices to the browser so Vibium can easily automate them. if you're a small team with a "how the f*** do i get claude/codex to test my app on a real phone?!" problem, let's talk!
one symptom of the problem is that you have everything automated, but one dumb thing still requires manual checking, and it's annoying everyone on the team -- it feels like a speed bump when you're driving at top speed.
i'm specifically looking for those problems that can only be done on a real device -- audio in/out, video in/out, camera, calls and texts (that require real sim cards) -- that kind of stuff. emulators and simulators do a lot, but they can't do everything. roadie can.
oh heh, vibium drives phones now, too.
demo video is google gemini + antigravity cli + vibium + roadie + a google pixel & an iphone with esims & cell service texting a knock-knock joke. because why not. worked on the first try! (not recorded: anti-gravity crashing on the second try.)
two fun things here: the demo uses some new features coming in the next version of vibium - firefox support and video (webm) recording.
and the nice thing about roadie is the phones are *mostly unmodified and plug-and-play and both controlled via a raspberry pi 4.
*accessibility mode needs to be enabled on ios for usb mouse and keyboard input, but android works as-is - no adb or dev mode required.
back home in chicago after a great week in the uk. sure ai is amazing, but it's kinda nuts i took this photo this morning.
jason pinged me for my thoughts on the openai news below. posting my dm reply; might as well go on the record, too. #thoughtleadership
the a16z post "Avoiding Death on the Yellow Brick Road" by @joeschmidtiv is the one to meditate on every time the big ai labs drop another startup-killing feature. at least on the business side of @VibiumDev , the word i'm leaning into is *independent*. even if openai (or your frontier model of choice) does testing, openai testing its own work isn't the same thing as an independent check.
my bet (and i might be wrong! yeah, startups!) is that cto-types (perhaps some reading this post?) will be ok with a frontier model writing and testing the code, but they'll trust the process more with an additional 3rd-party/independent check in the loop.
other industries often rely on independent verification (accounting, electronics, aviation, transportation, food, medicine, etc.). and the question is whether that same thing comes to software, too.
@OpenAI @OpenAIDevs
just launched a cloud API that you can use to test web pages
Video: youtube.com/live/Fls_onRviPM…
@sama actually said the word “TEST”—finally ;)
This is going to disrupt a lot of the test cloud players, and enable testers around the world to move faster 😳
But it’s still not as smart as the CARBON test cloud 🤩 testers.ai
this was me with @saucelabs and "browsers as a service" for.... years...
18 years later, now everyone and their ai agent is running a browser in the cloud to do all kinds of things. 💁
vibium v26.8.21 has shipped! 🚢🚀
the biggest news is we added firefox and screencast/video-recording support (thanks to new functionality in the webdriver bidi spec and firefox shipping official support for it this week.)
and bug fixes. so many bug fixes.
my favorite part of the release notes?
new contributors!
"@ vincebln2 made their first contribution in #180
@paulsmith made their first contribution in #175"
will you be next? 🫵
v26.8.21 release notes ➡️ github.com/VibiumDev/vibium/…
more info about recording video/screencasts with vibium ➡️ github.com/VibiumDev/vibium/…
firefox 154 officially ships tomorrow with screencast support ("browsingContext.startScreencast") in their webdriver bidi implementation.
if all goes well, i'll tag the next official vibium release in the next day so people can start trying out the new feature.
v26.8.18 is mostly bug fixes, but i'm sneaking in a few new features, too.
🦾🫡🎥✅👍
oh heh, vibium drives phones now, too.
demo video is google gemini + antigravity cli + vibium + roadie + a google pixel & an iphone with esims & cell service texting a knock-knock joke. because why not. worked on the first try! (not recorded: anti-gravity crashing on the second try.)
two fun things here: the demo uses some new features coming in the next version of vibium - firefox support and video (webm) recording.
and the nice thing about roadie is the phones are *mostly unmodified and plug-and-play and both controlled via a raspberry pi 4.
*accessibility mode needs to be enabled on ios for usb mouse and keyboard input, but android works as-is - no adb or dev mode required.
it only takes one thing to automate mobile phones: courage
ALT two smartphones (google pixel 10a on left and iphone 15 pro max on right) connected to adapters, circuit boards (adafruit qt py rp2040), video capture cards, and a raspberry pi 4. driven by vibium and roadie.
i would love to consolidate all those dongles, adapters, and microcontrollers into one cool, custom board. @antmicro, perhaps?