Founder of A11yNYC. ITP alum. Building https://xitter.cf/t.co/IEpHxaGQWT.
Brooklyn, NY
Joined May 2008
- Tweets2.1K
- Following1.4K
- Followers1.4K
- Likes1.4K
HT @The_Alex for the inspiration xitter.cf/The_Alex/status/209596…
great work from the Stark team
Stark Designer accessibility agent 😻
github.com/stark-contrast/sk…
excited about how #WebMCP can make canvas content more accessible to screen reader users. combining accessible markup with an agent makes working in a canvas better for everyone, and helps out blind users especially.
youtube.com/watch?v=F-HO5xBw…
I stood up an example of how you can pair WebMCP with an accessible canvas application to help agents and assistive tech users work together.
Video shows me navigating a flow diagram in the ChatGPT browser and prompting the agent to create new keyboard accessible diagrams from text and from a screenshot.
alight, I take it back; allowing a tool to pull focus into the in-app browser is a poor UX; instead, I set up a tool call to queue up a focus event for the target when the user moves focus back into the window.
@OpenAIDevs I'm testing out WebMCP for managing focus in the in-app browser for accessibility / screen reader use cases, but noticed that the text input in the agent window pulls focus automatically. This breaks the workflow for assistive tech users. Possible to forward to a11y team? youtu.be/V1MT0lIVe-Y?si=Fo_s…
really wishing Chrome's Gemini Nano supported tool use for local WebMCP support. maybe in the works? @ChromiumDev 🙏
@OpenAIDevs I'm testing out WebMCP for managing focus in the in-app browser for accessibility / screen reader use cases, but noticed that the text input in the agent window pulls focus automatically. This breaks the workflow for assistive tech users. Possible to forward to a11y team? youtu.be/V1MT0lIVe-Y?si=Fo_s…
this is the use case btw, I think this could be impactful, but right now it can't fully work xitter.cf/ckundo/status/20933439…
Excited to realize that WebMCP can be used to manage focus in a web app, like a supercharged "Skip to main content" link. Here's why its interesting:
This has applications for assistive technology users using agents with browsers. Instead of tabbing through the UI, move focus directly to an object. This can help screen reader users, switch users, dictation users, etc, reducing fatigue and deep linking to changes in the UI to verify the agent's work.
Just add a `focus_node` tool that specifies a focusable DOM node, and the web application can simply call `.focus()`.
I stumbled up this while working on the @OpenAI WebMCP hackathon, working on agent action verifiability for screen reader users:
devpost.com/software/screen-…
ALT screenshot of ChatGPT UI, asking the agent to focus a newly created node in the in-app browser, and the agent responding accordingly.
Excited to realize that WebMCP can be used to manage focus in a web app, like a supercharged "Skip to main content" link. Here's why its interesting:
This has applications for assistive technology users using agents with browsers. Instead of tabbing through the UI, move focus directly to an object. This can help screen reader users, switch users, dictation users, etc, reducing fatigue and deep linking to changes in the UI to verify the agent's work.
Just add a `focus_node` tool that specifies a focusable DOM node, and the web application can simply call `.focus()`.
I stumbled up this while working on the @OpenAI WebMCP hackathon, working on agent action verifiability for screen reader users:
devpost.com/software/screen-…
ALT screenshot of ChatGPT UI, asking the agent to focus a newly created node in the in-app browser, and the agent responding accordingly.
Remembered recently this drawing I did in 2017. I wasn't thinking big enough! Titled "OH in Williamsburg in 2022"
ALT Comic of 2 robots charting, one saying "I just started my second novel"
adding accessibility to the list. LLMs have improved over time, but still have a checkered track record there.
LLMs still produce bugs, but those bugs are different than what they used to be. It’s less off-by-ones and more about system design, ui usability, missing broader context. Some kinds of coding has been solved, but not all.
While models continue to improve, adversarial code review has been an incredibly powerful tool to catch many of these kinds of bugs.
It can be as simple as a one line prompt - “use a dynamic workflow to adversarial test every edge case in an iOS simulator”, or use Claude’s built in /code-review (or /code-review low, /code-review medium, etc.)
these evals are behind the state of the art, but illustrate the point around LLMs and accessibility: microsoft.github.io/a11y-llm…
Loving the Amplitude MCP with Claude for digging into accesslint.com analytics. Feels right for working through the metrics that are opaque or ambiguous.
Using Claude desktop for scheduled accessibility testing.
Claude uses the AccessLint MCP to scan and run user journey against your website, and creates a report every morning with changes 😎
claude.ai/directory/accessli…
ALT screenshot of the header for a scheduled task in claude desktop called "Weekday accessibility check — AccessLint", that is active an enabled for a daily accessibility check at 7AM