I’ve found it so useful for this, being able to see “wow, my session for ___ cost HOW MUCH?”
It’s also super useful for observing cache hit rates across harnesses.
Our session affinity logic keeps you on the same provider whenever possible. By default, you’re only falling back if the provider is degraded (in which case your cache is gone, but your request still gets served).
And, we’ll keep you on that new provider to preserve that new cache
It would need to go through AI Gateway, but would highly recommend shifting provider direct traffic to AI Gateway using BYOK.
That way, when a lab has downtime, you can fall back to AI Gateway’s provider catalog and not even feel it.
You can filter out dev users from projects (will make this clearer). but IMO the unfiltered view is useful as a top level “where are my tokens going?”
My favorite part about working on this was getting the chance to highlight AI Gateway’s best in class session affinity infrastructure.
It handles things like tracking cache state for multiple models within a session, automatic fingerprinting, and much more.
Cache hit rates can make or break inference economics, and AI Gateway helps you maximize them right out of the box. Any model, any provider.
I've been playing around with muse wonder via @vercel AI Gateway for agentic work and am super impressed with how well the model communicates.
Here's an excerpt from me getting some help working on one of my motorcycles:
- It starts with the lead clearly in an easy to read table and cites where it got the info from.
- Next, it tells me exactly what I need to buy, and then gives me genuinely helpful bullet point tips without overdoing it (this is the taste part that's so tough to get right).
- Finally, it suggests a follow up that actually makes sense. The tone and "feel" of this model just... clicks. This is all in fx.sh too, so the model is doing this by itself w/ minimal harness intervention.
Haven't tested it for coding yet, but switching all of my @eve agents to use muse wonder as the orchestrator.
Sam Chitgopekar retweeted
Meet Glyph Cluster on AI Gateway.
𝚜𝚝𝚎𝚊𝚕𝚝𝚑/𝚐𝚕𝚢𝚙𝚑-𝚌𝚕𝚞𝚜𝚝𝚎𝚛
• Free while in stealth for a limited time
• Access restricted to paid users
• Use it for coding & agentic knowledge work
• Prompts may be used for model improvement
vercel.com/changelog/glyph-c…
Sam Chitgopekar retweeted
Vercel is the best security team we have worked with. The entire management chain is hyper paranoid and wonderful to work with! We will release more on these container escapes once the patches land! Stay tuned!
Sam Chitgopekar retweeted
Meet Pixel Canary on Vercel AI Gateway.
• 𝚜𝚝𝚎𝚊𝚕𝚝𝚑/𝚙𝚒𝚡𝚎𝚕-𝚌𝚊𝚗𝚊𝚛𝚢
• Free in stealth for a limited time
• Agentic coding & mobile app dev
• Ties GPT-6 Astra on Next.js evals
• Prompts may be used for model improvement vercel.com/changelog/pixel-c…
Sam Chitgopekar retweeted
happening next week!!
Sam Chitgopekar retweeted
Grok 4.7 is live on AI Gateway, fx, and @eve.
40% off on all until September 27th.
𝚜𝚙𝚊𝚌𝚎𝚡𝚊𝚒/𝚐𝚛𝚘𝚔-𝟺.𝟽
vercel.com/changelog/grok-4-…
Sam Chitgopekar retweeted
Jev by @typesafeai is free on Vercel AI Gateway until Sept 25.
Build with the fastest adopted model on the Gateway at no cost.
vercel.com/ai-gateway/models…
xitter.cf/vercel/status/21010773…
Jev was adopted faster than any other model in AI Gateway history.
In the first day, @typesafeai reached ~13% of teams, 2x the GPT-5.6 family and 6x Fable 5.1.
Sam Chitgopekar retweeted
New model index found that only @vercel has Jev.
Check it out y’all: computesdk.com/benchmarks/ai…
