getting your inference right shouldn't be hard. use https://xitter.cf/t.co/cY1dYE2kNN
Joined April 2026
- Tweets74
- Following9
- Followers1.4K
- Likes156
RouterPlus (YC F26) retweeted
Krea 3 by @krea_ai is live on @useRouterPlus
great to have partnered with the team for day 0 support
would love to see examples of what you folks do with it on RouterPlus !! What models would you like us to see support next ?
Krea 3, the new image editing model from @krea_ai, is now on @useRouterPlus .
High-quality edits in under 6 seconds. Reimagine the scene while keeping your subject consistent, or blend multiple reference images with a simple text prompt.
Krea 3, the new image editing model from @krea_ai, is now on @useRouterPlus .
High-quality edits in under 6 seconds. Reimagine the scene while keeping your subject consistent, or blend multiple reference images with a simple text prompt.
RouterPlus (YC F26) retweeted
CDMs are the future! Was great working with @scaledown_team on this release. Scaledown's purpose built CDMs - classify, extract and summarize are now available on @useRouterPlus
You already gave the LLM the answer. So why is it writing it out from scratch?
Introducing a new AI model architecture: Constrained Decoding Models (CDMs)
LLMs were built to write. Most production workloads just need an answer: fast, consistent, and affordable.
CDMs are purpose-built for 1 task each. They read your input once and skip the token-by-token generative loop that makes LLMs slow and expensive.
Not just a decision model, but a family of 3 CDMs live today:
• Classify: define your labels and get back a calibrated probability + reason for each. Used for routing, tagging, and intent detection.
• Extract: describe the entities in natural language and get back well-formed JSON with confidence scores + citations. Used for web scraping, document processing, and PII redaction.
• Summarize: pass in long text, images, or PDFs and get back a consistent summary under 1,000 tokens. Used for context compaction, metadata generation, and transcript digests.
Built for production:
• 1M tokens of multimodal (text + image) input, 31x larger than Jev
• 77.2% on DecisionBench vs Jev's 72%
• 200x faster, 467x cheaper than frontier LLM APIs
• $0.05/M input tokens. Output is free (yes, even for summarize)
Live on your favorite platforms today:
@OpenRouter @vercel_dev @LiteLLM @LangChain @TrustedRouter @itsArthurAI @useRouterPlus
Get 50M input tokens for free (for the first 50 hours) at link below 👀
ScaleDown's Constrained Decoding Models (CDMs) are now available on @useRouterPlus. You can now use their Classify, Summarize, and Extract models which feature a 1M input token context window, multimodal (text + image), and are reported to be 200x faster and 467x cheaper than traditional LLMs!
Try to today at: app.routerplus.com/playgroun…
You already gave the LLM the answer. So why is it writing it out from scratch?
Introducing a new AI model architecture: Constrained Decoding Models (CDMs)
LLMs were built to write. Most production workloads just need an answer: fast, consistent, and affordable.
CDMs are purpose-built for 1 task each. They read your input once and skip the token-by-token generative loop that makes LLMs slow and expensive.
Not just a decision model, but a family of 3 CDMs live today:
• Classify: define your labels and get back a calibrated probability + reason for each. Used for routing, tagging, and intent detection.
• Extract: describe the entities in natural language and get back well-formed JSON with confidence scores + citations. Used for web scraping, document processing, and PII redaction.
• Summarize: pass in long text, images, or PDFs and get back a consistent summary under 1,000 tokens. Used for context compaction, metadata generation, and transcript digests.
Built for production:
• 1M tokens of multimodal (text + image) input, 31x larger than Jev
• 77.2% on DecisionBench vs Jev's 72%
• 200x faster, 467x cheaper than frontier LLM APIs
• $0.05/M input tokens. Output is free (yes, even for summarize)
Live on your favorite platforms today:
@OpenRouter @vercel_dev @LiteLLM @LangChain @TrustedRouter @itsArthurAI @useRouterPlus
Get 50M input tokens for free (for the first 50 hours) at link below 👀
RouterPlus (YC F26) retweeted
Self hosting has inference providers alternative now. So unless you have reasons to own hardware and self host or you are post training, everyone doesnt need to download anymore. You can access these services like subscription. Routers make switching, discovery and experimentation far more convenient than you downloading and trying.
Routers are also catalog now, an active one and not a passive listing. People now go to marketplaces like @openrouter, @vercel , @useRouterPlus for their discovery vs huggingface
h/t to @ParikhRajesh07
We've been monitoring a decline in HuggingFace downloads across the top open LLMs we track for interconnects. It looks like there was a slight change in counting of downloads, as we saw a ~30% download rate drop for many top models, but this is continuing over time. What else do people think it may be?
Nimble v3 from @bespokelabsai is now FREE on RouterPlus ! check it out
app.routerplus.com/models/be…
RouterPlus (YC F26) retweeted
Nimble is available on @useRouterPlus!
Launching Decision models on @useRouterPlus. You can now use various system 1 decision models such as Jev, @perplexity_ai Decider, @bespokelabsai Nimble, Mercury Decide, Clef, Kev 4B and Decider 2B with 3 models free !!
app.routerplus.com/playgroun…
checkout decision models on routerplus.com/playground
we have some free decision models as well !
Launching Decision models on @useRouterPlus. You can now use various system 1 decision models such as Jev, @perplexity_ai Decider, @bespokelabsai Nimble, Mercury Decide, Clef, Kev 4B and Decider 2B with 3 models free !!
app.routerplus.com/playgroun…
RouterPlus (YC F26) retweeted
btw at @useRouterPlus we have been using numerous techniques to ensure that inference providers are ACTUALLY delivering what they claim - this involves running relatively expensive live evals , checking quality and also measuring symmetric KL divergence to monitor deviations from actual expected behaviour
since a lot of inference contracts go through us we can actively monitor for such discrepencies and hold providers accountable. its only done now for certain larger contracts - but we expect to roll it out to the wider platform soon
Yesterday I posted about inference price war pushing margin down. But I since learned inference scammers are a thing with shady price manipulations:
As inference providers,
- You can lower your input token price 100x cheaper but up your output price 5x. You rank higher in openrouter in price but tots cost is much higher.
- You can quantize the model to serve cheaply but users are buying a shittier model.
- You can retain and sell user data elsewhere to make more money.
- You can secretly swap to DeepSeek v4 flash but say it’s GLM 5.3 10% of the time.
Therefore it’s hard to know
1. You get the model you ask for
2. You pay for the tokens you actually burn across cache input, input, output and
3. Your data is safe.
I’m sure openrouter is working on it AND baseten, fireworks, together and other enterprise brands will continue to thrive even with higher pricing because they sell peace of mind.
RouterPlus (YC F26) retweeted
Buying inference today means picking between two bad options.
Commit for a year and pay for capacity you don't use. Or go serverless and get rate limited the moment you grow.
But usage arrives in bursts: a weekend eval, a two-week backfill, US business hours.
And not all tokens are equal. You might need BF16, not quietly quantized FP8. Served in the US. A latency target. Zero retention. Fast tokens for your product, cheap batched tokens for the rest.
The capacity exists. It's just scattered across big clouds and long-tail providers you've never heard of.
So we built a marketplace.
Introducing the @useRouterPlus managed inference marketplace.
Post the inference you need, when you need them. Commitment you are comfortable with. Providers bid. You pick the one that meets your spec.
we're already on track for 1 trillion tokens a month.
routerplus.com/capacity
RouterPlus (YC F26) retweeted
we are starting to host small gatherings around AI and inference with the first of the series tonight at 6:30pm at Hogpatch ( pizzas and snacks included ! ). Great chance to hang out with folks thinking and building in the space. luma link below
ps : we will be rebranding to @userouterplus, would love to show you what we’re upto as well!
RouterPlus (YC F26) retweeted
loved the GTM brainstorming with the guys. @XLN1999 and @alokbishoyi97 are the team I'd bet my life savings on. @EVO__HQ is gonna be the biggest force in managing inference loads, will come back to this post after the demo day, and again by '28 when they hit a billion in ARR (pardon for my lack of vision if you guys get there faster)
Kill the space, my Gs.
small get-together by friends and well wishers at @evo__hq before @XLN1999 and I set off to SF for YC fall batch
RouterPlus (YC F26) retweeted
@EVO__HQ got into YC!!
we are eager to talk with
- engineers who are hungry to become the best in the world at inference, posttraining and distributed systems.
- ops folks who have worked with datacentres and inference providers
RouterPlus (YC F26) retweeted
deployed and served custom models for a big @evo__hq customer over the past week
insane and rapid learnings
understood their traffic patterns, setup evals - did lot of ablation runs on multiple models
eventually we weren't getting enough savings, so we posttrained a model - first SFT and then other RL policies
then came inference optimization setup end to end , including custom spec dec models as per their data distribution, tuning vLLM configs, quantization etc
then more work around compute allocation , inference capacity planning / warming up as per traffic profiles etc
all orchestrated by our in house autoresearch / AI engineer
all this to ensure our customers get best bang for buck
we are now rapidly onboarding inference providers who can reliably service our growing demand
and if you are someone whose agentic/AI workload spends are more than $20k a month, then do reach out. We would love to figure what we could do for you!
Frontier performance with 20x cheaper cost
At @evo__hq we hillclimbed a smarted router setup for agentic workloads ( ITSM bench by @vibrantlabsai ) that achieves SOTA results while being 20x cost effective vs frontier models - beating as well the newly released models of Grok4.6 and Deepseek v4 pro in both quality and cost
For enterprise agentic workloads, what we have repeatedly found is that frontier models still do not get that last mile reliably and are prohibitively costly.
It becomes necessary to hillclimb on your specific data distribution, schema, policy and traces. This is what we do at @evo__hq
RouterPlus (YC F26) retweeted
preach ! been telling ya’ll
also the most important part is the last line. “the challenge is customizing them to an org/team/individuals requires data”
this is exactly what @evo__hq solves for btw
RouterPlus (YC F26) retweeted
“There is no best model for every task. Routing, harnesses, evals, and the right mix of open and proprietary models can radically change the economics.”
exactly the thesis of us at @evo__hq btw :)
At Databricks, AI coding token spend is growing exponentially.
1. The “efficiency frontier” matters. On Databricks Coding Bench, GLM 5.2, Opus 4.8, and GPT 5.6-Sol sit on that frontier: the best quality per dollar.
2. We see cost regressions when comparing Opus 5.0 to 4.8. Newer model doesn’t always mean more efficient.
3. Hard budgets are the wrong primitive. Your biggest AI spenders may also be your most AI-leveraged engineers.
4. There is no best model for every task. Routing, harnesses, evals, and the right mix of open and proprietary models can radically change the economics.
We’re investing heavily in all 4, both for ourselves and for our customers.
RouterPlus (YC F26) retweeted
Introducing EVO Router - a smart router that continuously optimizes and hill-climbs on your AI inference workloads.
It learns from your code, prompts, production traffic, use cases, and SLAs, then searches for the best setup for every workload.
That can mean more than choosing just a single model or routing through a classifier. @EVO__HQ can optimize across model × provider combinations, fusion systems, cascades, routing policies, and prompts — whatever the workload allows, while staying within your quality, latency, and reliability constraints.
As open-source models move closer to frontier-model performance, we believe this kind of workload-specific hill-climbing will become an important part of how AI systems are deployed and operated.
Our early users have seen 30–60% lower inference costs across use cases ranging from complex multi-turn agents to asynchronous batch workloads.
EVO Router has also reached the Pareto frontier on several benchmarks, outperforming existing setups on cost, quality, or both.