Pinned Tweet
Artificial Analysis?
I thought AA stood for ....
Asubquadratic's Anewdecisionmodelistiedforfirst!
Today we're introducing the smartest, most capable version of our enterprise-ready decision model: SubQ Snap 1.1.
Snap is tied for #1 on the Artificial Analysis Decision Index and has made big gains on agentic tasks, outperforming Jev by +7.2 points across 16 leading enterprise benchmarks.
All while maintaining 384k tokens of context, in a single pass, as the only model with constant compute per token.
- Powered by Subquadratic Sparse Attention.
- Tied for #1 on the @ArtificialAnlys Decision Index.
- 96.2 on When2Call, up from 75.2, and up 6 points on @huggingface 5 Tools & Automation benchmarks. It knows when to call a tool, when to ask, and when to decline.
- 81% fewer confident errors on questions it can't answer.
- $0.04 per 1M input tokens. No charge for output.
Go to subq.ai for access, and 100M free tokens.
8
258
Attending LA Tech Week?
I build a free tool that will rank which events you shouldn't miss based on your GitHub, LinkedIn/resume, and interests.
It only costs me 1¢ on SubQ Snap, a new long-context decision model released today
Full transparency: I'm biased, I helped build the model. But this thing rips.
Get your ranking: techweekranker.app
Access Snap: subq.ai
Introducing SubQ Snap, the first long-context decision model for real enterprise work.
Snap is the only decision model or LLM where compute per token stays constant as context grows.
• 384K tokens of multimodal context, 12x larger than Jev.
• 61.7 on Decision Index v0.2.1 vs Jev’s 57.9, outperforming on 13 of 16 enterprise benchmarks.
•Processes 100K tokens and returns a decision in under one second.
•$0.05/MM input tokens. Output is free.
Available today to anyone.
The first 10,000 sign-ups get 100M free input tokens.
We’ll likely hit capacity fast and will add users as we add compute. Compute offers welcome 🙂.
Latency will be variable as we tune the platform to this new model and category of users.
Model speeds are 80ms for minimal context, 250ms for 32k, 660ms for 100k, and 2.5 seconds for 384k.
Visit: subq.ai/
Kevin Young retweeted
NEW SOTA DECISION MODEL: SubQ Snap
SubQ Snap absolutely crushed the S1 Bench, beating Jev!! Insane.
1,999 decisions, 0 errors. Macro accuracy 0.825, the best full run the bench has ever recorded. Jev scores 0.775. Mercury Decide, the previous best API model, scores 0.802.
Subset scores: paws 0.940, vitaminc 0.917, boolq 0.910, massive 0.909, aegis2 0.856. Only helpsteer2 lags at 0.416, and that subset drags everyone down. To be fair, none of the other decision models do well on helpsteer2... And it seems to have done better than Jev.
Subquadratic claimed 61.7 vs Jev's 57.9 on their own Decision Index. On an independent battery it is not even that close. It beat its own benchmarks by a lot.
It didn't hit the speed benchmark on my bench but the accuracy more than makes up for it. About a second per decision, and the entire 1,999-question battery cost roughly 3 cents of input tokens, output free. To be fair, I felt it was the fastest but my agent said it was a bit slower. Nevertheless, this is the SOTA decision model.
Hosted API only and still a preview model, so the open questions are weights and what the 384K context does on long inputs. But as a decision model, this is the new bar.
This is my new go to decision model.
Full results: bench.jakecuth.com
subq.ai
Exciting news in the world of decision models!
Introducing SubQ Snap, the first long-context decision model for real enterprise work.
Snap is the only decision model or LLM where compute per token stays constant as context grows.
• 384K tokens of multimodal context, 12x larger than Jev.
• 61.7 on Decision Index v0.2.1 vs Jev’s 57.9, outperforming on 13 of 16 enterprise benchmarks.
•Processes 100K tokens and returns a decision in under one second.
•$0.05/MM input tokens. Output is free.
Available today to anyone.
The first 10,000 sign-ups get 100M free input tokens.
We’ll likely hit capacity fast and will add users as we add compute. Compute offers welcome 🙂.
Latency will be variable as we tune the platform to this new model and category of users.
Model speeds are 80ms for minimal context, 250ms for 32k, 660ms for 100k, and 2.5 seconds for 384k.
Visit: subq.ai/
7
206
Kevin Young retweeted
Great conversation with @craigss at @EyeOn_AI about how the industry spends too much time talking about compute and data-centers, when we should be talking about more creative, more efficient ways to scale AI.
We cover:
- The hidden cost-multiplier within every model in production today.
- How the scaling laws of transformer-based models have become the biggest bottleneck in AI.
- How our team at Subquadratic solved the problem with a new model architecture that is 128x more efficient than frontier models at 2M tokens without compromising on performance.
Thanks to Craig and the Eye on AI team for the thoughtful conversation.
Check out the full episode below.
Kevin Young retweeted
Join Subquadratic in SF for a casual gathering this Saturday (link in comments)!
We are a foundation model company building the most compute-, memory-, and sample-efficient foundation model architectures for the next era of AI computing! We love to chat about challenging base assumptions of the industry.
We are hiring folks to work on large-scale pre- and post-training, long-context modeling, model architectures beyond attention, world models, efficient inference and training, and more.
Kevin Young retweeted
We've partnered with Appen to evaluate the benchmarks we published last week.
Results are in and we've actually improved across the board.
Link below to the full report.
@AppenResearch independently evaluated @subquadratic's SSA kernel - a learned sparse attention mechanism designed to reduce the quadratic scaling limitations of full attention.
Results at 1M-token context lengths:
- 56.2× wall clock speedup vs. FA2
- 62.8× FLOP reduction (validated via torch.profiler, <4% variance from theoretical)
- 95.6% average score across RULER tasks at 128K
- 86.2% average score on the hardest MRCR 8-needle bucket (512K–1M contexts)
- 81.8% SWE-Bench Verified resolved rate
Full report: appen.com/whitepapers/benchm…
Kevin Young retweeted
Asked Alexander Whedon, CEO of @subquadratic, if SubQ will replace or augment existing vanilla autoregressive LLMs like GPT-5.5/Opus 4.7.
SubQ specializes in long-context tasks:
> 12 mil token context length
> 52x faster than FlashAttention
> 20x cheaper than Opus
Important caveat is that SubQ doesn't provide a significant lift outside of long-context.
Clearly, models like GPT-5.5/Opus 4.7 can use SubQ as a *tool* within an agent harness. It is invoked for the long-context use cases and passes responses back to the AR LLM.
This alone would be a gamechanger for you if you build with AI.
Kevin Young retweeted
Yes, we are using weights from open-source models as a starting point, as a function of our funding and maturity as a company. This is something we intend to change, and we have run many from-scratch experiments at smaller scale already, including with further architectural variations. We take the weights, port them into our architecture, and do CPT, SFT, and RL for the behaviors we want.
To date, sub-quadratic architectures have required a significant quality tradeoff on long context. Our algorithm changes that. We are using that to do faster training, faster inference, and longer-context training and inference.
We just shared a technical blog post (subq.ai/how-ssa-makes-long-c…) with more details and will share more details again in a model card next week. If there is anything you think is missing, let us know, and we can make sure to include them!
Kevin Young retweeted
We were a little slow on this, but we just got a technical blog post up with more details. Please take a look!
subq.ai/how-ssa-makes-long-c…
We have a model card coming next week, and we are happy to take requests for any specific details there.
I am happy to answer any questions here!
Kevin Young retweeted
Attention Is All You Need (2017): most cited ML paper of the decade.
For 8 years, every frontier model has been built on quadratic attention. Process every possible word-to-word relationship. Compute explodes with context length. Accuracy degrades past 200k tokens.
Sub-quadratic attention was always the endgame. The labs just had too much invested in transformers to admit it.
SubQ is the first production-ready sub-quadratic LLM. 12M token context. Linear scaling, not quadratic. Outperforms Opus 4.6 on long context at less than 10% the cost. 52x faster than FlashAttention.
Linear vs quadratic. That's the whole game.
Introducing SubQ - a major breakthrough in LLM intelligence.
It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA),
And the first frontier model with a 12 million token context window which is:
- 52x faster than FlashAttention at 1MM tokens
- Less than 5% the cost of Opus
Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention).
Only a small fraction actually matter.
@subquadratic finds and focuses only on the ones that do.
That's nearly 1,000x less compute and a new way for LLMs to scale.
Kevin Young retweeted
Huge context windows are the biggest lie in AI.
Honestly, I haven't seen any benefits to scaling past 1M tokens. The more data you show the model, the dumber they get, so larger windows are pointless.
Attention is quadratic:
If you double the context, you are quadrupling the compute.
Past a certain point, models will get slow and expensive and start making stuff up. And they have a lot of trouble remembering the details in the middle of the context.
There are a million workarounds for this:
• Chunking
• Summarization layers
• Retrieval patches
• Sliding windows
But, honestly, they are just meh.
Here is something new, and potentially a solution that will help with this:
Subquadratic built an LLM that uses a subquadratic architecture. This means that the cost of increasing the context doesn't explode as it does with standard transformers.
Their LLM:
• 12M tokens of usable context
• No chunk-and-stitch workarounds needed
• The full context goes into the model, not summaries of it
If this works as advertised, it will completely change what Context Engineering means.
🤝 Paid partnership
Introducing SubQ - a major breakthrough in LLM intelligence.
It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA),
And the first frontier model with a 12 million token context window which is:
- 52x faster than FlashAttention at 1MM tokens
- Less than 5% the cost of Opus
Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention).
Only a small fraction actually matter.
@subquadratic finds and focuses only on the ones that do.
That's nearly 1,000x less compute and a new way for LLMs to scale.
Kevin Young retweeted
Introducing SubQ - a major breakthrough in LLM intelligence.
It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA),
And the first frontier model with a 12 million token context window which is:
- 52x faster than FlashAttention at 1MM tokens
- Less than 5% the cost of Opus
Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention).
Only a small fraction actually matter.
@subquadratic finds and focuses only on the ones that do.
That's nearly 1,000x less compute and a new way for LLMs to scale.