
Building Faster LLMs | PhD @ UVA | Prev. ML @Cerebras | DNeo
Virginia, USA
Joined December 2023
- Tweets73
- Following116
- Followers83
- Likes95
Pinned Tweet
We just made LLM inference up to 30% faster ⚡
Introducing DNeo, a new diffusion drafter designed in collaboration with @cerebras researchers.
The secret? Your draft model needs a working memory, and the "right" context encoding mechanism.
📄 Paper: alphaxiv.org/pdf/2611.21925
5/6 — The Fused DNeo Head
Parallel drafting struggles with left-to-right consistency.
We combine DSpark's full-vocabulary correction with DFlash-2's context-aware scoring in a shared low-rank space.
The result: More expressive corrections.
1
7
417
6/6 — The Next Generation of Diffusion!
Diffusion language models can be powerful, if you architect them correctly. DNeo provides a strong blueprint for dLLMs.
💻 Code: github.com/J-sandler/DNeo
🤗 Weights: huggingface.co/J-sandler/Qwe…
Questions + comments welcome!
15
438
The curse of ‘max’ reasoning is so consistent now.
The conclusion may be that penalizing reasoning length may actually support better generalization?..
1
2
97
The fact that 1 line code changes are still being discovered proves that AI research may be a lot harder to automate than people think.
Multi-Token Prediction (MTP) can be done much better.
🚀 Turn out,you can recover the gradient for KL self-distillation to next-token prediction (NTP) for free without ever computing the KL. You just need a one-line code change:
h_ntp += h_mtp.detach() - h_mtp
Soft Multi-Token Prediction is pretty much a free lunch for MTP.
📜 Paper: arxiv.org/abs/2610.38331
💻 Code: github.com/JaydenTeoh/Soft-M…
🤗 HuggingFace: huggingface.co/papers/2610.3…
2
93
Someone should make a “benchmaxed” index if it doesn’t exist already.
I.e., (public eval score - private eval score)
1
70
Argon > Fable5.1 is going to infuriate everyone that has used both models.
Why is it so hard to evaluate (1) “big model smell” and (2) model strength on everyday tasks?
Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets
The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers.
Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale.
New to the Intelligence Index:
➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change
➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment
Upgraded:
➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions
➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised
➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool
Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0
Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1
Key results:
➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50
➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)
➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04
2
78
Not exactly.
This is relative to the api pricing for 6.1 sol and opus 5.5. However opus 5.5 is ~4x more expensive per task than sol.
So while Claude might give you 5x more ‘api credit’ it doesn’t give you 5x more bang for your buck.
Anthropic Subscriptions Offer 5x+ More Value Than OpenAI
Limit testing every AI subscription plan from Anthropic, OpenAI, Meta, SpaceXAI, MiniMax, Moonshot, Zdotai, Cursor, and Cognition
newsletter.semianalysis.com/…
2
66
We don’t understand consciousness well enough to asses this argument.
Two processes being mathematically equivalent doesn’t guarantee that they share the same consciousness status.
E.g., Is a perfect computer simulation of you conscious if you are conscious?
Replying to @Steve_Yegge
LLMs are pretty obviously not conscious though, and it's worth being clear about that (and why)
LLMs are mathematically equivalent to a book.
This is literally true:
LLM = Large Language Model = f(w, s, t) -> o
(w: weights, s: seed + settings, t: token input, o: output tokens)
They are *pure* functions (no side effect)
Thus they exist in the class of f(ℕ) -> ℕ
Since t and o are fixed size, any such function may be rewritten as an *explicit lookup table* (ie, literally printed into a book that maps from input args to output)
Thus, LLMs are mathematically equivalent to a book.
You do not think books are conscious.
QED.
Note: I am talking about LLMs! This doesn't mean that agents (with tools, that execute over time, etc) couldn't be conscious.
However, it's worth recognizing that here's something categorically different between humans (and other living things) and LLMs (and other mathematical constructs)
1
1
71







