DeepSeek V4.1 Flash 4× CMP 170HX optimized drop 🎙️⚡️ 178 tok/s, one user. 587 tok/s across 8. 🔥 Prefill 1,570 tok/s at 2K. KV cache 617K tokens. EXL3. Full graphs and coop underneath, tensor-core MoE from row 43 up. @Morrowmake github.com/CurtisAccelerate/…

DeepSeek is a beast!

😂 really does not want to reopen the mixed weights lane...

Pushed the bot too hard.

See the live: curtisaccelerate.github.io/a… I put the prompt into Hermes, it worked about 60 minutes with no feedback and produced what you see above. No claim is made to respecting the original benchmark requirements. I provided 4-5 turns of very generalized feedback. bench / prompt from @niklaslenz_ai

Quants? you want to complain about the DeepSeek is 🔥🔥🔥 This is 60 minutes autonomous

my bad, these are the better links to lead with: github.com/CurtisAccelerate/… github.com/CurtisAccelerate/…

New DeepSeek v4.1 1017 Optimizations! - Extended the new attention/drafter performance improvements from 1 user out to 8 user. The relative improvement is less with more concurrent streams because there's less slack and 7 requests fare worse due to specific padding but overall we saw gains across the board. - We added sampled drafting. Most benchmarks are tested at temp 0 but Hermes/agentic default temp is 1. This drops the acceptance rate of greedy drafters on real-world tasks where the temp is not zero. Our current recommendation is greedy for temp 0, sampled for 1 request, and greedy for 2-8. We haven't tested sampled under concurrent requests. It requires some harness/serving intelligence. You can use the new optimizations with either greedy or sampled depending on your use case. For future optimizers/optimizations, testing & optimizing drafters under real-world conditions may yield most value. github.com/CurtisAccelerate/…

That's fast.

Also that learning to code + developing is different is not that different from today where in school one is encouraged to build data structures, sort algos, etc. where those things aren't practiced in professional development. But I do agree that the era of new programming languages developed for people is probably over.

You're both wrong. But, to understand why we must pull back the veil on what it means to learn to code, Level 1: You learn to think in a certain logical way and can think in multiple layers of abstraction. This is going to be valuable regardless whether coding preserves. Level 2: You have earned confidence and grit. Earned confidence only comes from struggling through real difficulty. Level 3: As a byproduct of developing, you develop experience in building and shipping. It is true you don't need to code to get to level 3 but there's no path to level 3 without developing: you are already level 3 and see your coding isn't valuable. Level 1 and 2 can be attained in 2 years provided one isn't using AI. Level 3 takes 10 years and there's no easy path to level 3 without being a developer. While task of coding is easier, being a software engineer is now more difficult than before because there's no simple formula for getting to a level where one can provide value. Before, it was simple-- just specialize in something. There's another distinction to be understood: the promise of coding was always a false-promise. Because the promise was always: learn to code and develop anything. That was never true because specialized knowledge gaps. Here's how I'd structure a 4 year program for a developer of tomorrow, fwiw: First 2 years: coding + developing with no AI permitted. Second 2 years: focus exclusively on agentic development and specifically focusing on building loops, observability, security, devops, selling.

DeepSeek 4.1 1017 Quality? I notice some people reporting the EXL3 weights on DeepSeek 4.1 1017 produce poor outputs in reply to my build and others. It has not been my experience with my build. It has been extremely strong in my testing. For example, I gave it a single prompt and it worked for 30 minutes building an extremely polished game that worked perfectly on a single prompt. And, I have it working on even longer run tasks. But we did something different: we use some of the FP8 weights for important aspects. I don't know if that makes the difference or if its the harness or server. But, this model has impressed me and worked extremely well.

Understand the sentiment but I haven't found it poor at all! It has been truly excellent. Do not assume my build is same as others: I use the full quality NGRAM weights. I have seen it work for 30 minutes perfectly in Hermes on a single prompt to produce perfect working game.

Something to know: The largest perf gains, from the new attention + drafter, are only seen at concurrency/request depth 1 for now when more lanes are active, it does not use the new drafter and falls back to the previous optimized default. This is not an inherent limitation though the relative advantage would likely decline at higher concurrency due to having more parallel work. In order to support multiple requests, requires batch-aware buffers. The next obvious target would be request 2. Example, on this sweep only users 1 is the new attention:

And our A/B matched pairs testing on the new attention and drafter:

More information on our testing/results