Qwen3.8-Flash-Next is starting to feel less like a local model and more like a cloud API running on your desk. vr8vr8’s dual DGX Spark recipe is now hitting: 138 tok/s structured 133 tok/s code 81.6 tok/s prose 383 tok/s aggregate structured at 4 streams But the interesting part wasn’t just the speed. It was getting Hermes to behave. The first Hermes tests produced garbled Chinese and endless loops. The quant wasn’t the problem. The serving stack was. Hermes omitted temperature, so vLLM 0.30 fell back to temperature 1.0 with top_p 0.95 and top_k 20. Worse, Hermes sent reasoning as a reasoning object while vLLM was looking for reasoning_effort. So “thinking off” never reached Qwen3.8. That meant sampling at 1.0 plus thinking enabled at xhigh. Not exactly a recipe for stable tool use. I patched both issues: • Hermes reasoning now maps to Qwen’s thinking settings • none actually disables thinking • Missing temperature now defaults to 0 • Explicit temperature and enable_thinking still take priority Three temperature-free, thinking-off requests came back cleanly in English, and a medium-thinking request also completed normally. The PR is open, so stock Hermes integration still has the old behavior. Then v4.1 landed with FlashInfer GDN prefill. My tests showed roughly 4 to 6% higher decode numbers, although I’m not attributing all of that to FlashInfer. The bigger results: 383 tok/s structured at 4 streams 18/18 needle retrieval through ~900K context 70.6 tok/s decode at the 880K rung 2,577 tok/s prefill at 880K 107.4 tok/s on a 2,048-token write_file agent turn at 35K context 81 tok/s on a Hermes research turn with two web searches There’s also a huge optimization in the output head, dropping it from 1.18 GiB to 0.33 GiB. That alone cuts a lot of memory traffic during decoding. The lesson here is bigger than one benchmark. Local inference speed isn’t just the model or quant. The runtime, KV format, draft strategy, output head, sampling defaults, chat template and protocol layer can completely change the experience. vr8vr8 had the speed in part one. Part two was making the rest of the stack stop getting in its way.