Qwen3.8-27B reportedly went from 82 to 381 tokens/sec on a single RTX 3090 with 24GB VRAM. That’s a 4.6× increase without upgrading the GPU or changing the model. The improvement came from optimizing the inference stack. Here’s how the reported speeds progressed: Baseline: 82 tokens/sec Optimized MTP: 114 tokens/sec DFlash2 + context lookup: 138 tokens/sec Optimized lookup verification: 381 tokens/sec Three techniques explain much of what’s happening. 1. Speculative decoding reduces sequential generation work. Language models normally generate one token at a time, with each new token depending on the previous output. Speculative decoding uses a faster drafting mechanism to propose multiple tokens, then has the main model verify them. When several proposals are accepted, the system can produce more tokens per verification cycle. MTP and DFlash2 use different approaches to accelerate this process. In this experiment, optimized MTP raised throughput from 82 to 114 tokens/sec. Combining DFlash2 with context lookup pushed it to 138 tokens/sec. 2. Context lookup reuses text already available. If a model needs to reproduce text that already exists in its context, generating every token from scratch can waste computation. Context lookup finds matching sequences and proposes them for verification. The reported implementation checked blocks of up to 16 tokens, with roughly 15 accepted per block. On a 25,000-token document reproduction task, throughput reportedly increased from around 260 to 382 tokens/sec. This works particularly well when the output closely matches existing text. Open-ended conversations typically offer fewer opportunities for reuse, so the same peak shouldn’t be expected across every workload. 3. Prefix caching makes repeated prompts cheaper. Coding agents often resend the same system instructions, repository information, and code snippets across multiple requests. Prefix caching stores reusable computations from previously processed prompt tokens, allowing the runtime to skip repeated work when the prefix matches. In the reported test, time to first token dropped from 22.4 seconds to just 0.56 seconds. That’s approximately a 40× reduction in initial waiting time. For agents working with large codebases or documents, that can make repeated interactions considerably more responsive. One important distinction: 381 tokens/sec isn’t regular chat speed. The developer reports around 133 tokens/sec for ordinary chat. The 381 tokens/sec peak comes from a workload that benefits heavily from text reuse and optimized verification. Both numbers are useful, but they measure different conditions. The broader lesson is that GPU specifications alone don’t determine local inference performance. Speculative decoding, verification efficiency, context lookup, and caching can all affect how much work a system completes on the same hardware. The RTX 3090 still has 24GB of VRAM. The gains come from using its resources more efficiently. Project: TeksEdge’s Qwen3.8-27B RTX 3090 implementation.