Eric Henderson

Run Poolside Laguna S 2.1 (118B MoE) on a Single RTX 4090 (24 GB VRAM) - 250k context - 11.5 tokens/sec decode - 1000 tok/sec prefill! Poolside just dropped the official GGUFs for Laguna S 2.1, a 118B Mixture of Experts (MoE) model that goes toe to toe with trillion parameter giants such as Deepseek V4 Max, GLM 5.2, Minimax M3 on agentic benchmarks like Terminal Bench 2.1 and SWE Bench. And now you can run it on a single RTX 3090 or 4090 if you have 50+ GBs of RAM. (check the replies for benchmarks) I built llama.cpp from source on Ubuntu 22 and CUDA 13.0 to stress test its absolute limits. The VRAM limits are an illusion if you know the right flags. Here is the benchmark breakdown and the exact cheat code to push up to 250,000 context locally with 1,000+ tokens/sec prefill speed: # The Test Rig & Setup - Hardware: Single NVIDIA GeForce RTX 4090 (24 GB VRAM) | PCIe 4 -OS: Ubuntu 22.04 LTS, CUDA 13.0 -Model: laguna-s-2.1-Q4_K_M.gguf (118B total params, 8B active per token, 75 GB gguf) -Benchmark Prompt: Massive 28k token code/context prompt used across all runs # 1. The Baseline Run (High Context, Standard Flash Attention) Starting with a default build using Flash Attention (-fa on), batch size -b 4096 -ub 4096, and 12 CPU threads: 80k Context: Prefill: 472.19 t/s | Decode: 10.58 t/s | VRAM: 23.05 GB 125k Context: Prefill: 452.71 t/s | Decode: 9.40 t/s | VRAM: 22.72 GB 250k Context: Prefill: 409.80 t/s | Decode: 9.26 t/s | VRAM: 23.14 GB llama.cpp flags: -m laguna-s-2.1-Q4_K_M.gguf -c 250000 -fa on --threads 12 -b 4096 -ub 4096 don't sleep on the --threads, -b and -ub flags. without the --threads flag, decode struggled at 2 t/s. this flag alone helped it jump past 9 t/s without the -b and -ub flags, prefill was stuck under 90 tokens/sec. It ran stably up to 250k context under the 24GB ceiling, but the prefill speeds were sitting under 500 t/s. # 2. The --no-mmap Breakthrough (Doubling Prefill Speed) The single biggest performance multiplier came from disabling memory mapping (--no-mmap). Bypassing OS file paging overhead forces llama.cpp to allocate the model weights directly in memory upfront. 80k Context (--no-mmap): Prefill jumped from 472 t/s -> 968.53 t/s | Decode: 10.57 t/s | VRAM: 23.15 GB 125k Context (--no-mmap): Prefill jumped from 452 t/s -> 962.08 t/s | Decode: 10.52 t/s | VRAM: 22.84 GB 250k Context (--no-mmap): Prefill jumped from 409 t/s -> 883.00 t/s | Decode: 9.92 t/s | VRAM: 23.26 GB llama.cpp flags: -m laguna-s-2.1-Q4_K_M.gguf -c 250000 -fa on --threads 12 -b 4096 -ub 4096 --no-mmap Result: Adding a single flag instantly doubled prefill processing throughput without touching VRAM usage or decode speeds. # 3. KV Cache Quantization at 250k Context Scale To squeeze out maximum decode throughput and stability at extreme context window sizes, I combined --no-mmap with 8-bit (q8_0) and 4-bit (q4_0) KV cache quantization: 250k Context + q8_0 KV Cache + --no-mmap: Prefill: 951.93 t/s | Decode: 11.36 t/s | VRAM: 22.90 GB 250k Context + q4_0 KV Cache + --no-mmap: Prefill: 986.73 t/s | Decode: 10.64 t/s | VRAM: 22.97 GB q8_0 KV cache gave the absolute sweet spot: pushing >11.3 t/s decode with nearly 1,000 t/s prefill at 250,000 context tokens while sitting at a comfortable 22.9 GB VRAM. # The Golden llama.cpp flags If you want maximum performance for giant local codebases (250k context) without crashing your GPU: ./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf -c 250000 -fa on --no-mmap --threads 12 -b 4096 -ub 4096 If you’re rocking a single RTX 3090 or 4090, local frontier class agentic coding is fully here. Dust off your terminal. Official hugging face GGUF links uploaded by Poolside are dropped in the replies below.