I got a 1T (trillion) parameter model running on my MacBook Pro. Kimi-K2. 1.029T params. ~1 TB raw weights. 524 GB converted. ~1.7 tok/s. Yesterday it was 671B. Today it's 1T. Same laptop. Same M4 Max. No cloud. When I say we: I mean Claude and me.

i proved a post-trained 600M parameter model outperforms a 27B parameter model. three expert-labelled decision domains, same prompt, same letter readout, same box Qwen3-0.6B fine-tuned vs Qwen3.8-27B off the shelf, zero-shot, test accuracy NOAA space weather forecasts: 0.92 vs 0.64 UK Airprox board reports: 0.56 vs 0.43 typed-decisions benchmark: 0.77 vs 0.72 being transparent about what this is and is not: the 0.6B is the April 2025 Qwen3 generation. the 27B is Qwen3.8, the current flagship. older and 45x smaller both ran in bf16 through the same loader. no quantization on either side the 27B is off the shelf, zero-shot with its chat template. the 0.6B saw the training split. that is the point, not a loophole before training the 0.6B sat at or below the majority class on all three. 0.30 on typed-decisions, 0.29 on airprox trained on one DGX Spark. full fine-tune, 3 epochs, batch 16, 5.6 GB peak memory. 2.9 GPU hours for all three models, one seed each. headline numbers are means over 3 seeds 19 ms per decision vs 504 ms for the 27B on 500-token prompts. 26x faster the thesis: small models can be fine-tuned, then pruned and quantized, to beat much larger models inside a specific domain. on-prem, cloud, ideally on the edge device itself. this run is the fine-tune leg. pruning and quantization are next datasets are public so you can rerun it: NOAA SWPC daily reports and forecasts, UK Airprox Board monthly reviews, LocalLLaMA typed-decisions on Hugging Face. airprox is research use only more to come..

i trained a Laya decision model that's 2.6x faster on my DGX Spark Laya (Convai - original): 421M | 0.787 | 18.8 ms mine: 144M | 0.780 | 7.2 ms same accuracy 3x smaller 2.6x lower latency same scorer head, same trainer i swapped the encoder for JHU's mmBERT-small 26 minutes of training on one box four encoders under the same Laya head: ModernBERT-base 164M -> 0.619 Ettin-150m 164M -> 0.719 mmBERT-small 144M -> 0.759 at 4 epochs mmBERT-small -> 0.780 at 12 epochs, three seeds, sd 0.002 the head starts random so epochs were the lever. 4 -> 12 bought 2.2 points honest bits - ModernBERT-base is the same family as Laya's own encoder and i still can't explain why it lands 16 points lower. one aggressive-lr run collapsed at loss 1.2 and i killed it after 20 min. latency is batch-1 bf16 on the Spark, not a Jetson yet. accuracy is one benchmark, LocalLLaMA's typed-decisions, 2,000 test decisions

GLM 5.3 Flash, Q4, 186 GB running on my single 128 gb DGX Spark does not fit. ran it anyway full gpu offload: OOM -ngl 0: also OOM (a cuda build pins host memory, not the escape hatch you think it is) cpu mmap, the only config that survives: 1.3 tok/s expert streaming from nvme with O_DIRECT: 4.5 bigger expert cache (104 GB): 6.2 same config on code generation: 8.6 tok/s how: there is an unmerged llama.cpp PR (25294) that streams MoE experts off the SSD through an on-device cache. it targets an API that was deleted 789 commits ago, so we ported it onto the GLM branch: 72% came across untouched, the 336 lines got reviewed, and the output is byte identical to the in-memory model across every cache partition we forced. lossless, verified, not vibes learned the hard way: O_DIRECT is mandatory. buffered streaming runs 1.0 tok/s = slower than not using the feature at all llama-server's default 8 GB prompt cache silently kills the spark this full. --cache-ram 0 under streaming, plain decode becomes workload dependent (6.2 prose vs 8.6 code). different tasks route to different experts = different miss patterns where the next speed is: decode stalls on expert misses 49% of the time while the SSD sits at 28% busy. the bottleneck is queue depth, not bandwidth: layer L+1's experts depend on layer L's output, so you cannot prefetch deep. fix the stall and the ceiling is ~12. stack speculative decoding on top and 15+ is the target we'll see how far we take this

it’s actually a good video.. makes me think about how ai is another technology that people must leverage, it is not the end all solution. it will change fundamentally how we function as a society just like other technological revolutions 80% of society historically performed - farm work - industrial work - office work (current) i wonder what the future looks like

Qwen3.8 Flash Next: 128B total, 6B active, 4-bit, 97 GiB GLM 5.3 Flash: 321B total, 17B active, 2-bit, 101 GiB ran a 1:1 on the spark: two flagship flash MoEs at the same memory budget decode at depth 0 / 16k / 32k: flash next: 24.2 / 21.2 / 19.0 tok/s glm 5.3: 14.7 / 7.5 / 3.2 tok/s glm does not just lose, it collapses with depth. the interesting part is why. the branch that runs glm today requires flash attention off. turn it on and the 16k number jumps from 8.0 to 13.3 = most of the collapse is a flag, not the architecture. that branch is a draft and the fa-on numerics are unverified, so i am not counting it yet. but the gap should mostly close as the port matures the other finding: i have a bandwidth law fitted on flash next that prices decode from bytes moved per token. re-ran it out of sample tonight: 3 of 3 cells within 2-6%. pointed the same law at glm: missed by 2-5x. the law is architecture local. fit on one moe, it does not transfer to the next two models, one spark, ~100 Gb/ea. the one moving 3.6 Gb per token beats the one moving 5.8.

i got GLM 5.3 Flash running on my DGX Spark with DFlash2 speculative decoding 321B params (17B active), 2-bit quant 101 GiB of weights, all in memory, zero nvme offload plain decode: 17.1 tok/s prefill: 289 tok/s with dflash2: up to 41 tok/s (2.37x) on structured output it accepts 73% of drafts = 41 tok/s. on prose it accepts 23% = no speedup. plain decode is dead flat across all three workloads (17.2 / 17.2 / 17.3), so the entire spread is acceptance. speculation pays for structure, not language. quality at 2-bit : passed all 5 gates i threw at it. 0.85 on code generation, 10/10 needle retrieval at ~11k context, zero degeneration loops. inco's own numbers come from 4x GB300 through sglang. this is one box, llama.cpp, and the 1B drafter converted to gguf locally output is byte identical to plain greedy on the workload where it wins. lossless, faster, one spark.

Why is no one talking about this?

👀

Good read btw blog.exolabs.net/nvidia-dgx-…

qwen 3.8 27B on a DGX Spark, NVFP4 +MTP exploring spec decode same box, one variable: multi-token prediction (mtp) on or off 8k: 11.3 -> 26.2 (2.32x) 32k: 10.9 -> 26.0 (2.39x) 64k: 10.4 -> 27.7 (2.66x) 128k: 9.5 -> 23.0 (2.42x) 256k: 8.2 -> 20.7 (2.52x) 512k: 6.4 -> 6.0 (0.93x) 1M: 4.4 -> 4.2 (0.95x) it holds 2.3-2.7x across a 32x range of context. then between 256k and 512k it goes negative the mechanism is clean once you see it. drafting amortizes the weight reads: you pay for the weights once and cash in several tokens. but past a quarter million tokens the step is not weight reads anymore, it is re-reading the KV cache, and speculation cannot shrink that. the extra verification passes cost more than the accepted tokens save so MTP is a short and mid context lever. at 1M you are attention bound and no amount of drafting helps

qwen3.8 · 3 quants DGX Spark some initial tests. i ran NVFP4, FP8, and GGUF Q4 at 1k, 8k, 32k. at 32k NVFP4 is 2.67x GGUF on prefill. decode barely notices the pack.

Seems like open weight/source caught up fast

replicated @anemll's W4A4 harness on one Spark GB10. real DeepSeek-V4-Flash NVFP4 layer 0, 256 experts, rank 0, b12x-first. W4A4 is 1.40-1.75x W4A16 on cuda-graph medians. the note said 1.4-1.8. JSON never hits 1.80. B12X wins tiny decode (M=1-2) and M>=2048 prefill. CUTLASS takes M=4 through 1024. peak B12X edge 1.15x at M=8192. failures: [] one box. not their 2xGB10 TP=2 serve table.