Building Atlas, Rust inference engine w/ custom CUDA kernels on DGX Spark and Strix Halo | MLPerf Agentic Edge Task Force MLCommons | Ambassador @Alibaba_Qwen
Joined March 2026
- Tweets344
- Following65
- Followers875
- Likes541
Azeez retweeted
We said we'd update SparkGLM with the new weights. Here it is running on an advance fork of Atlas (@AtlasInference)! Better, stronger, faster, more KV cache. Give it a try!
github.com/Enntity/sparkglm?…
More magic for us local AI guys!
GLM-5.3-Flash-FP4-Local combines QAD-trained NVFP4 experts with selected MXFP8 attention projections. NVIDIA reports ~166 GiB of VRAM, validated on two DGX Sparks.
A similar quant gave us 10–15% faster decode in SparkQwen, with no noticeable quality drop. Looking forward to seeing what this does for GLM. SparkGLM update incoming next.
Thanks, @NVIDIAAI!
huggingface.co/nvidia/GLM-5.…
Azeez retweeted
If you want to run one of the best versions of Qwen 3.8 Flash Next on 2 DGX Sparks with @NVIDIAAI NVFP4 weights, blazing fast 4043 tok/s prefill and awesome decode and concurrency on a fully native inference engine (@AtlasInference), this is an excellent option! It’s a great local workhorse and what we often run for continuous entity cognition.
GitHub.com/enntity/sparkqwen
Azeez retweeted
There are a lot of ways to run big models on DGX Sparks and the landscape is always changing. If you're considering running one of our model packages, here’s an informal FAQ 🧵
First up: what’s with the arcade cards? We think they’re cool. That's all. :D
Azeez retweeted
We told you we'd keep improving GLM 5.3 Flash on 2 DGX Sparks! We increased the KV cache size from 1.38M -> 2.36M tokens, a 71% increase, and enabled concurrent 1M token context profiles. We did it without changing a single output bit. Greedy output is also now identical whether a request runs alone or beside three others. Four concurrent ~204K-token sessions answered every cached follow-up exactly. And it all works with our long term NVMe cache offload. Receipts in the repo.
If you have 2 DGX Sparks and you want to run a near-frontier coding model that balances quality, speed, and concurrency, you should try SparkGLM. It runs the GLM 5.3 Flash NVIDIA NVFP4 weights on an advance fork of Atlas (@AtlasInference). It's now our daily driver - we love it and are improving it constantly.
Azeez retweeted
Qwen 3.8 Flash Next is another of our favorite models to run on 2 DGX Sparks. We use it all the time for cognitive processing because of its excellent speed and superb concurrency, and we have an optimized version that, like our popular SparkGLM, runs the NVIDIA NVFP4 weights on an advance fork of Atlas (@AtlasInference). If you like Q38FN you should give it a try!
Azeez retweeted
If you have 2 DGX Sparks and you want to run a near-frontier coding model that balances quality, speed, and concurrency, you should try SparkGLM. It runs the GLM 5.3 Flash NVIDIA NVFP4 weights on an advance fork of Atlas (@AtlasInference). It's now our daily driver - we love it and are improving it constantly.
Azeez retweeted
@NVIDIAAI Nemotron Lightning served with @AtlasInference is definitely something you should try if you have a DGX/RTX Spark! 🔥🔥🔥
Nemotron with DSpark is supported on Atlas with incredible concurrent speeds 🚀
Try the one-line recipe here:
sparkrun run @atlas/nemotron-3.5-lightning-30b-a3b-nvfp4-dspark --hosts localhost
Thanks to @mr_r0b0t for many contributions on this work (using his bench below too!)
Azeez retweeted
Preview of what I’ve been working on:
@AtlasInference standard AR (no spec config) concurrency ladder 🪜
@NVIDIAAI Nemotron 3.5 Lightning NVFP4
Azeez retweeted
Application start -> API Ready: Median 33.906 SECONDS
@AtlasInference / @NVIDIAAI
Nemotron-3.5-Lightning-NVFP4
Azeez retweeted
@AtlasInference just took the single-Spark decode crown for Qwen3.8-27B.
30+ tok/s server-measured on one DGX Spark:
32.0 / 29.8 / 30.5 across 3 runs. Deterministic replication across labs, same day.
Same protocol, same checkpoint, same box on vLLM: 28.2. Atlas wins decode with thinking on too (24.1 vs 23.1).
And 30 looks like the floor, not the ceiling: this engine's DFlash drafts already hit 37.5 tok/s on Qwen3.6, the 3.8 drafter is in training, and a prefill fix showing +80–165% is in test. The Spark decode ceiling is moving weekly.
github.com/Weschera/Atlas-DG…
Azeez retweeted
Grok 4.6 is 🔥
Cooked up a custom kernel to get @NVIDIAAI Nemotron 3.5 Lightning + DSpark implemented using @AtlasInference! Testing ongoing but c1 decode is looking solid 👀
Azeez retweeted
@Alibaba_Qwen can we get the amazing @AtlasInference added to the day-0 list for engine support? This is a fabulous project aimed at supporting the GB10 community who so loves your models. They recently received @NVIDIAAI backing and community support grows by the day!
Azeez retweeted
Qwen3.8 drops today, but we gave Qwen3.6-27B one last C32 run on 1 DGX Spark.
Atlas: 12,127 tokens in 50.44s, 240.4 tok/s
vLLM: 11,934 tokens in 53.71s, 222.2 tok/s
32k context
Atlas: +8.2% aggregate throughput.
VLLM wins TTFT: 1.24s vs 9.76s.
@AtlasInference is ready for 3.8.
Azeez retweeted
@AtlasInference + Qwen3.6-27B NVFP4 on ONE DGX Spark.
128 concurrent sessions. 759/759 succeeded.
Peak aggregate: 277.8 tok/s @ C128
Same box. Same model.
Video = median C128 cell replayed from measured
TTFT / tokens / decode: 45,517 tokens in 164s wall.
Stack:
• Atlas main @ 5b32437 (PR #388 profile)
• centml/Qwen3.6-27B-NVFP4-W4A4-mlpinf
• MTP drafts=3, prefix cache, BF16 KV
• 1× Spark only (nobody else’s nodes)
C32 is where I’d actually park agents.
C128 is capacity porn — TTFT p50 ~37s.
@NVIDIAAI
Azeez retweeted
Got you covered bro, 27B is landing this week
Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b
Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b
Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b
Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b
Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b Qwen 3.8 27b
We believe in you!!!!
Azeez retweeted
Updated @AtlasInference numbers
27B NVFP4 on one DGX Spark:
My generic config: 14.8 tok/s
Their official recipe: 19-22
right on their claimed 21.78
Their real lane:
boots in ~1 min (vLLM ~4, SGLang ~6) and runs 97 tok/s solo on the 35B
Azeez retweeted
Rust製のInference Engine (atlasinference.io/), 起動が早くて vLLM に戻れなくなりそう。
Azeez retweeted
Most people that run local inference know about llama.cpp, vLLM, SGlang but very few know about this pure rust code inference engine named Atlas Inference dedicated exclusively to DGX Spark.
Made by a much smaller team than the regular engines we used to run, but with very promising performance.
Let's give the a shout-out!
github.com/Avarok-Cybersecur…
Azeez retweeted
If you haven’t tried Atlas yet you should.
