ata (@atasrb) header
ata (@atasrb) profile picture

measuring models and agents, making them run faster. local ai on 2x DGX Spark, RTX 4090, 64GB M1 Max

Joined August 2012
Pinned Tweet
What fits on one DGX Spark, and what needs two. Only models I ran myself, one user, real prompts. One Spark: Qwen3.8-Flash-Next at 66 tok/s on JSON. Nemotron 3.5 Bolt at 96 with its draft model. A dense 27B at 11. Two Sparks: DeepSeek-V4.1-Flash (763B) at 103 tok/s, but only 7-9 GiB stays free on each Spark. MiniMax-M3.1-Flash (428B) at 27-29 with 2.15 bpw, 20-22 with 3.06 bpw. Qwen3.8-Flash-Next goes from 66 to 99.
6
1
1
36
4,792
I ran the same six coding harnesses with three great local models you can run on DGX Spark, 198 takes in total. The fastest harness depends on the model. opencode is first with Qwen3.8-Flash-Next, Pi with GLM-5.3-Flash and DeepSeek-V4.1-Flash. I would not judge a new model in only one harness.
4
3
2
12
1,273
DeepSeek Harness is last with Qwen, but there too the tests pass in 2 to 6 minutes. In 6 of 11 takes the model then keeps going: the box has no browser, so it writes its own DOM simulator to test the page's drag and drop. With GLM it is second. One task for all: a task board from 15 given tests, in a container with no internet. Median of 11 takes each, thinking off. Recipes: github.com/atakann/dgx-spark…
2
134
Sage S2.1 Flash runs on two DGX Sparks. It is a 320B MoE with 18B active on the GLM-5.3-Flash architecture, in @MiaAI_lab's 4 bpw EXL3 build with TensorFold by @ashxhart. I get 55 tok/s on code and JSON, and 37 tok/s with a 128K prompt. Next to GLM-5.3-Flash on the same pair and the same engine, it scores higher on GPQA (75.8% vs 68.7%) but runs slower: 55 vs 83 tok/s on code.
🚀 Introducing Sage S2.1 Flash, our new multimodal model designed for research tasks requiring multiple rounds of analysis and verification. 1⃣Combines scientific data understanding with information retrieval, coding, and tool use. 2⃣Uses specialized software to perform analyses, verify results against intermediate outputs and visual feedback, and adjust subsequent steps accordingly. 3⃣Built on the unified representation architecture of Sage SciPerception, supporting more than 10 types of scientific input, including molecular graphs, crystal structures, protein sequences, and 3D protein structures. 4⃣Ranks first on MLE-bench Lite (90.3), OfficeQA Pro (72.2), Harvey LAB (25.0), and SpreadsheetBench 2 (37.1). 🥰Now supported by vLLM (@vllm_project) and SGLang ( @lmsysorg ) — with more ecosystem integrations on the way. 🤗 Model: @HuggingFace huggingface.co/internlm/Sage… @ModelScope2022 modelscope.cn/models/Shangha… 🤗 Try it now: discovery.intern-ai.org.cn/t…
1
11
818
GPQA Diamond 75.8% and IFEval 76.7%, thinking off, 16K reply limit. GLM-5.3-Flash is ahead on IFEval, 82.3%. 35 of 198 GPQA answers hit the limit and count as wrong. I reran four of them at 64K and three still did not finish. Same recipe as GLM-5.3-Flash, only the weights swapped: github.com/MiaAI-Lab/GLM-5.3…
82
Six coding harnesses, one local model, one task, two DGX Sparks. This time GLM-5.3-Flash on vLLM. Pi builds the task board first in 1:50, Qwen Code last in 3:29. All six pass the 15 tests, and the spread is much smaller than with Qwen3.8-Flash-Next. DeepSeek-V4.1-Flash was faster on the same two Sparks, Pi 1:11 there, and Pi won with both models.
2
2
7
1,000
The median take of 11 for each harness: Pi 1:50, DeepSeek Harness 2:08, opencode 2:18, oh-my-pi 2:47, Hermes Agent 3:02, Qwen Code 3:29. Replays at 8 times the real speed, reasoning effort low, no internet in the containers. Recipe: github.com/atakann/dgx-spark…
1
135
YOLO26s from @ultralytics on one DGX Spark: tiling finds 81 vehicles a frame against 29, at 35 FPS. The model sees 640x640 (a whole frame shrunk to 640 leaves the far cars a few pixels), so tiling runs it on 640 windows of the full-resolution frame. Green boxes are the ones only tiling finds. For a drone video I would not run it without tiling now.
3
1
6
533
Tiling is one argument in Ultralytics since 8.4.204: tile=640. TensorRT FP16. On 4K it is 32 windows a frame and 17 FPS, or 52 FPS with 1280-pixel windows. Only cars, trucks, buses and motorcycles are counted. On tiles the COCO weights also call some trees broccoli.
1
36
Hy4 flash in NVFP4, on day 0. Tencent's 220B open-weight model with 12B active now runs on two DGX Sparks. The 4-bit build is 165 GB. vLLM splits it across both Sparks: 36-38 tok/s on code and JSON, 256K context. It needed a small vLLM fix, issue in the replies. Better than expected for a model this size on two desktop boxes. Stories are slower, 17 tok/s, because the drafter misses more there. Weights and recipe in the replies.
Hy4 flash is now open-weight, including Hy4 flash-FP8 and Hy4 flash-Base. 224B (12B active), 1M context, natively multimodal. Fast, efficient and built for everyday productivity. It's yours now. And stay tuned for Hy4 coming up next! 🤗: huggingface.co/tencent/Hy4-f… 🐙: github.com/Tencent-Hunyuan/H… 📄: hy.tencent.ai/research/hy4-f… Released under the Tencent Hy License. See full terms here: huggingface.co/tencent/Hy4-f…
6
25
2,227
Weights: huggingface.co/atakans/Hy4-f… Recipe, one command for two Sparks: github.com/atakann/dgx-spark… (llm/hy4-flash-nvfp4-vllm-tp2) vLLM's Hy4 flash code stops on the Spark's GPU. The fix was small: the kernel already takes the model's attention sink, vLLM just did not pass it. The changes are in this issue: github.com/vllm-project/vllm… Experts are NVFP4 with NVIDIA ModelOpt, the Engram tables FP8, the DFly drafter is on. One user, thinking off.
3
234
DeepSeek-V4.1-Flash on two DGX Sparks, six coding harnesses, one task. Pi is first in 1:11, oh-my-pi 1:20, opencode 1:22, DeepSeek Harness 1:50, Qwen Code 1:51, Hermes Agent 2:11. All six finish much faster than with Qwen3.8-Flash-Next on one Spark. TP=2 is a big part of that.
5
3
31
3,070
DeepSeek Harness has a failing test on its first test run in 7 of 11 takes, Pi in 3, and each fix loop costs about 30 s. All 66 takes pass 15/15. Median of 11 takes for each, video at 8x speed. EXL3 on TensorFold, thinking off.
2
272
I think people underrate Qwen4 Max. Nobody should expect Fable 5.2 or Opus 5.5 from it. It is 1 point behind Muse Wonder and 3 behind Kimi K3.1 Pro, at 11% lower cost per task. It is 8 points ahead of Grok 4.7 and 17 ahead of Mistral Large 4. And most importantly, smaller models will be open source.
Alibaba has launched Qwen4 Max, gaining 9 points over Qwen3.8 Max on the Artificial Analysis Intelligence Index and landing 3 points behind Kimi K3.1 Pro at ~11% lower Cost per Task Qwen4 Max, using xhigh reasoning effort, scores 43 on the Intelligence Index, 9 points ahead of Qwen3.8 Max (xhigh) and 3 points behind Kimi K3.1 Pro (max, 46) Key takeaways: ➤ Qwen4 Max improves on every evaluation in the Index vs Qwen3.8 Max (xhigh, 34): HLE-Diamond +21 points (24% to 45%), AA-Omniscience +18 points, AutomationBench-AA +17 points, CritPt v2 +13 points, Terminal-Bench-Science +11 points, Terminal-Bench 4.0 +9 points, GDP.pdf +3 points, FrontierCode 2.0 +3 points and AA-Briefcase +82 Elo. ➤ Qwen4 Max costs $3.27 per Intelligence Index task, ~3x less than Qwen3.8 Max ($10.02) and ~11% less than Kimi K3.1 Pro ($3.68): This is driven by 25% lower per-token pricing than Qwen3.8 Max, 70% cheaper cache hits and lower token usage. Tencent Hy4 costs ~3x less per task ($1.03) than Qwen4 Max but scores 8 points lower on the Index. ➤ Qwen4 Max uses 39% fewer output tokens per task than Qwen3.8 Max: ~105k output tokens per Intelligence Index task vs ~171k for Qwen3.8 Max, ~72k for Kimi K3.1 Pro and ~78k for Tencent Hy4. Qwen4 Max uses fewer tokens, while increasing 9 points on the Intelligence Index ➤ Improvements in AA-Briefcase (knowledge work): On AA-Briefcase, our evaluation of agentic work across real business files, Qwen4 Max reaches 1686 Elo vs 1674 for Kimi K3.1 Pro and 1446 for Tencent Hy4. ➤ Qwen4 Max trails Kimi K3.1 Pro on Terminal-Bench 4.0 and Tencent Hy4 on FrontierCode 2.0: Qwen4 Max scores 48% on Terminal-Bench 4.0 vs 67% for Kimi K3.1 Pro and 37% for Tencent Hy4. On FrontierCode 2.0, @cognition’s private evaluation of real-world coding tasks, it scores 26%, level with Kimi K3.1 Pro (26%) and behind Tencent Hy4 (31%). Key model details: ➤ Context window: 1M tokens ➤ Multimodal: Text and image input with text output ➤ Pricing: $1.50/$4.50 per 1M input/output tokens on Alibaba's first-party APIs, with a $0.075 cache hit price. This is lower than Qwen3.8 Max across the board ($2.00/$6.00, with a $0.25 cache hit price) ➤ Availability: @Alibaba_Qwen 's first-party APIs
4
160
Edge0-145B from @Edge0_AI is Qwen3.8-Flash-Next rebuilt as a sparse MoE with 512 experts. One DGX Spark runs it at 36 to 39 tok/s. What I like: 52 GiB of 128 stay free, room for a second model. It needs one option for that, --mmap-experts: the stock launch left 11 GiB.
1
1
11
796
Recipe with the build for the GB10 and all the numbers: github.com/atakann/dgx-spark… GPQA Diamond 82.3% and IFEval 81.0% with thinking off. A 128K prompt reads in 119 s. The engine serves one request at a time.
2
147
Six coding harnesses, one local model, one task, one DGX Spark: opencode, Pi, Hermes Agent, Qwen Code, DeepSeek Harness and oh-my-pi. Qwen3.8-Flash-Next builds the same task board from 15 tests in each. In these takes it needs 2:29 in opencode and Pi, and 9:58 in oh-my-pi.
11
2
74
8,117
One take is not a ranking. Over 11 takes of each, the median goes from 2:44 in opencode to 11:07 in DeepSeek Harness, whose takes run from 2:10 to 30:01. All six replays run at 20 times the real speed. The containers have no internet. Recipe: github.com/atakann/dgx-spark…
1
8
986
I ran Alpamayo 2 Edge on a DGX Spark. It reads six cameras and writes a new driving plan about every second, with a sentence that says why. Out of the box a plan took 2.8 s. More than half of the gap was a file that the code read again for every plan.
Meet NVIDIA Alpamayo 2 Edge, a 2B-parameter reasoning VLA model designed for real-time, in-vehicle deployment on NVIDIA DRIVE AGX Thor. Generate driving trajectories and causal explanations powered by adaptive reasoning. Learn more 👉 nvda.ws/4k3vG7m Try it out on @huggingface 👉 huggingface.co/nvidia/Alpama…
5
10
569
Clips are from NVIDIA's PhysicalAI-Autonomous-Vehicles dataset. Green is the model's plan, white is what the car then drove. The video runs in real time: each plan shows up when the Spark finishes it. 2.8 s is the stock code, 1.0 s is with the file read once and NVIDIA's --accel tuned, at 5.9 GB of GPU memory.
2
67
Qwen3.8-Flash-Next on one DGX Spark built this task board in 2 min 29 s, with opencode and no internet. It got one prompt and a test file. The first run passed 1 of 15 tests, then the model read the errors and fixed its code until all 15 passed.
6
1
39
3,711
The agent ran in a closed container. Its only way out was the model on the Spark. vLLM with NVIDIA's NVFP4 weights, thinking off. 23 tool calls, 7,839 tokens. The video runs at 5x and the end is real time. Recipe: github.com/atakann/dgx-spark…
4
331