Alex D retweeted
People who have worked on actual lidar can immediately tell the “lidar” image on the right is artificially generated.
“Lidar” gives depth returns, not semantics, so the “cat” would not be brighter than the ground. It would actually blend into the ground.
Nor would the streetlight register as a bright orange blob.
Alex D retweeted
PSA if you’re running DeepSeek/Qwen Flash with SSD-backed Engram/PLE tables on a DGX Spark: check your NVMe interrupt coalescing.
Found NVIDIA ships a boot service that enables it specifically for Samsung, Kioxia and Micron drives. My Samsung PM9E1s default to OFF. NVIDIA turns it ON.
Tested both Samsungs. 20,000 random 4K reads, QD1, direct I/O:
coalescing ON: ~200µs
OFF: 55–57µs
back ON: ~200µs again
That’s ~3.5× the latency because of a boot-time setting.
If your token generation is waiting on uncached Engram lookups, this can fuck up your latency. A PCIe 5.0 sticker and a giant sequential bandwidth number won’t save you.
My first 4 sparks have a no name PCIe 4.0 SSD and my newest 2 came with a Samsung PCIe 5.0. But when I was actually testing SparkNest out, it was preferring the old devices for engram paging into the mmapped backend… because the Samsung drives get configured to buffer 8 read response interrupts… stupid… @NVIDIAAI time to delete that customization script now the new models need fast latency random read.
Check:
sudo nvme get-feature /dev/nvme0 -f 8 -H
The service is nvidia-nvme-interrupt-coalescing.service.
This is measured storage latency, not a claim of 3.5× more tokens/sec. Cache hits, batching and prefetching matter. But check this before blaming the model or buying another SSD. For low free memory hosts the impact is higher cuz the mmapped PLE can’t hold the whole engram table.
Amazing work.
Ran your Qwen3.8-Flash-Next EXL3 3.05bpw recipe on a single RTX 3090. It reproduced, and having your reference numbers published is what let me tell a correct setup from a merely running one. Thank you for that.
What I measured:
Ready in:
195s warm
345s cold
21,416 MiB at ready, against the 21,562 you list
56.5 tok/s decode, single user (your 62.7 is pure decode, mine includes prefill)
Same card, same model family through llama.cpp: 23.7 tok/s unpruned IQ3_S, 34.2 tok/s pruned IQ1_M
Your EXL3 path is 1.65–2.4x faster, at higher precision than either
Two flags worth adding to the documented launch:
--tool-call-parser qwen3_coder — without it the model emits well-formed markup and SGLang returns it as plain content with finish_reason=stop. Nothing reports an error, so an agentic client just sees a model that will not call tools. Adding it gives finish_reason=tool_calls with parsed arguments, holding with thinking on, thinking off, and tool_choice=required.
--reasoning-parser qwen3 — puts the thinking block into reasoning_content instead of leaving the closing tag inline in the content.
Neither flag changed memory use or decode rate. The server also runs clean with --cap-drop ALL and --security-opt no-new-privileges, nothing added back, and works as a llama-swap container entry so it swaps alongside GGUF models on the same card.
Just doubled the prompt processing speeds from 539 token/s to 1290 tokens/s.
Qwen3.8-Flash-Next on consumer GPU is now blazing fast.
Next on the list is further optimizations on kernels and swapping which should boost results by 3X-4X.
github.com/Niko1221/Strata
Alex D retweeted
Ayn Rand was just casually giving a lecture when someone hit her with, ‘Why do you hate poor people?’
Her response was way more savage than you’d expect.
Alex D retweeted
ok everyone, i took one for the team
here's the data we all wanted to see - real token value of each LLM subscription, empirically measured through usage on my real subscriptions
- supergrok heavy has now become the highest value at $12k worth of tokens (40x ROI)
- the $200 plan from openai and anthropic roughly give the same amount of token value, $7k give or take
- cursor ultra's ROI is the lowest among mainstream subscriptions, roughly half of OAI or Ant. for individual consumers i'm afraid this no longer makes sense to purchase
this was done in a clean environment. i ran each provider across 5 whole % points of my quota through repeated eval tasks in open source repos, and measured the total token value from all the agent sessions based on API pricing
caveat 1 - this only measures their coding agent quota (which is the majority of the offered value), and does not include things like chat bot or grok bot
caveat 2 - none of the providers commit to never changing this, so this is a snapshot of what the offering looks like today. i cannot guarantee it won't change again
caveat 3 - the token value assumes you exhaust your subscription completely, which may not be realistic in real world usage
Alex D retweeted
The fact that this dude went on ANDERSON COOPER to talk about how AI is going to kill us all - that’s all you need to know. This is 100% a CCP-paid op. Bookmark this.
