Cofounder & engineer. Running local AI, building things with it, and sharing what I learn.
Joined April 2022
- Tweets9.5K
- Following771
- Followers8.1K
- Likes16.3K
netrunner retweeted
Issue 3 of AGI Checkpoint is out 🎉
A few highlights for this week 🌟
・@MiniMax_AI put 428B M3.1 Flash weights up at $0.15/$0.60 per million tokens
・@Recursive_SI shipped Flint, a local coding model at
125+ tok/s on a MacBook Pro
・@plotarmordev got TensorFold cloning voices locally
・@ch0mpaa ran TensorFold on a Snapdragon
・@justinn_builds pushed REDSNOW to 275k context on two Sparks
・@Blackwellboy hit 2,008 tok/s prefill with BLACKFOLD
Follow these community builders, they are killing it 🔥
netrunner retweeted
sparkDash 2.70 is out ⚡️
This is a MAJOR release.
New pages
- Cost compare: see what your fleet's token usage would cost on any hosted model, including cached-prefill savings
- Temps: daily and weekly GPU temperature graphs for every unit
- Metrics history per unit (48h and 30 days, opt-in), with a History panel on the unit page
Engines & hardware
- New engines: Strata & Flare (tok/s and benchmarks) and Radiance (live rates)
- Intel Arc GPU support
- AMD GPU telemetry
- GPU clock cap shown on each unit, and you can set it from Edit unit. If the unit has no passwordless sudo, the dashboard asks for the password, or you can run it on the Spark yourself
Benchmarks
- Decode bench: pick the temperature (default 0.7) and turn thinking on or off for prose. Both show on copied results and the share card
- Tool Eval Bench "Show card": a shareable image of a finished result
- Comparison card: tick two runs and get one image showing the difference
- Compare a run from one Spark with a run from another, plus an expandable scenario list
- MAX_ACTIVE_BENCH_JOBS setting to raise the fleet benchmark limit
Overview & monitoring
- Overview cards list every loaded vLLM model, plus CPU temp, CPU power and the engine's parallel slot count
- Token totals for the last 24h, 6h and 1h, with one row per unit and model
- Optional card showing inference request activity
- Alerts to ntfy, Discord, Slack or a webhook (opt-in), plus a tab badge and desktop notifications
Other
- "Run both" button in the Models panel
- "Back to Overview" link on every unit page
- Published container image
Link below.
netrunner retweeted
I finally have some numbers behind my DeepSeek V4.1 Flash 1017 quant. EAQ v1 is 11% to 38% closer to the original than LibertAIDAI's EXL3.
Seems like the gap is bigger on longer context. To be clear, LibertAIDAI's quant is NOT bad. It's a solid quant, and it gave me a strong baseline to beat.
Already in the works: EAQ v1.1, v1.2 and finally EAQ v2.
Full details in the model card:
huggingface.co/Mia-AiLab/Dee…
netrunner retweeted
D1 decision model by @liquidai now runs on your DGX Spark, up to 4.3x faster than stock llama.cpp!
Ask it 3 questions about one message and all 3 answers come back in 27ms instead of 119ms.
The video is those 27ms, slowed down 100x:
github.com/plotarmordev/d1-t…
netrunner retweeted
Fourth Ablit Recipe now in the same repo
Thank you @MiaAI_lab
MiaAi_Lab - DeepSeekv4.1Flash-1017-EAQ-EXL3-2.9BPW now abliterated.
HP repo: huggingface.co/drowzeys/keys…
Third Ablit Recipe now in the same repo
Thank you @jayleaton
LFG!!!>.
Jayleaton - DeepSeekV4.1 Flash -1017 - EXL3- 2.8 BPW is now abliterated inside the HF repo... who is next...
100% abliterated + full C.S. capability enabled
huggingface.co/drowzeys/keys…
netrunner retweeted
The 'old' vLLM recipe for DeepSeek v4.1 Flash *should* work with the new 1017 release. I haven't tried it though. Just try replacing the model.
That being said, I'm working on a TensorFold recipe.
vLLM recipe here:
github.com/MiaAI-Lab/DeepSee…
netrunner retweeted
TensorFold 1.0.12 is out with Nemotron 3.5 Super VL 120B vision on 1 or 2 DGX Sparks, Qwen3.8-27B vision and Nemotron oQ on Macs, LFM2.5-24B, plus source previews for GLM-5.3-Flash NVFP4 across Sparks and Intel Arc.
github.com/ashhart/TensorFol…
Thanks to the 20 people in 1.0.12:
alenperic, BHCC2025, Bizuayeu, BlivionIaG, BobClawblaw, cwschroeder, harrisonfriia, heitke, jayleaton, jishnuvenugopal, jschmied, kelchm, liumorrisclaw, MiaAI-Lab, Mirrdhyn, mrv777, philip-pentatonic, plotarmordev, sethforprivacy, ThinkOffApp.
And thanks to jkuepker and SvangenStudios for runs on their own hardware, and grantoverton, whose oQ work this release builds on.
@petruspennanen @plotarmordev @volatilemarkts @MiaAI_lab
netrunner retweeted
I’ve been using this for my tests 🔥
Do you run local models?
You've probably tried a setting or an update and wondered: did that make it faster, or does it just feel faster?
I built a free tool that tells you. Test, change something, test again. Faster, slower, or too close to call 👇🧵
github.com/plotarmordev/theg…
netrunner retweeted
sparkDash 2.30 is out ⚡️
Windows PCs as units
Monitor any Windows PC with an Nvidia GPU over its built-in OpenSSH. That covers RTX Spark machines, which run Windows, and any other Windows box with an Nvidia card.
RoCE / RDMA panel
- Link and traffic: link state, rate, MTU, RX/TX traffic and interface errors.
- PFC and QoS: which priorities have PFC on, the trust mode (PCP or DSCP), the DSCP map when DSCP trust is on, and the global pause settings.
- Counters: a collapsible list of RDMA counters, plus ethtool discards, CRC errors and pause frames. It shows the change since the last sample.
What else is new
- Installable mobile app (PWA) with an in-app install prompt
- Permanent monthly energy history, with a month/year view
- Timezone settings
- Prometheus /metrics export (opt-in)
- Per-GPU graphs on multi-GPU hosts
- Model picker in the decode benchmark for multi-model servers
- Per-Spark llmHost for LLM probes and benchmarks
- Restart sparkDash from the UI when the fleet changes
- Tooltip on “Partial coverage” naming the nodes with no fresh reading
- FreeToken KV cache, Running and TTFT tiles on the LLM panel
- README guide for running the dashboard on a non-Spark machine
Fixes
- SSH tunnel fallback for benchmarks and Showcase
- Poll interval setting now really sets the polling rate
- "Frontend not built" after a Docker upgrade
- Blank page on a token-protected remote bind
- Model launchers no longer inherit PORT=5555 or the access token
- Two-unit Overview row stretches to the full width
- Efficiency on the Overview card keeps its digits
- Add/Edit dialog wording follows the unit type
- Clearer SSH error when a Windows PC is added as a Linux unit
- Dev server listens on loopback only
30 issues and PRs closed.
Thanks to every contributor who opened an issue or PR, or shared feedback here on X.
netrunner retweeted
TensorFold 1.0.3 lands with a few cool things.
- Living Weights (beta with Nemotron on Metal) and coming to CUDA in 1.0.4
- Nvidia Ampere (beta) for Nemotron, and native Qwen3.8-27B with DFlash2
- Conversation prefix caching
- Various bugs fixed and improvements
Twelve contributor PRs helped make this happen, so a massive thank you to @niklaslenz_ai and GitHub contributors akol1, jschmied, chaog992, AdrianBinDC and BobClawblaw, and everyone testing, reporting bugs and helping us improve TensorFold.
FYI, Living Weights is in very early beta and has sharp edges, but I think I’m on to something.
I’d love community feedback to make it better: github.com/ashhart/TensorFol…
@plotarmordev @petruspennanen @MiaAI_lab @volatilemarkts
tensorfold.dev
netrunner retweeted
As I said many times before, TensorFold is a real turning point for local AI. Just try it for yourself, we have a recipe for everyone!
All of my TensorFold recipes can be found here:
mia-ai.net/models?for=engine…
TensorFold by @ashxhart is one of our newest Inception startups and has been making waves in the local AI community. It's an open-source inference engine for running models on your own hardware. Finally got around to experiencing it for myself.
I'm running Qwen3.8 Flash Next in NVFP4 on it across my two DGX Sparks, and built a paper reader to put its concurrency to work. It splits an arXiv paper into its sections and reads them all at once, one request per section, then merges the results into a one-page brief tagged back to each source section. Per-request and combined tok/s update live as it runs.
netrunner retweeted
GLM 5.3 Flash on 2x DGX Sparks v1.10 is out ⚡️
Lots of PRs merged and bugs squashed!
Here are some highlights:
- Works with Claude Code & Anthropic SDKs directly
- Much faster at very long chats (500k+ tokens)
- Opt-in disk cache: long chats resume in 0.2 s instead of 11 s, even after a restart
- Sampling is now as fast as greedy decoding
Full changelog here:
mia-ai.net/models/GLM-5.3-Fl…
netrunner retweeted
I went through every GLM, Qwen, Kimi and DeepSeek release to predict the next release dates.
GLM (5.4 or 5.5): Oct 13-22
Qwen4: Oct 12 - Nov 15, with 27B as the priority
Kimi K3.1: mid Oct to mid Nov, then K4 around Jan
DeepSeek v4.1 Pro: Oct 20 v Nov 30, with V5 in 2027
China's Golden week ends tomorrow, and Chinese labs usually ship right after it.
The next 6 weeks are going to be busy for local AI 🔥
netrunner retweeted
Let's push these 3 DGX Sparks!
Running experiments with tool-eval-bench with various temp, top-p, top-k, parallel calls (1,4,8)
So far no great results, but in any case I'll share this to avoid others trying the same path in case 💪
Let's help each other 🚀
netrunner retweeted
Huge respect to the DGX Spark community.
We had the intuition early on to see the potential in this little machine and instead of waiting for someone else to unlock it, we worked together to push it further, optimize it, experiment, share what worked, and bring out the best in it.
Now the world is starting to notice just how capable DGX Spark really is.
But as James McPherson said, maybe the bigger story isn’t just how great the hardware is, it’s how great this community has become around it.
Let’s not lose sight of that.
Keep building. Keep sharing. Keep pushing the limits.
The hardware was the spark. The community is the accelerator.
@mr_r0b0t @Blackwellboy @NeoAIForecast @Tech2Wild @MichaelGannotti @aijoey @BosonJoe @with_gene2626
@volatilemarkts @ivanfioravanti @thatcofffeeguy @WescheNex1q @SpaceTimeViking @yume_arasaki @GordianKnotDev @LytninCrypto @light_foundry @MichaelHutu @StefanMaier @anemll @Q_Beaux @SixZzshOtRipZz @GumbiiDigital @GBminA @ayayalar @chrittazz @jonathan_conn @Dave_Charland @MarkSunner @haydonryan @Defenestrator22 @kkiran @NsitnovSitnov @DJLougen @imost @wikiwayne @ViC305 @ashxhart @jvr0x @majewskizby @plotarmordev @justinn_builds , @MiaAI_lab @0xBakeer @JamesMeadlock @tysn_dev @mmastrac @CK2084 @jnardiello @dangerm00se
netrunner retweeted
The new GLM 5.3 Flash EXL3 quant is calibrated differently. You don't have to believe me, try it yourself.
HF link:
huggingface.co/Mia-AiLab/GLM…
Recipe:
mia-ai.net/models/GLM-5.3-Fl…
I have been using Mia’s quant for GLM 5.3 Flash and I can tell anecdotally that the error rate has gone down. My setup is 60% data engineering and it does a lot of tool calls and it’s getting it right the first go and just powers through the analysis.
I also built a feature in typescript for a client and the adversarial reviews came back with less errors and shipped faster 👍🏻
netrunner retweeted
Everything you need to know about DeepSeek V4.1 Flash, from a guide that fits in your pocket and four DGX Sparks that fit on your desk.
Six experts out of 384. Twelve operations per byte. A million tokens, mostly remembered.
Bring your towel. Best enjoyed with something almost, but not quite, entirely unlike tea.
netrunner retweeted
There are people who leave a mark on the world, and then there are people who change it entirely. Steve changed the world and so many lives in the process. He certainly changed my life forever. His spirit lives on in everything he created and everyone he inspired.
ALT A photo of Steve looking at the camera wearing glasses and a black turtleneck.
