@deepu105@bsky.social 🍉 🇵🇸 @jhipster co-chair. Developer 🥑 @okta. Polyglot dev/Speaker/Author. @Java_Champions. Java, Rust, JS, DevOps. ADHD.
The Netherlands
Joined November 2009
- Tweets11.8K
- Following1.2K
- Followers4.6K
- Likes17K
Pinned Tweet
Today I'm releasing LlamaStash 0.0.2: a zero-overhead, terminal-native launcher for llama.cpp.
One Rust binary that's a TUI, a CLI, a daemon, and an OpenAI-compatible proxy.
Demo below 🧵
Qwen3.8-Flash-Next vs Claude Opus 5.5: Six-way benchmark on a real Rust feature
Opus xhigh: 87%, $17.78, 43 min
Flash-Next medium, run locally: 69%, $0.32/$0.02, 45 min
All 12 PRs pass the end-to-end test.
deepu.tech/qwen3.8-flash-nex…
ALT Results table, six arms: Claude Code + Opus 5.5 xhigh 6.94 of 8 (87%), 43.5 min, 17.78 dollars; Opus medium 6.25 (78%); Claude Code + Flash-Next medium 5.5 (69%), 45.3 min, 0.32 dollars; Pi + Flash-Next xhigh 5.44 (68%); Claude Code + Flash-Next xhigh 5.38 (67%); Pi + Flash-Next medium 5.06 (63%), plus token counts per arm.
LlamaStash v0.9.0 is out 🦙
Highlights:
- 'llamastash session claude' points a coding tool at one local model for one run, with nothing written to the tool's config.
- RECIPES.md holds tested configs for llama.cpp, vLLM, SGLang and the generic engines.
llamastash.dev
LlamaStash v0.8.0 is out 🦙
Highlights:
- Engines with no web UI of their own (vLLM, SGLang, generic) now get a chat page at '/ui'.
- The request log adds prompt processing per row and 'Agg tok/s', which counts the wall time concurrent streams share.
llamastash.dev
LlamaStash v0.7.0 is out 🦙
Highlights:
- Proxy request log and metrics: 'llamastash requests', a TUI tab, and '/metrics' in Prometheus format.
- 'proxy.aliases' maps a model name a tool hard-codes to a local model.
llamastash.dev
Just published pi-automode-classifier, an auto mode plugin for the Pi coding agent that uses Jev or Kev/Laya (running locally) to classify commands.
Risky commands need my approval before they run.
pi install npm:pi-automode-classifier
github.com/deepu105/pi-autom…
I gave the same complex Rust feature to Opus 5.5 and Qwen3.8-Flash-Next running locally. Opus did it 7 times faster but Qwen made the better PR which eventually got merged
Full writeup deepu.tech/local-ai-qwen3.8-…
LlamaStash v0.6.0 is out 🦙
Highlights:
- a request that doesn't fit now unloads idle models to make room, and 'presets save --idle-ttl 0 --preload' keeps a model warm.
- Claude Code's /effort now reaches llama.cpp models.
llamastash.dev
I benchmarked engines for Qwen 3.8-Flash-Next on Strix Halo (Flow Z13, 128 GB). Halogen with its native weights is fastest at 39-40 t/s decode, then gufo and CIRU. But Halogen is closed source. gufo is open source and loads 4x faster from cold.
redd.it/1wu0m53
LlamaStash v0.5.0 is out 🦙
Highlights:
- generic backend: any OpenAI-compatible server you declare in config.yaml becomes a managed model. I run Halogen and gufo this way now.
- both beat llama.cpp on Qwen3.8 Flash-Next.
llamastash.dev
LlamaStash v0.4.0 is out 🦙
Highlights:
- SGLang backend next to vLLM for safetensors repos. 'start owner/repo --backend sglang' picks it per launch.
- safer memory checks on unified-memory hosts, so a launch that would freeze the machine is refused.
llamastash.dev
Qwen 3.8 replaced Claude Opus for my coding. 27B and Flash Next on a 128GB Strix Halo laptop, no cloud subscription. Flash Next scores 40 on the AA index against 42 for Opus 4.8. Decode 10-15 tok/s, prefill is the real cost.
deepu.tech/local-ai-qwen3.8-…
LlamaStash v0.3.0 is out 🦙Highlights:- one model, several copies, each under its own name. 'start qwen3 --name coder', and it answers to qwen3@coder on the proxy, CLI and TUI.- 'llamastash run model.yml', a preset file you can commit.llamastash.dev
LlamaStash v0.2.0 is out 🦙
The big change: a vLLM backend. The safetensors repos in your HuggingFace cache were invisible before; now they show up in list, launch from the TUI, and answer on the proxy. llama.cpp still owns GGUF.
llamastash.dev
LlamaStash v0.1.0 is out 🦙
The big change: MTP speculative decoding, on by default. Roughly 2x faster decode on models that support it.
Also new: pick a CUDA/ROCm/Vulkan build per launch, and one JSON shape across list and show.
llamastash.dev
How much local LLM can you run on an AMD Strix Halo with 128GB memory?
I managed to fit DeepSeek v4 Flash 284B and Gemma 4 E2B on GPU, Whisper and Qwen3.5 4B on NPU.
#strixhalo #amd #deepseek #llamastash
LlamaStash v0.0.6 is out 🦙
A experimental ds4 backend runs @antirez DeepSeek-V4 GGUFs through DwarfStar (ds4)
Plus: Lemonade on by default, and saved presets that auto-apply.
llamastash.dev
#ds4 #AI #deepseek
LlamaStash v0.0.5 is out 🦙
New: named launch presets. Tune a model's launch knobs once, name them, reuse them, per-model or per-arch. They live in plain config.yaml, so you can hand-edit, comment, and commit them to your dotfiles.
llamastash.dev
LlamaStash v0.0.4 is out 🦙
- Auto launch is now the default: llama.cpp's --fit sizes context and GPU offload.
- A browser UI on a stable port
- Anthropic Messages API support.
llamastash.dev