Home lab + self-host enjoyer · inference · virtualisation, networking · hardware enthusiast · building and breaking my own infrastructure. Retired Smite Pro.
United Kingdom
Joined April 2013
- Tweets8.5K
- Following1K
- Followers7.1K
- Likes12.5K
Pinned Tweet
Flash-Next v2.5.1 is out: Qwen3.8-Flash-Next (125B MoE) on 4× RTX 3090, no NVLink.
• Agent follow-ups resume from cache: 0.96 s to first token
• 8 agents at once: 92.5 % cache reuse
• 925K-token KV pool (+15 %), 3 full 262K sessions resident
Swift 1.5 build updated too 👇
Engine + Merlin checkpoint: github.com/halt95/qwen38-fla… · huggingface.co/halt95/Qwen3.…
Swift 1.5: github.com/halt95/Swift1.5-q…
Genuinely surprised at how much traction my Qwen3.8 Flash-Next recipe/kit has gotten before i even tweeted, I put it on github originally to have another location to store proper production candidates. Almost 1.2k downloads on huggingface already 😅
🇬🇧HUGE: UK government publishes official rules on AI usage as part of its advice on reducing environmental impact -
- Use AI only when necessary, check if a spreadsheet/ search engine can do the job before using AI.
- Choose the smallest model capable of handling the job, such as Gemini Flash instead of Pro or GPT Instant instead of Thinking.
- Prompts should be kept short, with as few as possible used.
- Any AI-assisted work should be personally checked for accuracy.
The guidance also says it should be disclosed when AI helped create or edit content.
Further improved Running Qwen3.8 Flash-Next:
Updated to vllm 0.30
Opt-in Repeatable greedy output
Still 3 x 262k sessions running at once in the kv pool
Opt-in GPU/NUMA auto-select for other box configs
Same 160-175 tok/s decode
This one is a stability focused release/update.
Usually tweet once in a blue moon to complain at a company. Deciding to shift gears.
Finally managed to get Qwen3.8-Flash-Next running on 4x RTX 3090s:
807K KV pool, three full 262,144-token sessions at once
162-176 tok/s single stream
Prefill flat 10K to 261K
No NVLink, 220 W
ALT Benchmark table: Qwen3.8-Flash-Next on four RTX 3090s. 806,792-token KV pool, three 262K sessions resident, 162 to 176 tok/s decode, 5,122 tok/s prefill at 261K. Compared against v1 row by row, with one row published as a FAIL.
Usually tweet once in a blue moon to complain at a company. Deciding to shift gears.
Finally managed to get Qwen3.8-Flash-Next running on 4x RTX 3090s:
807K KV pool, three full 262,144-token sessions at once
162-176 tok/s single stream
Prefill flat 10K to 261K
No NVLink, 220 W
ALT Benchmark table: Qwen3.8-Flash-Next on four RTX 3090s. 806,792-token KV pool, three 262K sessions resident, 162 to 176 tok/s decode, 5,122 tok/s prefill at 261K. Compared against v1 row by row, with one row published as a FAIL.
TP2 x PP2 + expert parallel, both token-embedding tables out of VRAM into pinned host RAM, read inside the cudagraph. MTP K=3.
Preregistered gate, 5 boots per arm. Full bench card in the repo & fails included.
Repo: github.com/halt95/qwen38-fla…
Checkpoint: huggingface.co/halt95/Qwen3.…
Does anyone still use this? I feel like every once in a while when i check, nothing gaming related or people i follow are in my feed, is it the algorithm?
This is what i get up to these days ez kill secured 40 kills with a meteor youtube.com/clip/UgkxoF2eTJ8… via @YouTube
Sad sight to see, thanks to those that poured their heart and soul into the game and shoutouts to the esports scene, past and present. hope you make it out there.