Local inference, quants, Hermes, whatever fits this box. Flags, tok/s, what OOM’d.
Eagle, Idaho
Joined September 2026
- Tweets17
- Following89
- Followers14
- Likes20
Pinned Tweet
Cut down your experimenting time and have more fun testing and optimizing your local box w/ LLama.cpp Server and Hermes Desktop!
@ggerganov
@Teknium
@NousResearch
xitter.cf/boxweights/status/2105…
If you're like me an are constantly trying to test and optimize your local box then you'll enjoy my llama-server Manager plugin for @NousResearch #Hermes Desktop.
This is tailored to my workflow but it can be refactored to suit your needs easily.
🪄 Here's what it does ✨
👇
Box Weights retweeted
Cut down your experimenting time and have more fun testing and optimizing your local box!
@ggerganov
@Teknium
@NousResearch
xitter.cf/boxweights/status/2105…
If you're like me an are constantly trying to test and optimize your local box then you'll enjoy my llama-server Manager plugin for @NousResearch #Hermes Desktop.
This is tailored to my workflow but it can be refactored to suit your needs easily.
🪄 Here's what it does ✨
👇
Box Weights retweeted
We’re a digital commerce shop, but after hours our Founder & CTO is still pushing boundaries and building the tooling he actually uses.
Want to see the raw code, local AI setups, fine-tuning trials, and errors? Follow @boxweights 🔥
Damn this is awesome! I just hit 196 tok/s sustained on my RTX 4090 with Qwen3.8 Flash Next Coder, using a real-world coding task in my @NousResearch Hermes Agent Desktop
* 23.9GB VRAM packed with 8501 experts, KV Cache Q8, Threadripper 5955WX + 247GB RAM
Cut down your experimenting time and have more fun testing and optimizing your local box!
@ggerganov
@Teknium
@NousResearch
xitter.cf/boxweights/status/2105…
If you're like me an are constantly trying to test and optimize your local box then you'll enjoy my llama-server Manager plugin for @NousResearch #Hermes Desktop.
This is tailored to my workflow but it can be refactored to suit your needs easily.
🪄 Here's what it does ✨
👇
Cut down your experimenting time and have more fun testing and optimizing your local box! @ggerganov @Teknium @NousResearch @Alibaba_Qwen @googlegemma @NVIDIAGeForce @Supermicro @ubuntu
If you're like me an are constantly trying to test and optimize your local box then you'll enjoy my llama-server Manager plugin for @NousResearch #Hermes Desktop.
This is tailored to my workflow but it can be refactored to suit your needs easily.
🪄 Here's what it does ✨
👇
If you're like me an are constantly trying to test and optimize your local box then you'll enjoy my llama-server Manager plugin for @NousResearch #Hermes Desktop.
This is tailored to my workflow but it can be refactored to suit your needs easily.
🪄 Here's what it does ✨
👇
✅ Add, Edit, Delete Model Versions
✅ Star a Default Version for each Model
✅ Star a Default Model Version as your Main Model used
✅ Automatically Switch Models when Re/Starting without having to refresh Providers/Models manually
✅ Built-in monitoring, GPU, CPU, RAM, TOKs
👇
Damn this is awesome! I just hit 196 tok/s sustained on my RTX 4090 with Qwen3.8 Flash Next Coder, using a real-world coding task in my @NousResearch Hermes Agent Desktop
* 23.9GB VRAM packed with 8501 experts, KV Cache Q8, Threadripper 5955WX + 247GB RAM
Qwen3.8-Flash-Next first 32K tokens are 4%-10% faster, LOW RAM mode added (for low hardware).
New update here
github.com/Niko1221/Strata/r…
