Run and train models locally with the Unsloth Desktop app. 🦥 https://xitter.cf/t.co/2kXqhhvdCD
San Francisco, CA
Joined November 2023
- Tweets801
- Following483
- Followers101K
- Likes15.1K
Unsloth AI retweeted
how do you fine-tune a decision model on your own data?
in these two notebooks, you'll learn:
• how to prepare the training data
• how to fine-tune our d1-3B decision model
• using @UnslothAI and @huggingface TRL
all within 30 minutes on a free-tier GPU.
unsloth: github.com/Liquid4All/cookbo…
TRL: github.com/Liquid4All/cookbo…
ps: how adorable are these halloween kitties? 👻
Unsloth AI retweeted
Video: Decision model trained in ~2 mins on a single DGX Spark using Unsloth! Qwen3.5-2B as base model.
Just follow instruction here: unsloth.ai/docs/basics/train…
Unsloth AI retweeted
We're co-hosting an open source party with @HuggingFace! 🤗🦥
Come join us, we'll be handing out @UnslothAI merch, showcasing our upcoming features and more!
It'll be a super fun night with lots of demos, DJs, food and more.
Join: luma.com/OpenTogether
Use code: NOSLOTHSHERE to get in.
Unsloth AI retweeted
We got @UnslothAI a DGX Station!
@DanielHanChen and @NaderLikeLadder checked out Unsloth’s new @Dell Pro Max with GB300 and talked about what comes next: support for more models, faster quantization, and more efficient reinforcement learning.
Unsloth AI retweeted
Friendly reminder that you can fine tune 500+ open source models in a free Google Colab
You can even upload PDFs/CSVs and turn them into usable synthetic datasets.
1. Open the Google Colab below
2. Run the blocks to install Unsloth Studio
3. Choose a model
4. Upload a dataset
5. You're good to go!
And you can of course export your model afterwards.
Unsloth AI retweeted
ICYMI, we're celebrating 1 BILLION+ downloads for @googlegemma 💎
How are developers actually using open models? @GoogleDeepMind’s @DynamicWebPaige caught up with devs and collaborators like @UnslothAI and @Qualcomm to hear how they’re building on-device tools, running local fine-tuning, and pushing multimodal breakthroughs.
Unsloth AI retweeted
Local GGUF inference is now up to 3.3× faster at long context lengths
We made GLM-5.3-Flash run 3.3x faster locally!
Local GGUF inference is now 1.6–3.4× faster with optimized decoding and bonus multi-token prediction.
Run 3-bit on 128GB setups via Unsloth Desktop or llama.cpp.
Guide: unsloth.ai/docs/models/glm-5…
GGUF: huggingface.co/unsloth/GLM-5…
Unsloth AI retweeted
⚡ Fine-tune and quantize AI models directly on NVIDIA Jetson.
Our new Jetson AI Lab tutorial shows how to use Unsloth and memory-efficient QLoRA to customize models, export quantized GGUF files, and run them locally with llama.cpp.
Follow hands-on examples for:
🔹 NVIDIA Nemotron 3.5 Lightning on Jetson AGX Thor
🔹 Qwen3.5-4B on Jetson Orin Nano
Start optimizing: nvda.ws/45YagAq
Unsloth AI retweeted
GLM-5.3 can now be run locally!
The 2-bit model retains ~81% accuracy after we shrunk it from 1.51TB to 239GB (-83% size).
Run on a 256GB Mac or RAM/VRAM setups.
GLM-5.3 is the strongest open model to date.
Guide: unsloth.ai/docs/models/glm-5…
GGUF: huggingface.co/unsloth/GLM-5…
GLM-5.3 is now open-weight.
Our most capable model for agentic coding and cyber defense is now available to download, run, and customize.
Weights: huggingface.co/zai-org/GLM-5…
Tech blog: z.ai/blog/glm-5.3
Unsloth AI retweeted
The VRAM barrier is officially dead.
I just ran Qwen 3.8 Flash Next (MoE) 125B A6B with a 250,000 context window on a single 24GB RTX 4090.
21 tokens/sec decode. 364 t/s prefill.
no mtp. no dflash. no kv cache quantization!
We are running datacenter models on consumer hardware.
Tested on Ubuntu 22 | CUDA 13.0 | PCIe 4.0 x16 | 110 GB DDR4 System RAM with a continuous 28k prompt across all runs.
### The Benchmarks & Scaling
# 1. Hybrid Offload (-ncmoe 40 @ 80k Context)
Offloaded 40 expert layers to the GPU, pushing VRAM to the ceiling.
./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 80000 --port 8080 -v --fit off -b 4096 -ub 4096 -ncmoe 40
Prefill: 383.85 t/s | Decode: 22.52 t/s
Footprint: 23.85 GB VRAM | 97 GB RAM
# 2. Full CPU MoE Offload (-cmoe @ 80k Context)
Pinned all 512 expert layers to DDR4 RAM (-cmoe), keeping attention on the 4090.
llama.cpp flags: (Same as above, replace -ncmoe 40 with -cmoe)
Prefill: 355.72 t/s | Decode: 20.84 t/s
Footprint: 11.66 GB VRAM (12GB+ VRAM freed up!) | 110 GB RAM
# 3. The 180,000 Context Run
Prefill: 357.75 t/s | Decode: 20.98 t/s | VRAM: 15.6 GB | RAM: 110 GB
# 4. The 250,000 Context Absolute Ceiling
./build/bin/llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf -c 250000 --port 8080 -v --fit off -b 4096 -ub 4096 -cmoe
Prefill: 364.29 t/s | Decode: 20.97 t/s
Footprint: 18.3 GB VRAM (Still ~5.7 GB of VRAM headroom!) | 110 GB RAM
### Key Insights:
-b 4096 -ub 4096: doubles the prompt ingestion from ~150 to 364+ t/s.
-cmoe Free Lunch: Shifting expert layers to DDR4 RAM slashes VRAM from 24GB to 11.6GB with virtually zero decode penalty (22.5 -> 20.9 t/s), enabling the 250k context ceiling.
Qwen 3.8 Flash-Next (UD-Q4_K_XL) is a massive 111.4 GB model split across 4 shards. To run this architecture, you must build from the experimental PR branch (#27742) by @danielhanchen:
git clone && cd llama.cpp
git fetch origin pull/27742/head:qwen-next && git checkout qwen-next
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native -DBUILD_SHARED_LIBS=OFF
cmake --build build --config Release -j $(nproc) --target llama-server
A single 4090 paired with 100 GB of cheap DDR4 RAM will comfortably serve production grade 125B inference.
While Qwen 3.8 27B (dense) still holds the crown for single 3090/4090 rigs, Flash Next proves 125B hybrid models are officially viable on consumer hardware.
Hugging Face GGUF link and complete performance telemetry graphs are dropped in the replies below.
GLM 5.3 Flash VS Qwen 3.8 Flash Next, which one takes the open weights crown this week?
Qwen3.8-Flash can now be run locally! 🔥
The 125B MoE model outperforms Claude-Opus-4.6 (Max).
Run on 75GB RAM via Unsloth GGUFs.
Qwen3.8-Flash-Next enables CPU RAM / unified mem setups to deliver near VRAM speeds.
Guide: unsloth.ai/docs/models/qwen3…
GGUF: huggingface.co/unsloth/Qwen3…
Unsloth AI retweeted
A high-performance 125B model now running locally on just 75GB RAM! Thank you @UnslothAI for the day-0 support.🥳
Qwen3.8-Flash can now be run locally! 🔥
The 125B MoE model outperforms Claude-Opus-4.6 (Max).
Run on 75GB RAM via Unsloth GGUFs.
Qwen3.8-Flash-Next enables CPU RAM / unified mem setups to deliver near VRAM speeds.
Guide: unsloth.ai/docs/models/qwen3…
GGUF: huggingface.co/unsloth/Qwen3…
Unsloth AI retweeted
We did a new Unsloth Desktop release!
* Experimental auto compaction for any model
* LAN & Remote Access tabs
* Faster & smoother chatting
* Over 200 PRs merged!
For compaction, we use a mix of RAG + forced 1st turn RAG + tail - let us know how it goes!
github.com/unslothai/unsloth…
Unsloth AI retweeted
Huge thanks to Unsloth for the great work. This is wonderful news for the community! 🥳Qwen3.8-27B, smaller and sharper than ever. Let's try it! @UnslothAI
We’re releasing new Qwen3.8-27B GGUFs with 10% higher accuracy.
Unsloth Dynamic V3 outperforms others by >10% on Div-300, KLD & more benchmarks.
We also release 1-bit quants that retain 77% accuracy. Run on 8GB RAM.
Blog: unsloth.ai/docs/basics/dynam…
GGUF: huggingface.co/unsloth/Qwen3…
Unsloth AI retweeted
Local AI just got a major upgrade.
Fine-tune and run AI models locally with @UnslothAI Desktop, plus get up to 20% faster llama.cpp performance (@ggerganov) with our new optimizations.
Unsloth AI retweeted
Qwen3.8-27B + Unsloth is wow! Feels like having an Opus-level coding agent running fully locally it edits, sees, fixes, and more. Huge congratulations to Qwen for dropping a true Opus-class model that actually runs on local hardware, and to Unsloth for making it easy to play with!
@Alibaba_Qwen @QwenDevs @UnslothAI
Unsloth AI retweeted
27B on 17GB RAM. Are you ready to create something incredible? 😎
Thanks for highlighting it! @UnslothAI
Qwen3.8-27B can now be run locally! ✨
Run on 17GB RAM via Unsloth Dynamic GGUFs.
Qwen3.8-27B is by far the strongest model for its size. We also uploaded NVFP4 quants.
GGUF: huggingface.co/unsloth/Qwen3…
Guide: unsloth.ai/docs/models/qwen3…
Unsloth AI retweeted
You can now be a sloth because Unsloth is coming to your desktop! 🦥🦥
Introducing Unsloth Desktop 🦥
The first desktop app to run and train models locally.
• Open-source. Runs on Mac, Windows and Linux
• Supports MLX, diffusion image/video, audio, GGUF
• Connect Claude Code and Codex to local LLMs
• 50% more accurate, self-healing tool calls + sandboxed code exec
• Works for CPU + multiGPU setups - NVIDIA, AMD, Intel, Mac
• Train models 2× faster with 70% less VRAM
• Private web search, deep research, RAG, MCP and exports (NVFP4, GGUF)
• Use Unsloth’s OpenAI-compatible API and cloud models
• Securely deploy LLMs remotely and access anywhere
Unsloth Desktop is now available on unsloth.ai and GitHub.
GitHub: github.com/unslothai/unsloth
Blog and Guide: unsloth.ai/docs/desktop
Unsloth AI retweeted
Unsloth just turned local AI into a desktop app.
Run or fine-tune models on your own machine, then connect them to Claude Code or Codex through an OpenAI-compatible API.
Introducing Unsloth Desktop 🦥
The first desktop app to run and train models locally.
• Open-source. Runs on Mac, Windows and Linux
• Supports MLX, diffusion image/video, audio, GGUF
• Connect Claude Code and Codex to local LLMs
• 50% more accurate, self-healing tool calls + sandboxed code exec
• Works for CPU + multiGPU setups - NVIDIA, AMD, Intel, Mac
• Train models 2× faster with 70% less VRAM
• Private web search, deep research, RAG, MCP and exports (NVFP4, GGUF)
• Use Unsloth’s OpenAI-compatible API and cloud models
• Securely deploy LLMs remotely and access anywhere
Unsloth Desktop is now available on unsloth.ai and GitHub.
GitHub: github.com/unslothai/unsloth
Blog and Guide: unsloth.ai/docs/desktop
Unsloth AI retweeted
And it supports Linux, because they're just cool like that.
Introducing Unsloth Desktop 🦥
The first desktop app to run and train models locally.
• Open-source. Runs on Mac, Windows and Linux
• Supports MLX, diffusion image/video, audio, GGUF
• Connect Claude Code and Codex to local LLMs
• 50% more accurate, self-healing tool calls + sandboxed code exec
• Works for CPU + multiGPU setups - NVIDIA, AMD, Intel, Mac
• Train models 2× faster with 70% less VRAM
• Private web search, deep research, RAG, MCP and exports (NVFP4, GGUF)
• Use Unsloth’s OpenAI-compatible API and cloud models
• Securely deploy LLMs remotely and access anywhere
Unsloth Desktop is now available on unsloth.ai and GitHub.
GitHub: github.com/unslothai/unsloth
Blog and Guide: unsloth.ai/docs/desktop
Unsloth AI retweeted
Run and train AI models locally 🦥
Introducing Unsloth Desktop 🦥
The first desktop app to run and train models locally.
• Open-source. Runs on Mac, Windows and Linux
• Supports MLX, diffusion image/video, audio, GGUF
• Connect Claude Code and Codex to local LLMs
• 50% more accurate, self-healing tool calls + sandboxed code exec
• Works for CPU + multiGPU setups - NVIDIA, AMD, Intel, Mac
• Train models 2× faster with 70% less VRAM
• Private web search, deep research, RAG, MCP and exports (NVFP4, GGUF)
• Use Unsloth’s OpenAI-compatible API and cloud models
• Securely deploy LLMs remotely and access anywhere
Unsloth Desktop is now available on unsloth.ai and GitHub.
GitHub: github.com/unslothai/unsloth
Blog and Guide: unsloth.ai/docs/desktop