Hyoung Kyou Jeon

oMLX 0.7.4.dev1 is out! github.com/jundot/omlx/relea… With the new Prefill Accelerator, connect an M5 Max to your big Mac and prefill up to 2.9x faster! Prefill Accelerator (64K prompt, M3 Ultra 512GB + M5 Max 128GB) - DeepSeek V4.1 Flash: 1,060 -> 3,123 tok/s (+195%) - Qwen3.8 Flash-Next: 1,309 -> 3,666 tok/s (+180%) - GLM-5.3 Flash: 540 -> 1,388 tok/s (+157%) - MiniMax M3.1: 297 -> 569 tok/s (+92%) - An M5 Max 64GB still adds 30-118%, and decode speed stays the same. Faster DeepSeek V4.1 with A8 and buffer pool improvements - Prefill on M5 Ultra is 43-48% faster, 32K goes 2,631 -> 3,779 tok/s (+44%). - oQ4e builds now run INT8 Activation Prefill on M5. Faster Qwen3.8 Flash-Next - FP16 builds get the fused kernels. Decode on M3 Ultra goes 38.9 -> 63.0 tok/s (+62%), and +23% on M2 Max. - Decode after long prompts is 3-5% faster on M5 Max. Better for agents - Parallel tool results from Claude Code reach the model in call order, so Qwen no longer pairs a result with the wrong call. - Agent sessions on a cluster reuse the prompt cache across tool turns, 743 -> 9,266 cached tokens by the fifth turn. - Switch oMLX models inside Codex and Copilot with /model. Also in 0.7.4.dev1 - Ming-3.0 TTS with a 2x faster decode loop. - Ask AI can use an oMLX server on another Mac. - Check for model updates in the macOS app. - German across the web dashboard and the macOS app. - Batched decode with 6 requests on 8-bit models is 16% faster. A big thank you to everyone who sent PRs, reported bugs and tested builds for this release, and a warm welcome to the five new contributors!

All my oQe models have been updated with Improved oQe Quantization, available from oMLX 0.7.1. Compared with the previous oQe builds, KL divergence is 11.6–18.1% lower on code and 10.2–13.8% lower on prose, so please give them a try. The full model list is below. huggingface.co/Jundot KL divergence from the original BF16 model (mean, lower is better) - Qwen3.8-27B oQ4e: code 0.0143 → 0.0117 (-18.1%), prose 0.0696 → 0.0600 (-13.8%) - Qwen3.8-Flash-Next oQ4e: code 0.0877 → 0.0776 (-11.6%), prose 0.1909 → 0.1713 (-10.2%) What's new - Below oQ4, the Engram and N-gram tables stay at 4 bits and are left out of the bit budget. oMLX can serve these tables from SSD instead of memory, so the full bit budget goes to the rest of the model. - Model cards now show the bit distribution and storage breakdown. If you downloaded an earlier oQe build, please re-download it. Improved oQe models (69) - Qwen3.8-Flash-Next: 2.5 · 2.7 · 3 · 3.5 · 4 · 6 · 8 bits - Qwen3.8-Flash-Next FP16: 2.5 · 3 · 3.5 · 4 bits - Qwen3.8-27B: 2.5 · 3 · 3.5 · 4 · 6 · 8 bits - Qwen3.6-35B-A3B: 2.5 · 3 · 3.5 · 4 · 6 · 8 bits - GLM-5.3-Flash: 2 · 2.5 · 3 · 3.5 · 4 · 5 · 6 bits - DeepSeek-V4.1-Flash: 2 · 2.5 · 3 · 4 bits - DeepSeek-V4-Flash-0731: 2.5 · 2.7 · 3 · 3.5 · 4 bits - gemma-4-31B-it: 2.5 · 3 · 3.5 · 4 · 6 · 8 bits - gemma-4-26B-A4B-it: 2.5 · 3 · 3.5 · 4 · 6 · 8 bits - gemma-4-12B-it: 2.5 · 3 · 3.5 · 4 · 6 · 8 bits - clef: 2.5 · 3 · 3.5 · 4 · 6 · 8 bits - clef-flash: 2.5 · 3 · 3.5 · 4 · 6 · 8 bits