oMLX 0.7.0rc1 is out! This release brings faster Qwen prefill & generation, MiMo V2.6, Ternary Bonsai 2, and partial block caching. DFlash now handles concurrent requests together, and Lightning MTP gets faster batch decoding! github.com/jundot/omlx/relea… Performance on M5 Max, 128 GB (Prefill, oQ4e quant) - Qwen3.8-Flash-Next: 1,522 -> 2,007 tok/s (+32%) at 16K context. (Decode, batch=4, oQ4e quant) - Qwen3.8-27B with DFlash2: 56.9 -> 131.5 tok/s (+131%) - Qwen3.8-27B with Lightning MTP: 88.9 -> 136.9 tok/s (+54%) Full benchmark details are in the release notes. New models and features - MCDMA RDMA support for Mac + CUDA deployments, contributed by @ashxhart. - Partial block caching. No more reprocessing thousands of tokens just because they didn't fill a complete cache block. In one test, next-turn prefill dropped from 1,174 tokens to 37. - Ternary Bonsai 2 text and vision support. - MiMo V2.6 image, video, and audio understanding, plus Lightning MTP and DFlash for compatible checkpoints. - Broader MoE expert offload, including Lightning MTP alongside expert offload for DeepSeek V4.1 and GLM-5.3-Flash. This RC also includes the improvements from the dev releases, including Cluster v2, one-click model settings from community benchmarks, and a customizable dashboard. The GDN prefill kernels are adapted from @ddalcu's excellent mlx-serve! After a short round of testing, I'll publish the stable release and keep moving forward!

I'm happy to announce that I've joined Hugging Face. What started as a personal project back in February is now something I get to work on full time. Local AI has grown explosively this year. I've said this since the early oMLX releases: I want my friend who bought a MacBook yesterday to be able to run AI on it today. I believe MLX has that potential. Apple Silicon is the easiest entry point for a regular person to get started with AI, with no complicated hardware to assemble. And the open source community, including Hugging Face, has been growing that potential. That support has already made a huge difference. Hugging Face is the best place for me to support oMLX and the MLX community with everything I have. For anyone getting into local AI, the first step usually starts at Hugging Face. Mine did too. I'm proud that I get to work at that entry point, where new ideas can be tried out. oMLX stays exactly where it is, under the same Apache 2.0 license in the same repository, and I'll keep leading the project, same as before. What changes is that I can now spend far more time on it, move faster, and build something sustainable for the long term together with all the contributors who have put so much into it. I also want to do more for MLX as a whole. oMLX is built on top of transformers, mlx-lm, mlx-vlm and the rest of the ecosystem, and I'm grateful to the people behind them. Rather than keeping everything inside oMLX, over time I want to push work upstream where it makes sense. And where the community needs something that doesn't exist yet, oMLX is a good place to try it first. Thank you to the 264 contributors who have built oMLX with me, and to everyone who filed issues with detailed logs and reproductions so we could fix things. oMLX would not be what it is without you. oMLX continues in the same place, in the same way. Just much faster. HF's announcement: huggingface.co/blog/omlx