MLX + TurboQuant + Apple's M5 Max = 🔥 Some more details for you on the @Prince_Canuma MLX port of TurboQuant running on Apple's M5 Max. Rather than convert a PDF to text and then process it, I loaded the PDF (arxiv.org/pdf/2603.06728) as 150 dpi images. 20 pages at bf16 uses 1.33 GB; however, using TurboQuant at 2.5-bit, it's only 229 MB. The document in this form is 42k tokens. Building the cache (normally the pre-load) takes 18.2 seconds, this is a one-off if we're processing the same document. However, we can save the cache and re-load it next time. For 230 MB, it takes 23 ms to load the entire quantised cache of the entire document. Working with a quantised cache is slower than "raw" (bf16). TTFT with the 160 MB TurboQuantised cache is 250 ms against around 130 ms for the 1.3 GB bf16 cache. Similarly, and more significantly, the generation speed is down to 29 tok/sec from 80 tok/sec with an un-quantised cache (using an 8-bit quantised model). What's amazing about processing the document visually is that you can query the images AND the text. You can ask questions about images and graphs and get a response in a few seconds across the entire document. You can pre-process your documents and save them as KV-caches and load them in a fraction of a second. Pick the document and query it. Load from disk, and first token in less than half a second. The video represents queries on the PDF above, on the left, demo query UI on the right. When mlx-vlm is updated for the TurboQuant I will push the code for the demo. Remember, this is the worst this is ever going to be.