We have added Cluster inference support to Qvac Fabric. You can cluster together multiple machines and split the inference across them. Supporting both tensor and layer parallelism. Here 2x DGX Spark running Deepseek-v4-flash-0731 over RDMA.
github.com/tetherto/qvac-fab…
Latest @qvac Fabric work added CoreML support to make Parakeet insanely fast. You can transcribe 25 mins audio in nearly 16 seconds on an Iphone 16.
check github.com/tetherto/qvac-fab…
QVAC Fabric now runs Parakeet's encoder through Core ML on Apple silicon, which moves it from the GPU to the Neural Engine.
On a MacBook Air M2, Parakeet TDT 0.6B v3 went from 78x real time on the Metal GPU to 164x through Core ML.
1
136
Did you check latest inference optimizations on @qvac for speech models ?
47
Deepseek v4 flash is now available on Fabric. We added Vulkan and Metal support, so you can run it on any gpu. Performance improvements come from optimized fused kernels. Go and try the new @deepseek_ai released version '0731' on github.com/tetherto/qvac-fab…
DeepSeek V4 Flash on Strix Halo just got a major Vulkan boost on Fabric.
Fusing different ops improved prefill way more than vanilla llama.cpp, closing the gap with @antirez ds4 ROCm implementation.
Coming soon to github.com/tetherto/qvac-fab….
