Scality

Scality has launched a software stack for running open-weight AI models on servers you own. It went on sale on October 8. The main idea is where the KV cache lives. That cache is the model's working memory for a conversation, and it usually sits in GPU memory. Scality moves it onto its own object storage. One pool of GPUs reads the prompt and writes the cache to storage. A second pool reads it back and writes the answer, so any free GPU can pick up any old session. Scality's own tests say a 14K-token context restored from storage reached the first token in 166 ms, 83 ms behind serving it from GPU memory. Reading the cache back was 14 times faster than working it out again, and up to 72 times faster on a 439K-token context. The cache held more than 80 times the memory of one GPU, which Scality says keeps up to 1,000 sessions ready to resume. These are Scality's lab numbers. The stack runs open models such as Mistral, Gemma, Qwen, Kimi, GLM and DeepSeek on standard Dell, HPE, Lenovo and Supermicro servers. You can buy it as a license or as a managed service. My read: the payoff grows with the length of the session. The context went from 14K to 439K tokens, about 31 times longer (439 / 14 = 31.4). The speedup went from 14x to 72x, about 5 times bigger (72 / 14 = 5.1). Long agent sessions that keep coming back to a huge history are where this pays most. globenewswire.com/news-relea…