Manage GPU clusters for running LLMs https://xitter.cf/t.co/UUXf6jWiBY
Joined June 2024
- Tweets87
- Following78
- Followers150
- Likes46
🚀 DeepSeek-V4.1-Flash-1017 is now supported on GPUStack!
Get started with self-hosted DeepSeek on your own infrastructure, with full control over where your model runs. 🔒
👉 github.com/gpustack/gpustack
🚀 DeepSeek-V4.1-Flash-1017 is here!
Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API.
🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model.
1/4
Love the pace of updates! 🚀 Thanks for continuing to push open-source AI forward. Looking forward to deploying this latest release with GPUStack!
Bring your model mix and a task you’re figuring out how to route. Join the GPUStack Discord to compare setups and routing policies with the community.
👉discord.gg/VXYJzuaqwD
A quick translation and a payment-system design shouldn’t need the same model.
GPUStack × Jev routes tasks across self-hosted Qwen and DeepSeek through one endpoint.
Here’s the setup—and how we checked which model actually answered.
We’ve integrated Jev into GPUStack’s model routing, using it to guide selection between self-hosted Qwen and DeepSeek through one endpoint.
All 9 test inputs matched our configured policy. Putting together a walkthrough of the setup and validation.
Keeping trajectory branching and training costs under control—definitely a direction worth watching~
Introducing QwenGyre, bringing elastic RL to extreme-long-horizon (XLong) agent training. ✨
It launches with two core designs: 🥳
- Elastic Scheduler: reallocates GPUs between rollout & training as demand shifts—live executions never interrupted. Up to 1.85× speedup over Colocate, 1.78× over Async.
- Trajectory Processor: turns branching executions (compaction, sub-agents, retries) into prefix-shared trajectory trees with bounded, deduplicated training samples. Partial progress after timeout still gets scored.
Scaled to Qwen3.8-2.4T-A95B with ~700K tokens per rollout, QwenGyre lifts NL2RepoBench from 52.5% to 58.5% in 48 steps, cutting end-to-end training time by ~44% vs Async.
🔗 Read the paper:
arxiv.org/abs/2609.33848
This is the kind of hardcore optimization we love to see. 🔥
GPUStack is ready to orchestrate those SGLang workloads at scale. Respect! 🙌
740M params + multimodal = 🚀 perfect for on-device AI. And it's Apache 2.0 open-source! We’re already gearing up to support EmbeddingGemma 2 on GPUStack. Can't wait to get it running on edge clusters!
The best model keeps changing. Switching without rebuilding workflows creates real value for users. We care about that at GPUStack too.
From generation to editing: pass the image back with “Replace the cup of coffee with a stack of books.”
☕➡️📚
Explore GPUStack and try it yourself:
gpustack.ai
github.com/gpustack/gpustack
With the model running, let’s generate an image.
Send your prompt and generation settings to /v1/images/generations.
👇The example request and output are below.Use your service URL, API key, and deployed model name.
One A800 80GB, from deployment to image generation and editing.
🍀Here’s our Qwen-Image-2.1 walkthrough with GPUStack and vLLM-Omni, starting with backend setup and model deployment.
Preview: replacing coffee with books
Building a self-hosted AI inference stack?
GPUStack unifies model deployment, GPU management, and API routing across engines like vLLM and SGLang—with monitoring and governance for day-to-day operations.
Explore the architecture below. 👇
🤖 Made with AI
