Control plane for agents & engineers to provision compute and run training & inference across NVIDIA, AMD, and other chips — on clouds, Kubernetes, and on-prem.
github.com/dstackai/dstack
Joined January 2020
- Tweets1.3K
- Following55
- Followers1.6K
- Likes1.1K
Here's one example of using the presets toolkit to optimize Qwen3.8-27B on a single @AMD MI300X.
Through compound learning across linked sessions and source-level patches, performance improved from 311 to 495 tok/s: +59%.
The resulting preset can be deployed on any AMD cloud, Kubernetes cluster, or bare-metal fleet.
Inference serving is increasingly open source. But inference optimization still happens inside each provider, and the optimized deployment remains tied to its proprietary stack. This has to change.
Introducing Presets: an open-source toolkit for agent-based inference optimization and a portable format for the result.
Deploying optimized Kimi K3 to any cloud, Kubernetes cluster, or bare-metal fleet, whether on NVIDIA, AMD, or other silicon, should be as simple as deploying a Docker image.
dstack.ai/blog/presets/
dstack 0.21 is out 🎉
A major release: the codebase has migrated to @pydantic v2. Improved CLI and server performance, plus Python 3.14 support. This clears the way to accelerate the project much further.
Also in 0.21:
* A new version of Presets, our experimental inference optimization toolkit
* Replicated gateways for high availability for inference deployments
* A ton of other improvements across the CLI, offers, and backends
Full release notes 👇
github.com/dstackai/dstack/r…
AI infra builders in SF, we are meeting on July 23.
@CrusoeAI, @lmsysorg SGLang, and dstack are hosting a meetup on GPUs, training, inference, and open source AI systems.
12 short technical talks with speakers from @sgl_project, @ByteDanceOSS, @NVIDIA, @radixark, @CrusoeDev, and more to be announced soon.
RSVP: luma.com/rxsn0u0h
Crusoe has been doing great work making GPU infrastructure more accessible for AI teams.
Now @CrusoeAI has a new guide on using dstack to manage clusters, training, and inference on Crusoe Cloud: docs.crusoecloud.com/third-p…
dstack 0.20.26 is out 🚀
This release makes run submission noticeably faster. Starting a run on an idle instance now takes ~1–2s instead of ~7s.
The server reuses pooled SSH connections to instances instead of reopening one per operation.
Full release notes 👇
github.com/dstackai/dstack/r…
dstack is now in the official @jarvislabsai docs 🙌
docs.jarvislabs.ai/dstack/
Orchestration is becoming the default way GPUs get used. Thank you, @jarvislabsai for the integration!
If you're looking for on-demand GPUs, they offer H100, L4, H200, RTX PRO 6000.
Native @NVIDIA Dynamo support is now in dstack.
Deploy high-throughput inference with PD disaggregation, without custom orchestration glue.
Works with SGLang, vLLM, and TensorRT-LLM, across GPU clouds, Kubernetes, and on-prem fleets.
Another step toward simpler AI-native orchestration for production inference:
dstack.ai/blog/nvidia-dynamo…
The agentic engineering wave is coming to AI infra, but Kubernetes is too messy to keep up, and is a real bottleneck.
The founder of dstack, @andrey_cheptsov, sat down with @readsail at @nvidia GTC to talk about GPU orchestration.
Big thanks to @makora_ai and @SemiAnalysis_ for organizing the interview!