@Chutes_AI | Shaping secure, decentralized private serverless compute
Joined November 2015
- Tweets25
- Following42
- Followers231
- Likes6
Confidential VM 1.5.0 is out on @chutes_ai. The VM now supports AMD-SNP and has improvements to reproducible builds. But AMD hosts stay off the network until AMD-SB-3030 has fixes we can verify through attestation.
Every input to the VM build is now pinned by hash, so if you rebuild it to check our published measurements, you pull exactly the bytes we measured.
The upgrade mechanism now supports hot-swaps for private chutes. When a host is upgraded, your private chute keeps serving for up to 3 hours while a replacement comes up elsewhere. If no replacement is running by then, the upgrade can still go ahead.
1.5.0 supports the latest Bittensor hotkey format in our ansible playbooks. Host tooling now checks that devices are fully released before launching a VM, so a reboot during a long memory reclaim can't leave a GPU host unresponsive.
Kyle Widmann retweeted
Kappa run is now stopped, not too bad overall! We passed llama 3.2 1b on ARC-C, OBQA, TQA, etc., nearly matched on ARC-E, SciQ, etc. Considering that was 9T tokens and we hit those numbers on the 576B checkpoint, AND we did so around 90% cheaper per token, I'd call it a solid preliminary win!
Issues with the run:
1. In one of the Gated DeltaNet-2 heads, the per-channel decay gate underflowed, causing state wipe, and ourput RMSNorm amplified near-zero outputs ~1000x into fleet wide grad spikes.
2. Very late/slow nodes were so far behind their evidence was discarded, not a huge effect but we should have had better tolerance for stale data and partial updates.
3. We packed samples but didn't do masking/etc. which is sub-optimal for pretraining (not a huge diff but worth fixing anyways).
4. The first and last layers being GDN2 may be suboptimal also, vs. SWA, no clear evidence but might be worth switching up the order a bit for those two.
We have a fix for this GDN2 issue, but while that was still running I also tested EDA (arxiv.org/abs/2606.26560v1) and it actually addresses some of the root cause directly (and seems to have better loss than GDN2 anyways) so we'll probably switch to EDA once all tests come back clean and pass the kappa's instability regime (and waiting until we have highly optimized tilelang kernels for EDA to keep MFU/throughput high).
Onwards and upwards 🚀
Weekly dev update for @chutes_ai 🚀
Continuing to improve reliability, support, and our confidential-compute stack, plus a look at what's coming.
TEE: Added the initial portion of some phased tooling so new/unrecognized GPU hardware can be onboarded quickly while staying hardware-attested, bringing more topologies online with less friction for miners.
Builds: our new remote build pipeline still improving. Most builds are smooth, we've added monitoring for the edge cases, and we're now tracking build + deploy times so we can work on making deployments faster.
Platform: better monitoring helped us find and ship database performance improvements for streaming and miner endpoints, plus full chute-lifecycle tracking so support can pinpoint issues fast.
Product: the model catalog now has curated docs guides for every first-party model, easier than ever to pick a model and start building.
What's next:
- Security enhancements that let us rotate credentials across the whole fleet on the fly, no VM rebuilds needed.
- And we're exploring on-demand TEE VM rentals (GPU or CPU). Would love to hear if that's something you'd use.
#Bittensor #ConfidentialComputing
Weekly Dev update for @chutes_ai :
Update confidential-compute support, clearer billing visibility, and a more reliable, observable GPU fleet
#Bittensor #ConfidentialComputing
Product Updates:
You can now see your real spend limits (monthly, hourly, and pay-as-you-go) right in the usage dashboard, plus smoother billing and a faster, more accessible mobile experience.
Kyle Widmann retweeted
Just for fun I kicked off a run of a 176b parameter model on ~140 steps to prove feasibility - 4 separate nodes across the internet using the "Parallax" method, works like a charm.
Still need a more concrete plan on dataset curation, phases, context elongation, etc. etc. before a full run is ready of this scale, but at least we know 176b should be no problem at all.
You can force an architecture into decentralized training, or you can craft an architecture specifically for decentralized training (and ultimately insanely performant inference). Stop trying to fight gravity.
"Parallax", so far, is working exceptionally well.
Surrogates, expert offload, stratified decoupled diloco, etc. All pretty great to make training work at scale across the internet. But... The biggest benefits are actually AFTER training, via specific model architecture choices that drastically reduce VRAM and increase throughput.
These giant MoEs are wasting so much VRAM and compute for all of the many routed experts, it's insane actually. So much research already showing that MoEs can be massively compressed/pruned/ternary-distilled/etc., not to mention they are generally comparable to around 2.5-3x dense model equivalents of the ACTIVE parameter count. KV cache is a huge bottleneck, and these huge routed experts are often "dead weight" that consume it needlessly.
We need to do better, so we will.
The Chutes.AI stack runs GPU attestation, TEE-secured VMs, and live inference traffic. Two backend engineers maintain it, on a team of ~10. AI tooling is part of what makes that ratio possible. But that leverage comes with a risk I don't think is obvious yet.
When two people own a live production system, there's no room for that gap. Failures are concrete and they show up fast. That constraint forces a different relationship with the tools.
Our TEE implementation was the foundation for secure compute at Chutes.AI, but we’re not stopping there.
We’re actively adding new capabilities to push closer to true trustless compute on the Bittensor network.
Here are two major improvements we’re working on right now 👇 #Bittensor #ConfidentialComputing
2. Workload Verification (Beyond infrastructure attestation)
We already enforce cosign signatures. Now we’re exploring ways to bind the actual running workload into the TDX attestation itself.
A trusted component inside the TEE would compute a deterministic workload identity (based on image digests, config, command/args, etc.) and bind it into the quote.
Users could then verify the attestation and check that the running workload matches what they expect.
These changes help make secure, decentralized serverless GPU compute more open, verifiable, and accessible across the Bittensor network.
Live TEE models: chutes.ai
Repo: github.com/chutesai/sek8s
#Bittensor #TrustedExecutionEnvironments #DecentralizedAI #OpenSource #AIInfrastructure
Do you actually know if your AI inference provider is secure — or are you just trusting their marketing?
Many platforms now say “we have TEEs.” But a basic TDX VM can still have SSH access, unprotected GPUs, and you still have to blindly trust the provider.
In decentralized networks the bar is much higher. Thread 👇 #ConfidentialComputing #DecentralizedAI
We combine Intel TDX + NVIDIA GPU attestation (nvtrust):
Guest disk is LUKS-encrypted → only decrypts after TDX boot attestation (RTMRs). Failed = VM never starts.
GPUs use PCIe passthrough with hardware isolation. Verified via NVIDIA Remote Attestation SDK to prevent host memory dumps.
Everything is open source so you can audit: no SSH, strict OPA policies, only signed workloads.
For client verification: you send a nonce → both TDX quote + GPU attestation bind to hash(nonce || e2e_pubkey).
This gives true end-to-end encryption where you control the keys. Even Chutes.AI or the miner can’t decrypt your traffic.
Verifiable privacy instead of “trust us.”
What would make you trust a decentralized inference provider?
chutes.ai
E2EE details: chutes.ai/news/end-to-end-en…
Repo: github.com/chutesai/sek8s