The most secure AI inference on earth. Powering your next AI App.
Bittensor
Joined November 2024
- Tweets797
- Following11
- Followers11.8K
- Likes519
14 open models, one endpoint, every one of them running inside a hardware-attested TEE.
That's the Chutes catalog today: Kimi K3 and GLM-5.2 with 1M context, DeepSeek V4 Flash, Qwen3.8-27B, Gemma 4 31B, and Nemotron-3-Nano from $0.0245 per million input tokens. The list changes often.
Any OpenAI-compatible client works. Set the base URL to llm.chutes.ai/v1 and keep the rest of your code.
Pretraining a model used to mean a datacenter and a nine-figure budget. Jon did it on rented RTX 5090s over the open internet for $11.27 per billion tokens.
The whole 8B run came to about $6,500. Retail hourly GPU prices, 574B tokens, 52 hours. The syncers were four CPU VMs with no GPU in them, and there was no object store anywhere in the system.
If a model costs about what a used car costs, a lot more people get to own one. That's the point of Parallax.
Jon's comparison from the same slide, his estimate from a published report: Llama 3.2 1B at about 370,000 H100 hours and retail H100 rental of $2 to $4 per GPU-hour works out to $92 to $184 per billion tokens.
The live training register: parallax.chutes.ai
Week 22. The short version:
→ Jon published Epigenesis, a way to teach a Parallax model new facts without changing a single base weight. New knowledge gets written into the zero slots the compressed model already has. Zero the slots and the original model is back, bit for bit. On the hardest test, a 3.3 MB patch beat an 88 MB LoRA adapter on recall. Early results, single run, not peer-reviewed, nothing serves it yet
→ Lambda is still training. Seven hosts have failed so far and the run absorbed every one of them without a restart. Boring results like that are the whole point
→ Confidential VM 1.5.0 is out to miners. Every file that goes into building it is now pinned by hash, so anyone checking our published measurements rebuilds exactly what we measured
→ Hot-swap upgrades for private chutes shipped. Maintenance on our side should stop taking your model offline
→ AMD hardware still waits. Nothing changed in the bulletin this week
Each one below, with what it means if you are not an engineer 👇
Two weeks, one issue. The short version:
→ A model our team built ran on an ordinary phone, no cloud involved. Jon showed it at Exploit: Parallax on a Snapdragon 8 Elite, CPU only, 59.6 tokens a second, and nothing left the device. A datacenter has an off switch. A billion personal devices don't
→ A model we trained on scattered consumer graphics cards matched a big lab's model on several tests, for a fraction of the cost. Kappa passed Llama 3.2 1B at around 90 percent lower cost per token. Early results from one run
→ SOMA (SN114) is live on Chutes inference
→ AMD disclosed a security flaw in its chips, so our engineers chose to wait rather than switch them on. A chip that says it is patched is not the same as a chip that can prove it
Full details in the article below 👇
Chutes retweeted
There's another magic trick (aka cool math) I believe this parallax enforced sparsity enables by the way...
arxiv.org/abs/1711.05769 <- prune weights, adapt the pruned weights, sharply cuts catastrophic forgetting risk.
If you have a model with the enforced sparsity, 50% of the weights are locked in to particular patterns. What if you mask the other expert weights leaving only those sparse contiguous weights to learn new data?
That's the intuition, perhaps it works, perhaps not. But... if it does, it would make the base model significantly more malleable for new domains/tasks/whatever. SFT/continued pretraining with less risk of catastrophic forgetting and without lots of magic tricks and RL and re-sampling in your own words etc.
Chutes retweeted
@ExploitSummit was an amazing event! So many brilliant, motivated people.
If you want to hear about the model training stuff we've been working on, and why we're working on it in the first place, here's the talk.
youtube.com/watch?v=4FOxXT66…
Chutes retweeted
If you're concerned about the consolidation of control of AI by a few powerful companies/governments the only alternative is models so small anyone can run one on their laptop, phone or consumer hardware (e.g. like a DGX Spark). This decentralized vision is the future @jon_durbin has set @chutes_ai to building.
Most AI runs in a datacenter, and someone else holds the off switch.
At Exploit Summit (@ExploitSummit ), Jon Durbin showed Parallax running on a phone. CPU only, no GPU. The trained 8B model decoded at 59.6 tokens per second. A 40.75B model ran at 26.9 tokens per second with 12 GB peak process memory.
Where that leads:
• You could use AI in a country that blocks it.
• A doctor could run a model on a tablet in the exam room, and no patient data leaves the building.
• A law firm could keep one on a laptop in a locked office and never worry about prompts leaking to a vendor.
• You could use it on a plane or through an outage.
• No monthly bill and no rate limits. No company deciding tomorrow that your use case violates its policy.
A datacenter has an off switch. A billion personal devices don't.
Most AI runs in a datacenter, and someone else holds the off switch.
At Exploit Summit (@ExploitSummit ), Jon Durbin showed Parallax running on a phone. CPU only, no GPU. The trained 8B model decoded at 59.6 tokens per second. A 40.75B model ran at 26.9 tokens per second with 12 GB peak process memory.
Where that leads:
• You could use AI in a country that blocks it.
• A doctor could run a model on a tablet in the exam room, and no patient data leaves the building.
• A law firm could keep one on a laptop in a locked office and never worry about prompts leaking to a vendor.
• You could use it on a plane or through an outage.
• No monthly bill and no rate limits. No company deciding tomorrow that your use case violates its policy.
A datacenter has an off switch. A billion personal devices don't.
He showed serving numbers too: 17,935 output tokens per second on one RTX 5090 at 256 concurrent requests.
It's a benchmark preview, running on a fork of vLLM that is not optimized yet. Expect that number to move.
More model runs and scores are coming. We're just at the starting point.
Chutes is building the Linux of AI. That's a future where AI access is decentralized, and models could run on consumer hardware sitting in a field, on solar and a battery.
No gatekeepers between you and intelligence.
Lots more to come. Stay tuned!
Watch the talk yourself (Jon is the final hour of the stream): xitter.cf/i/broadcasts/1nxnRBrrZ…
See the run live: parallax.chutes.ai
Chutes retweeted
"Doing sixty tokens per second on a phone."
At Exploit, @jon_durbin reported an 8B model running on a phone's CPU, without a GPU or NPU. The test used phones rented through Qualcomm Device Cloud, before any optimization work.
Jon Durbin takes the Exploit (@ExploitSummit) stage in Montreal today at 5:15 pm ET.
The talk is called "DeAI is dead. Long live DeAI."
We built Parallax to train large AI models with no data center, on ordinary GPUs scattered across the open internet.
It works. A 20B model trained over the open internet on 4 single H100s landed within 0.3% of the datacenter baseline at matched tokens.
That was the paper. Today Jon shows what we have built since.
If you are at New City Gas, get to the OTF stage.
Agenda: exploitsummit.com/agenda
Not in Montreal? Highlights from the talk go up here tomorrow.
