CJ 🇺🇸 retweeted
Florian's building super cool stuff, go check it out!
Introducing DecisionModels @decisionmodels_
I've spent the last four weeks benchmarking decision models (I run JevBench).
Today I'm launching the place to actually use the best ones:
decisionmodels.io
➡️Top open models
➡️Up to 4x faster on our own REAX engine
➡️From $0.019 per 1M input tokens.
Expect more reveals soon, and read 👇
CJ 🇺🇸 retweeted
github.com/chutesai/epigenes…
Still experimenting, but seems to check out. Smaller, faster, better? Models that grow with you (or your repo, or proprietary company docs, etc.)
Short term exact recall is the context window.
Context window feeds training continuously, binary optimizer.
This becomes stored knowledge and higher order perceptual understanding over time, not just RAG.
Sparse MSA global attention already selects most important tokens to attend to in longer context, so you can reinforce those parts more.
🤖 Made with AI
There's another magic trick (aka cool math) I believe this parallax enforced sparsity enables by the way...
arxiv.org/abs/1711.05769 <- prune weights, adapt the pruned weights, sharply cuts catastrophic forgetting risk.
If you have a model with the enforced sparsity, 50% of the weights are locked in to particular patterns. What if you mask the other expert weights leaving only those sparse contiguous weights to learn new data?
That's the intuition, perhaps it works, perhaps not. But... if it does, it would make the base model significantly more malleable for new domains/tasks/whatever. SFT/continued pretraining with less risk of catastrophic forgetting and without lots of magic tricks and RL and re-sampling in your own words etc.
CJ 🇺🇸 retweeted
Kappa run is now stopped, not too bad overall! We passed llama 3.2 1b on ARC-C, OBQA, TQA, etc., nearly matched on ARC-E, SciQ, etc. Considering that was 9T tokens and we hit those numbers on the 576B checkpoint, AND we did so around 90% cheaper per token, I'd call it a solid preliminary win!
Issues with the run:
1. In one of the Gated DeltaNet-2 heads, the per-channel decay gate underflowed, causing state wipe, and ourput RMSNorm amplified near-zero outputs ~1000x into fleet wide grad spikes.
2. Very late/slow nodes were so far behind their evidence was discarded, not a huge effect but we should have had better tolerance for stale data and partial updates.
3. We packed samples but didn't do masking/etc. which is sub-optimal for pretraining (not a huge diff but worth fixing anyways).
4. The first and last layers being GDN2 may be suboptimal also, vs. SWA, no clear evidence but might be worth switching up the order a bit for those two.
We have a fix for this GDN2 issue, but while that was still running I also tested EDA (arxiv.org/abs/2606.26560v1) and it actually addresses some of the root cause directly (and seems to have better loss than GDN2 anyways) so we'll probably switch to EDA once all tests come back clean and pass the kappa's instability regime (and waiting until we have highly optimized tilelang kernels for EDA to keep MFU/throughput high).
Onwards and upwards 🚀
CJ 🇺🇸 retweeted
@ExploitSummit was an amazing event! So many brilliant, motivated people.
If you want to hear about the model training stuff we've been working on, and why we're working on it in the first place, here's the talk.
youtube.com/watch?v=4FOxXT66…
If you're concerned about the consolidation of control of AI by a few powerful companies/governments the only alternative is models so small anyone can run one on their laptop, phone or consumer hardware (e.g. like a DGX Spark). This decentralized vision is the future @jon_durbin has set @chutes_ai to building.
Most AI runs in a datacenter, and someone else holds the off switch.
At Exploit Summit (@ExploitSummit ), Jon Durbin showed Parallax running on a phone. CPU only, no GPU. The trained 8B model decoded at 59.6 tokens per second. A 40.75B model ran at 26.9 tokens per second with 12 GB peak process memory.
Where that leads:
• You could use AI in a country that blocks it.
• A doctor could run a model on a tablet in the exam room, and no patient data leaves the building.
• A law firm could keep one on a laptop in a locked office and never worry about prompts leaking to a vendor.
• You could use it on a plane or through an outage.
• No monthly bill and no rate limits. No company deciding tomorrow that your use case violates its policy.
A datacenter has an off switch. A billion personal devices don't.
CJ 🇺🇸 retweeted
"Doing sixty tokens per second on a phone."
At Exploit, @jon_durbin reported an 8B model running on a phone's CPU, without a GPU or NPU. The test used phones rented through Qualcomm Device Cloud, before any optimization work.
CJ 🇺🇸 retweeted
i had no idea what jon durbin’s parallax was or why it mattered until listening to him talk at @ExploitSummit earlier today.
this one talk has been the highlight of the summit thus far.
reconstructing the training pipeline to max joules allowing inference of 50+tps locally from a cell phone is bonkers. puts tokenmaxxing and trainmaxxing to another level.
it’s worth the 1h listen and im looking for excuses to incorporate parallax training to build specialized models to advance the frontier of cyber in an open way
Dashboard is a quick work in progress, but for visibility into a run here ya go!
parallax.chutes.ai/
This is around $11/b tokens, insane actually. MFU also insane. The whole thing, pretty legendary, and inference...
Using a few nodes from @lium_io also!
CJ 🇺🇸 retweeted
Training and inference will run on the edge, by miners, myopically seeking energy wells like they do with Bitcoin mining.
@jon_durbin presenting the future and how humanity gets there.
CJ 🇺🇸 retweeted
We are excited to be working with the SOMA team on SN 114!
SOMA × CHUTES is live!
Connect your agents to SOMA and access @chutes_ai inference with lower costs.
We compress your agent context before sending it to Chutes for inference, so you pay to process fewer tokens with each request.
Compress. Pay less.
CJ 🇺🇸 retweeted
Very good chance that decision models unlock the next breakout AI consumer app, something that's been notably missing since Character AI.
AI will power some new games that were never feasible before!
CJ 🇺🇸 retweeted
User interfaces are not going away.
Yes, humans will use computers alongside agents. When working with agents, we will need the ability to audit what agents do in a human-readable way.
Yes, some interfaces (ex: dashboards, in-chat UI for ephemeral tasks, etc) will be generated. Most important interfaces will be intentionally designed. We will want both deterministic and non-deterministic interfaces in different situations. We're thinking a lot about the best ways to help designers shape non-deterministic systems at Figma.
Yes, new input modalities like voice, VR/AR and BCI will change the way we use computers and introduce lots of new interaction paradigms. This is all very exciting :)
Overall, the surface area of design will continue to grow much broader. Design is the new code and the key differentiator for software moving forward. Expect more and more energy to go into design / craft in the years ahead... it will be how you win or lose.
Eric is right about plenty of things, but the market will prove him wrong on this one...
"User interfaces are largely going to go away.
Because the agents speak English, or other languages. You can talk to them. You can say what you want. The UI can be generated.
So I can say, generate me a set of buttons that allows me to solve this problem, and it's generated for you."
- Eric Schmidt is the former CEO of Google.
Today’s websites are designed for humans, with carefully crafted UIs.
But we are probably moving towards UI-less web.
automated bot traffic has already surpassed human activity on the web. with AI agents, this number will explode.
In near future its very plausible that 90% or more of web traffic will be non-human.
Imagine a web with no websites. All your favorite apps are now just databases, queried for information by agents that zip around on your behest.
In a world primarily composed of bots, what happens to websites? Do agents learn to navigate them like a human? Do they go away entirely? 🤔
---
Video from 'Peter H. Diamandis' YT channel
CJ 🇺🇸 retweeted
A Harvard research team and @chutes_ai just released a public dataset covering one year of real-world LLM inference on Chutes: 6.12B requests across 9,174 models.
Technical usage data from a Bittensor subnet is now open to the wider AI research community.
Announcing one year of LLM inference metadata traces, with 6.12 billion requests.
We hope this dataset can support research on real-world LLM serving workload understanding, system design and infrastructure optimization. Explore the dataset and learn more: data.agentic-system.org
Driven by our great graduate student William Nixon and
in collab with @jon_durbin @airesearch12 @chutes_ai
CJ 🇺🇸 retweeted
Glad to see that Ethereum L1 will have a new strong prediction market contender that is dedicated to decentralization, and being ethical and not corposlop, and to actually trying to do interesting and meaningful things with this class of economic primitive.
firefly.social/post/x/210212…
Leaning into a world that sells agents. But when will a @Shopify store be able to leverage an in-house model trained on how to win on the platform - that's what I want to see.
If all you're doing is essentially plugging into someone else's model and offering that as a service you are an Accenture masquerading as an AI company.
every major company is arriving at roughly the same offering:
persistent memory, email/calendar/messages, browser + computer use, background tasks, proactive notifications, voice, app/tool execution, ambient context, & some notion of a personal agent sitting above everything.
remarkable levels of convergence with very little differentiation whatsoever.
AI has made the 1-person company a reality. @HubSpot is building products around it.
Do you love @meetgranola for taking meeting notes? Me too!
That's why I've spent this weekend building Granola integration into @YouSpotHub.
Now, you can grab your Granola personal API key and just paste it into YouSpot.
Then, the magic commences. YouSpot will sync your notes, extract entities (like company mentions) from those notes. Link everything together in the context graph and put it all in your Second Brain.
Then, you have access to it via chat, MCP and cloud agents. Oh and your daily brief emails will get smarter by factoring in your meeting notes.
YouSpot is the AI-native Solo CRM built specifically for one person companies. If you're operating a one person company, running a solo side gig or are a creator building a personal brand, you should check it out.
Just go to YouSpot .com -- it's just $10/month and there's a free version too.
Thanks.
The underlying thesis hasn't changed on Bitcoin, if anything it's more relevant today.
This quoted post is unavailable.
CJ 🇺🇸 retweeted
Very cool! Any chance we could get it on hf.co/datasets?
