codergoose retweeted
the labs are running Jurassic park. they’re birthing a T-Rex for the first time in 70 million years. it’s an enormous cost to understand how to safely hold him and what he’s capable of, as verified third party scientists and assessors. everyone is on edge, dotting their i’s and rattling the cages to see if they’ll break. they’re putting T. rex in all kinds of difficult scenarios to see what he can do, waiting for novel behaviors to arise
by the time that’s done, it’s far simpler for a new entrant to create contain and secure a Stegosaurus. They have to accept some safety costs, but are basically well understood because someone has done it before. The labs will have to publish their safety cases for why a certain set of model alignment and containment standards work for a certain level of capability. while it will impose some nonzero costs to secure Stegosaurus, it will be standardized and much cheaper than securing a T. rex for the first time
it is possible T. rex breaks loose and eats us all no matter what we do. this should buy us time so we at least avoid unforced errors and blunders
codergoose retweeted
RSI is not a binary event, it is a continuum of compounding loops -- and we are already in the continuum.
codergoose retweeted
We are at the dawn of Superintelligence.
Introducing the Recurrent Looped Transformer (RLT),
We now have Transformers with Infinite Reasoning depth.
From now on, we should pace progress at the Open Frontier of Superintelligence,
Until Safe Superintelligence is achieved.
github.com/yifanzhang-pro/re…
codergoose retweeted
sharing a new long-form blog post: ai chip architectures
it covers the leading chip architectures (nvidia, amd, tpus, trainium, cerebras, groq) across architecture, scaling (scale-up and scale-out), and software stacks.
it helps build an intuition for the architectures and their trade-offs.
jacobpeake.com/ai-chip-archi…
codergoose retweeted
All of this assumes you paid cash for the GPUs but almost nobody does.
Neoclouds borrow heavily to buy GPUs and pledge the GPUs themselves as collateral. And that changes the picture two ways:
1. Debt service. At ~10–15% financing, a $1.2M H100 pod carries ~$10–15K/mo in interest before you serve a single token. That eats roughly a third of your net and pushes break even math from ~2 years past 3.
2. Depreciation is even worse Loans amortize on an assumed 5–6 yr useful life but H100 rental have fallen from ~$8/hr to under $3/hr in only two years. If the collateral halves in its economic value while the loan is still on a 6-yr schedule, you’re underwater because the asset is depreciating faster than the debt.
Now the most dangerous part here is unlike regular cloud hosting this is a treadmill. Every new generation of chips makes the last one’s economics worse, meaning you have to keep borrowing to stay competitive:
Say you bought all those h100 pods and Rubin’s land with a 2 or 4x TCO efficiency gain, the collateral backing your loan is worth maybe half what you paid while you still owe most of the principal.
To stay in the neocloud game you need to take on MORE debt to buy Rubin pods. So now you’re servicing old debt on H100s that are earning less and worth less new debt on Rubin pods that themselves start depreciating the day Feynman is announced.
New GPU generations roughly coming every 2 years if stack this math across 2–3 generations the debt servicing compounds: each cycle you’re carrying a layer of underwater legacy inventory + fresh leverage on the newest tier. Depreciation isn’t a one-time hit either it’s a step down that repeats every ~18 to 24 months, and the debt from each cycle outlives the earning power of the hardware it bought.
The math only works if demand + rental rates keep rising fast enough to outrun the depreciation curve. The moment demand blinks the utilization dips or rates compress you’ve got fixed debt payments on a stack of assets losing value faster than you can pay them down. That’s how a an amazing “80% margin” business becomes insolvent without revenue ever going to zero.
Light-asset model layer holds pricing power more but have their own risks. It’s clear however that the neocloud layer is on a never ending treadmill and will probably get squeezed first when push comes to shove.
codergoose retweeted
this smells very bad. none of the claims make sense.
> low voltage hits 80% mfu
high mfu is easy if your peak flops is low
> low voltage solves power bottleneck
bleeding edge wafers is more scarce than power and it doesn't make sense to tape out lower perf chips on these wafers to save power if you want perf/$. the true bottleneck to frontier perf is max flops density to minimize going off die. power is the most fungible elastic commodity on earth
> half voltage = quarter power
presumably:
1. only matmul gates (~60% of die power) at half voltage. half voltage -> ~3x slower -> need ~3x more transistors, and their power leaks scale linearly. ~20% net savings at chip level 2. half voltage -> engineering complexity to fix timing violations + exponentially scaled soft errors (~20x). matmuls will randomly corrupt undetectably from bit flips (and drop model intelligence).
> btc miners run at 3x lower V
btc workload is hashing, so 1) error checking is literally in the problem 2) arithmetic intensity is like infinity so they don't care about packing flops density in single die unlike AI workloads. doesn't make sense to copy
> GPUs get low MFU from thermals
no, they get low MFU from mem bw and non-matmul ops.
> CSM 5x lower latency than Blackwell 4000ns
Blackwell nvswitch is 300ns? unless they are comparing their bare hardware to nvidia hardware + software.
the whole product design tradeoffs don't make any sense from first principles and only makes sense if their initial "transformer asic" had some horrible power issue that they solved by undervolting and they had to respin the whole story for investor marketing.
would love to be proven wrong if anyone from Etched wants to educate me :)
codergoose retweeted
GLM-5 paper, arxiv.org/abs/2602.15763
GLM-5.1 blog post, z.ai/blog/glm-5.1
GLM-5.2 blog post, z.ai/blog/glm-5.2
codergoose retweeted
Early results from the @snowflake's coco team on GLM-5.2 vs Opus-4.7 on dbt-bench — what the trajectories actually show 🧵
codergoose retweeted
There has been a lot of commotion in the photonics world after a SemiAnalysis report claimed CPO was delayed. This had a direct impact on photonics-related stocks: $LITE $AAOI $COHR $MRVL $TSEM
First, you have NVIDIA pushing back on it.
Gilad Shainer, SVP of Networking, addressed the delay claims during GTC/Computex in Taipei -- confirming Spectrum-X Ethernet CPO switches remain on track for mass production and customer deliveries in H2 2026. He also revealed NVIDIA has already begun shipping the Spectrum-X CPO switch (developed with TSMC) to select partners, with throughput up to 400 Tbps and production capacity expected to increase through the second half of the year.
Then you have Lumentum, one of the most important photonics companies.
CEO Michael Hurlston made the case that optical scale-up is a generational opportunity -- bigger than scale-out. He framed it as a fourth growth driver and a "generational game-changer," with first scale-up CPO shipments expected by late 2027, replacing longer copper connections inside data centers.
Mizuho projects CPO penetration growing from ~4% in 2026 to ~30% by 2029 -- a 153% CAGR over that window -- with a 3x intensity increase vs. current scale-out and a major inflection expected in 2028.
I'd rather bet on the companies making the moves than the analysts drawing their own conclusions.
Bullish photonics, I don’t believe in this FUD.
codergoose retweeted
BREAKING NEWS: Anthropic's latest model will NOT help you if it thinks your ML research/ML engineering is interesting, and/or will secretly degrade its IQ so that the average engineer won't notice. We are already seeing Anthropic's latest model's moderation filters our GPU inference research and programming 😭
codergoose retweeted
This is a super exciting release - Claude Fable 5 is the same underlying model as Mythos but with added safeguards. The benchmarks are great and it's SOTA on everything by a margin but I'll add that *qualitatively* also, this is a major-version-bump-deserving step change forward (imo of the same order as Claude 4.5 was in November), peaking especially for long problem-solving sessions on very difficult problems. You can give it a lot more ambitious tasks than what you're used to, the model "gets it" and it will just go, and it's never felt this tempting to stop looking at the code at all (but don't do this in prod!). The model still has quirks that people will run into and the safeguards are configured to be a little too trigger happy for launch, which can hopefully be tuned over time.
I feel a lot of things changing as working software increasingly comes out on a tap. The Jevon's paradox kicks in and I feel my own demand for software growing substantially. You can ask for anything - explainers, visualizers, dashboards, bespoke single-use apps (e.g. a full wandb that is hyper-specific just for your project), you can 10X your test suite, auto-optimize code, run giant research projects with custom HTML for the results, anything! "Free your mind" (Matrix ref). Really looking forward to all the things people build!
Fable 5 is state-of-the-art on nearly all tested benchmarks, with exceptional performance in software engineering, knowledge work, scientific research, and vision.
The longer and more complex the task, the larger Fable 5’s lead over our other models.
ALT Benchmark table titled Mythos 5 & Fable 5, comparing Claude Mythos 5 and Fable 5 against Claude Mythos Preview, Claude Opus 4.8, GPT 5.5, and Gemini 3.1 Pro.
codergoose retweeted
Each time we release a model, we run the same test: give it code that trains a small AI model, ask the new model to speed it up. It takes a skilled human 4-8 hours to reach 4x faster.
In May 2024, Claude Opus 4 averaged a ~3x speedup. This April, Mythos Preview achieved ~52x.
codergoose retweeted
For over a decade, we’ve accepted that end-to-end backprop is the only way to train deep networks. But holding the entire network in memory all at once is why AI training is hitting a resource wall.
We found a new way to break the network into blocks and train them independently. The trick? Treating the network’s forward pass like a diffusion model denoising a signal.
This reinterpretation slashes the memory needed to train deep models. In our #ICLR2026 paper (arxiv.org/abs/2506.14202), we matched end-to-end performance across ViTs, DiTs, and LLMs. We did this while training just one isolated block at a time.
Introducing DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation
pub.sakana.ai/diffusionblock…
What if we didn’t have to hold an entire neural network in memory to train it?
Standard neural net training optimizes all parameters jointly. As a result, the memory required during training grows linearly with the depth of the network.
In our #ICLR2026 paper, we propose DiffusionBlocks, a principled framework to train networks one block at a time, drastically reducing memory requirements while matching end-to-end performance.
With DiffusionBlocks, we split the network into blocks and train them one at a time, so you only need memory for a single block.
How? We explicitly assign each block a role: to move the representation a little closer to the target than the block before it did. That role turns out to be precisely what a diffusion model does, step by step. Each block only needs to optimize its own objective and can be trained independently.
We validated this across five different architectures:
• ViT
• DiT
• Masked diffusion
• Autoregressive transformers
• Recurrent-depth transformers
In each case, performance is competitive with end-to-end training while using a fraction of the memory.
This perspective also extends naturally to recurrent-depth (Looped) transformers, which apply the same network iteratively and normally require expensive backpropagation through time (BPTT). Viewed through DiffusionBlocks, we can replace those multiple iterations with a single forward pass during training.
Read our paper and code, to learn more.
Paper: arxiv.org/abs/2506.14202
GitHub: github.com/SakanaAI/Diffusio…
🐟
codergoose retweeted
instead of watching 2 hours of Netflix tonight, watch this 40-minute masterclass from the founder of a $20B China AI company
it's the clearest explanation I've seen of how Agent Swarms and AI systems actually work at scale
useful whether you've never built an agent in your life or have been using Claude every day for the past year
I took the key ideas and turned them into a practical guide on how to actually build with Kimi
find it below
codergoose retweeted
Since GPT-4o, frontier average scores on METR-Horizon have been remarkably predictable over time.
A simple linear fit of average score vs. release date gives R² = 0.984.
The relationship between average score and log time horizon is also extremely strong:
- p50 horizon: r = 0.998
- p80 horizon: r = 0.992
Claude Mythos scored 85.21%, slightly above the ~83.3% predicted by the pre-Mythos linear trend.
The implied doubling time for METR time horizons is still about 103 days, the same value we reported on February 12th, 2026.
If current trends continue:
- 90% score: July 7, 2026
- implied p50 horizon: 27.5 hours
- implied p80 horizon: 4.8 hours
- 95% score: September 18, 2026
- implied p50 horizon: 44.9 hours, or 1.9 days
- implied p80 horizon: 7.8 hours
- 100% score: November 30, 2026
- implied p50 horizon: 73.4 hours, or 3.1 days
- implied p80 horizon: 12.8 hours
Truthseeking.
boris responded to this in depth in the issue- it's mostly just that we stopped showing thinking summaries for latency (you can opt-in to showing it) which was affecting the thinking measurement in the post
github.com/anthropics/claude…
codergoose retweeted
Super interesting:
- Google has nearly 25% of all compute, a lot from their own TPUs
- Goog + Msft + Amazon + Meta is over 60% of compute
- Amazon has more Nvidia than Tranium
- China and Oracle have similar levels of compute
- A third of China's compute is from NVIDIA
