semis analyst. asking questions + explaining. read by chipmakers, funds, model labs. @chipstrat · @semidoped
Joined September 2007
- Tweets4.8K
- Following1.8K
- Followers7.8K
- Likes13.7K
Pinned Tweet
We are past the CUDA moat days.
Anthropic says Claude can bootstrap @amd Instinct + ROCm.
@NotTomBrown: "I think the thing that we were thinking about originally was whenever we're bringing up a new hardware platform, it's a big effort. It's like a huge thing. And so as we were thinking about this, we started doing our own evaluation of MI 355. You guys generously got us a rack to start working. And we expected this to be kind of a big process.
Our actual experience was we had one engineer who start doing it. They spun up Claude, asked it, hey, bring up this machine, left it going over the weekend. And we ended up with a graph of the actual performance of our leading model on it, just going up and up and up over the weekend."
Yes good visuals in this one!
Micro-VCSELs vs micro-LEDs
This was a hot topic at OCP 2026
@austinsemis and I sat down on @semidoped right after the show to discuss the future of short reach optics
(even if it means I'm in a garage, because some things are too important to delay 😅)
Enjoy the show! Lots of visuals in there, so make sure you watch on YouTube or right here on X!
Learned a lot from @vikramskr in this one!
MicroVCSELs vs MicroLEDs
And why $CRDO is pursing the former for AI interconnects
🎙️ NEW EPISODE: MicroVCSELs vs MicroLEDs, a MicroExplainer
Austin and Vik host an explainer on the two competing microemitter technologies for AI interconnects, microLEDs and microVCSELs. They break down Credo's surprising pivot at OCP from its microLED acquisition to a microVCSEL product, and the physics that give microVCSELs better speed and reach.
- The new battleground is the 5-50 meter "reach gap," too long for copper limited to ~7 meters and too short for expensive, supply-constrained InP optics
- MicroLEDs are limited to ~3-5 Gbps over 5 meters because their broad light spectrum causes chromatic dispersion that smears the signal
- Hitting 1 Tbps with microLEDs takes ~300 slow lanes, and coupling hundreds of fibers adds a lot of manufacturing complexity
- MicroVCSELs can hit 50 Gbps+ over 50 meters. Stimulated emission gives a narrow beam with less dispersion, for 10x the speed and reach
- MicroVCSELs use a mature Gallium Arsenide (GaAs) process on 8-inch wafers, shared with cell phones, for a cost and scale advantage over InP
- MicroLEDs' niche is heat. Gallium Nitride (GaN) is highly temperature-insensitive, a candidate for short-reach optics inside a hot server tray
This episode is presented by @CrusoeAI. Crusoe's Serverless Fine-Tuning trains an open model like Llama, DeepSeek, or GLM on your own data, and you pay per million tokens, not per GPU hour. About $2 buys roughly 5M training tokens on smaller models. Get $5 in credit to start: semidoped.com/crusoe-x
Chapters:
0:00 The Credo Pivot
3:49 The Talk of OCP
8:13 The 5-50 Meter Reach Gap
12:52 How MicroLEDs Work
21:10 MicroLEDs' Parallelism Problem
22:55 How MicroVCSELs Work
26:57 MicroVCSEL Performance Specs
31:51 The Physics of VCSELs
33:35 The Market Chooses MicroVCSELs
35:45 MicroLEDs' Niche: Heat
38:16 Teasing Future Interviews
Mentioned in this episode:
- PicoJool’s Al Yuen: The Case for GaAs VCSELs in Scale-Up Interconnects (Semi Doped episode): daily.semidoped.com/p/picojo…
- A MicroLED Field Trip to Avicena (Vik's Newsletter, paid): viksnewsletter.com/p/a-micro…
- MicroLEDs vs. Lasers: The Linewidth Tradeoff (Vik's Newsletter, free): viksnewsletter.com/p/microle…
- Understanding Energy Bands in Semiconductors (Vik's Newsletter, free): viksnewsletter.com/p/energy-…
Get more of Austin and Vik daily, free!
Sign up: daily.semidoped.com/
Connect with Vik and Austin:
Vik's Paid Substack: viksnewsletter.com
Austin's Paid Substack: chipstrat.com
@austinsemis @vikramskr
Smart
The @AMD AI DevDay world tour kicks off next Thursday, Oct. 29, in Austin, Texas. Register for an event in a city near you!
amd.cventevents.com/event/8c…
This is the opportunity for AI-enabled software. It’s the easy button for normal users to unlock daily benefits, especially in the enterprise. Does a salesperson want to vibe code their own tool or just use an amazing agentic tool that’s already built for them?
I’m extremely AI pilled and use AI in most facets of life with steady state usage always driving higher
I say that so no one accuses me of being overly bearish but I think the “why you need to stay / keep using” factor is sorely missing for most AI tools for the every day user
Most people don’t have the imagination / creativity / desire to want to pursue coding or even side projects. Daily assistant seems fine but they don’t / won’t offload all their day-to-day to them. It’s a neat parlor trick or treat for one off specific use case (subscription audit, book a trip, etc.)
You’re stuck in this spot where the average person doesn’t know the power of agentic tool capabilities but at the same time the labs/AI companies can’t force people to do more projects with the tools. Need to create power users that come BACK to the products every day
Austin Lyons retweeted
Maia 300's scale-up fabric allows prefill/decode disaggregation without crossing a scale-out network. The KV cache is passed all on the same fabric, from prefill to decode instances.
Austin Lyons retweeted
We just finished recording, “a micro explainer on micro LEDs and micro VCSELs” — should be a good episode!
Austin Lyons retweeted
The all new shiny Microsoft Maia 300, and the details brought straight to you by @semidoped
Check it out!
🎙️ NEW EPISODE: @Microsoft Maia 300 with Saurabh Dighe
Why Maia 300 makes memory bandwidth and capacity a first-order design principle.
- Moving data to feed the compute engine can cost two orders of magnitude more energy than the compute itself
- Maia 200 had six HBM3E stacks at 7 TB/s. Maia 300 has eight HBM4 stacks at 18 TB/s
- Compute grows about 35% while rack-level memory bandwidth grows 5x
- 7x token throughput per rack over Maia 200 at 2.5x perf per watt
- Prefill and decode are disaggregated, and the KV cache moves over the scale-up fabric instead of the slower scale-out network
- Memory interface chiplets "extend the shoreline" so the TSMC N2P compute die stays one efficient die
This episode is presented by @CrusoeAI. Crusoe's Serverless Fine-Tuning trains an open model like Llama, DeepSeek, or GLM on your own data, and you pay per million tokens, not per GPU hour.
About $2 buys roughly 5M training tokens on smaller models. Get $5 in credit to start: semidoped.com/crusoe-x
Chapters:
0:00 Intro: Saurabh Dighe
1:39 Microsoft's Silicon Philosophy
5:52 Workload Shift: Agentic AI
9:32 The Data Movement Problem
14:20 Crusoe Ad Read
17:47 Maia 300 Specs
24:07 Chiplets & HBM4
29:12 Prefill/Decode Disaggregation
30:33 Two-Tiered Network
35:45 NPO vs. CPO
36:46 Rack-Scale Power & Cooling
41:35 Agentic Kernel Optimization
49:08 What's Next: Maia 400
Get more of Austin and Vik daily, free!
Sign up: daily.semidoped.com/
Connect with Vik and Austin:
Vik's Paid Substack: viksnewsletter.com
Austin's Paid Substack: chipstrat.com
@austinsemis @vikramskr
Austin Lyons retweeted
Maia 300 networking insights:
1. The NIC is on the die and built only for scale-up: low latency, high bandwidth between accelerators.
2. Tier 1 is copper inside the rack. Neighboring Maias talk the most (tensor parallelism), so about half of each chip's bandwidth stays in the server and half goes to the in-rack switch. 96 accelerators per rack.
3. Tier 2 is optics between racks, and it's still scale-up. Up to 128 racks, 12,288 accelerators, one domain, one communication stack.
4. Standard Ethernet switches and cables for the scale economics, with Microsoft's own transport on top. The NIC and transport are where Microsoft does its own engineering.
5. Why go past one rack? Bigger models, more aggressive sharding, and flexible job placement. The scheduler can put a job anywhere in 12,288 chips.
6. Fast data movement. Prefill and decode run as separate pools, and the KV cache moves between them over the scale-up fabric, never the slower scale-out network. For agents with long context, that's the expensive data movement.
🎙️ NEW EPISODE: @Microsoft Maia 300 with Saurabh Dighe
Why Maia 300 makes memory bandwidth and capacity a first-order design principle.
- Moving data to feed the compute engine can cost two orders of magnitude more energy than the compute itself
- Maia 200 had six HBM3E stacks at 7 TB/s. Maia 300 has eight HBM4 stacks at 18 TB/s
- Compute grows about 35% while rack-level memory bandwidth grows 5x
- 7x token throughput per rack over Maia 200 at 2.5x perf per watt
- Prefill and decode are disaggregated, and the KV cache moves over the scale-up fabric instead of the slower scale-out network
- Memory interface chiplets "extend the shoreline" so the TSMC N2P compute die stays one efficient die
This episode is presented by @CrusoeAI. Crusoe's Serverless Fine-Tuning trains an open model like Llama, DeepSeek, or GLM on your own data, and you pay per million tokens, not per GPU hour.
About $2 buys roughly 5M training tokens on smaller models. Get $5 in credit to start: semidoped.com/crusoe-x
Chapters:
0:00 Intro: Saurabh Dighe
1:39 Microsoft's Silicon Philosophy
5:52 Workload Shift: Agentic AI
9:32 The Data Movement Problem
14:20 Crusoe Ad Read
17:47 Maia 300 Specs
24:07 Chiplets & HBM4
29:12 Prefill/Decode Disaggregation
30:33 Two-Tiered Network
35:45 NPO vs. CPO
36:46 Rack-Scale Power & Cooling
41:35 Agentic Kernel Optimization
49:08 What's Next: Maia 400
Get more of Austin and Vik daily, free!
Sign up: daily.semidoped.com/
Connect with Vik and Austin:
Vik's Paid Substack: viksnewsletter.com
Austin's Paid Substack: chipstrat.com
@austinsemis @vikramskr
Saw Saurabh at OCP yesterday!
Here's an insightful chat we had recently about the Maia 300.
$MSFT
🎙️ NEW EPISODE: @Microsoft Maia 300 with Saurabh Dighe
Why Maia 300 makes memory bandwidth and capacity a first-order design principle.
- Moving data to feed the compute engine can cost two orders of magnitude more energy than the compute itself
- Maia 200 had six HBM3E stacks at 7 TB/s. Maia 300 has eight HBM4 stacks at 18 TB/s
- Compute grows about 35% while rack-level memory bandwidth grows 5x
- 7x token throughput per rack over Maia 200 at 2.5x perf per watt
- Prefill and decode are disaggregated, and the KV cache moves over the scale-up fabric instead of the slower scale-out network
- Memory interface chiplets "extend the shoreline" so the TSMC N2P compute die stays one efficient die
This episode is presented by @CrusoeAI. Crusoe's Serverless Fine-Tuning trains an open model like Llama, DeepSeek, or GLM on your own data, and you pay per million tokens, not per GPU hour.
About $2 buys roughly 5M training tokens on smaller models. Get $5 in credit to start: semidoped.com/crusoe-x
Chapters:
0:00 Intro: Saurabh Dighe
1:39 Microsoft's Silicon Philosophy
5:52 Workload Shift: Agentic AI
9:32 The Data Movement Problem
14:20 Crusoe Ad Read
17:47 Maia 300 Specs
24:07 Chiplets & HBM4
29:12 Prefill/Decode Disaggregation
30:33 Two-Tiered Network
35:45 NPO vs. CPO
36:46 Rack-Scale Power & Cooling
41:35 Agentic Kernel Optimization
49:08 What's Next: Maia 400
Get more of Austin and Vik daily, free!
Sign up: daily.semidoped.com/
Connect with Vik and Austin:
Vik's Paid Substack: viksnewsletter.com
Austin's Paid Substack: chipstrat.com
@austinsemis @vikramskr
Austin Lyons retweeted
We take you on an exclusive tour inside a secretive AWS networking lab. If you care about networking, timing, optics, and hyperscale, this is an awesome one youtu.be/yFSDB1BatrQ
Austin Lyons retweeted
1.6T LPO from Linktel
No DSP, and you can see pre-FEC BER is about 1e-9.
Temps stay cool, at 35C due to no DSP
The deployment is LPO is expected to be limited to specific use cases.
LPO makes it difficult for the link to work with different vendors on either end. The link needs to be trained on a per port basis, and parts of the CTLE and amp are adjusted to close the links.
This extra work is what LPO difficult and limits its widespread use
So many fibers
Lots of NPO
But you can still engineer reliability by over provisioning and having telemetry
Austin Lyons retweeted
Semi Doped hits up OCP!
@austinsemis and @vikramskr grabbing front row seats to the keynotes
