Changing the game for AI inference with @ Cerebras. Played GAA far too long. Sunny days on the right bank or Barolo
Ireland
Joined July 2011
- Tweets10K
- Following2K
- Followers1.8K
- Likes11.5K
Jason ward retweeted
Today’s guest is @achterbrain , co-founder and CTO of @CallosumAI .
Jascha’s bet is that the future of compute is heterogeneous: different chips working together on different parts of a problem.
Beyond the multichip future, we discuss:
- What neuroscience taught Jascha about designing computing systems
- How chipmakers discover what their hardware is best at
- Where thermodynamic, biological, and quantum computing could fit
- Hiring people who welcome having their ideas challenged
- Building an AI infrastructure company in London
Timestamps:
0:00 Introduction
2:55 What heterogeneous computing actually means
11:02 From PhD research to Callosum
18:36 Turning heterogeneous compute into a product
23:24 Sourcing compute and working with chipmakers
28:53 Matching workloads to the right hardware
34:24 What heterogeneous computing can do today
38:29 The next waves of chips
48:02 Building the team
51:33 Why build in London?
56:11 Helping enterprises upgrade their compute stacks
1:01:04 Fundraising and building momentum
1:03:40 Where specialization wins, and where it doesn’t
Listen to the show, and follow along for new episodes every Tuesday.
Jason ward retweeted
A very thoughtful explanation of the benefits of disaggregated inference. At @cerebras we deliver the fastest AI in the industry and have pioneered disaggregation.
Jason ward retweeted
$CBRS SemiAnalysis causing the “misunderstanding,” then backpedaling behind the paywall lol 🤦♂️ Shameless.
$CBRS SemiAnalysis seems to have a grudge against $CBRS. Most of what they say about Cerebras comes across as out of context, negative spin, biased sniping. Pretty consistent now. Makes me wonder whether Cerebras not buying their very expensive “research products” has something to do with it?
Cerebras’s Sarah Chieng had a pretty fitting response: “all this just to get us to pay for your newsletter.” Their very expensive “research products,” reduced to “your newsletter.”
An industry CEO has publicly alleged a “pay-to-play” dynamic. Hot Aisle CEO Jon Stevens argues that SemiAnalysis’s negative coverage is deliberately used to create demand for its consulting services.
The allegation: bash a company, put pressure on it to “fix” the narrative, then sell consulting to help fix it. Stevens points to AMD and acknowledges the payment connection is speculative.
SemiAnalysis openly sells research and consulting to the industry it covers. Then we get “massively cope-tweeting” jabs at Cerebras employees and negative spin dribbled out that leaves out context and sometimes seems to leave out the facts too. People should be asking who pays them and whether those relationships shape the coverage.
jon4hotaisle.substack.com/p/…
Jason ward retweeted
There is some speculation about our partnership with Cerebras.
Cerebras is a close partner, and we have a deep engagement pushing on the frontiers of speed.
Jason ward retweeted
I joined AMD right after graduating from MIT. Two stints there and a decade building @cerebras later, I still lose sleep over how much memory and compute to put on a chip.
I’ve spent my entire career in computer architecture, and I’ve never seen the problems change this quickly. Models keep getting larger. The demands on the hardware keep growing. Decisions about where to put memory and how to connect it have become central to what people can build with AI.
We chose SRAM for its bandwidth. Now we’re working on wafer-scale stacked DRAM to give larger models more capacity while preserving fast inference. That brings us back to some very physical problems: how to power it, cool it, and manufacture it reliably.
This is the work I love. In part two of our AMA, I answered your questions about those decisions and where we’re going next.
0:30 What changes between CS4, CS5, and CS6?
2:19 HBM already stacks memory. What’s different here?
3:34 We chose SRAM for speed. Why add DRAM?
4:52 Drawing the stack is easy. Making it work isn’t.
7:23 The memory-versus-compute decision that keeps me up at night
Jason ward retweeted
Very interesting post by @mschoening of @NotionHQ .
In some domains AI speed even trumps intelligence, when the intelligence is good enough. Speed makes products better.
Is 500 tokens per second per user the new slow?
Jason ward retweeted
Great conversation with @jaltma and @ericvishria
Eric and Benchmark have been an excellent partner to @cerebras since the very beginning.
We get into the near-death years and much more.
Worth a listen.
New episode of Uncapped with the founder of Cerebras @andrewdfeldman and my partner @ericvishria.
We talked about the company's winding road to success, the market around chips, the current state future of the AI supply chain, the role of investors and Eric's relationship with the company, and more.
This was one of my favorites, hope you enjoy.
Timestamps:
(0:00) Intro
(1:07) Why Andrew started Cerebras in 2016
(2:54) Eric on investing without experience in chips
(4:10) Attacking Goliath
(9:44) Near-death experiences and the valley of death
(10:44) 18 months of "still can't make it"
(12:16) Solving a 75-year-old compute problem
(13:26) What comes after wafer scale
(16:19) The chip supply chain explained
(22:30) Why the US punted a strategic industry
(25:51) How to be a good hardware board member
(27:50) Hardware vs. software investing
(29:05) The pivot from training to inference
(27:00) Specialization vs. flexibility
(35:22) Building effective teams for chips
(39:14) External relationships and TSMC
(42:20) The AI infrastructure buildout
(44:12) The data center supply chain
(53:14) Speed creates markets
(54:10) Disaggregation with AMD and AWS
(55:24) What actually makes Nvidia great
(57:30) Near-death experiences
(59:58) Andrew's childhood next to William Shockley
Jason ward retweeted
Speed makes AI productive and fun.
@Alibaba_Qwen 3.8-27B is now live on @cerebras Inference Cloud.
Up to 16x faster than GPUs.
Thank you to the @Alibaba_Qwen and @cerebras teams for bringing this to life.
Build with it today: cloud.cerebras.ai
Jason ward retweeted
What excites me most about Jalapeño is the design methodology behind it. Using AI to help build chips faster and push both throughput and latency forward could benefit the entire industry.
I spoke with @swyx on @latentspacepod about why I see this as the future of chip design.
For those of you interested in learning more about our cool technology- Sean explains why we are so fast , so different but also so easy to work with
When we started @cerebras, we knew we would have to solve difficult problems in yield, power, and cooling. What became clear was that making wafer-scale work would require rethinking almost the entire system.
Today, we answered questions from the community about those engineering decisions, how we run larger models, and what it takes to program the hardware.
2:15 Why hasn't NVIDIA gone wafer-scale?
4:10 How we build a working chip out of an imperfect wafer
8:07 What happens when a model won't fit on one wafer?
11:34 Why we chose SRAM over HBM
14:26 How much of a barrier is NVIDIA's CUDA moat?
Jason ward retweeted
Our partnership with @mbzuai goes back many years. Today, Cerebras is a launch partner for K2 Horizon, a new family of six open models from the @IFM_AI team.
All six models are released as open weights under Apache 2.0, giving developers the freedom to inspect, reproduce, adapt, and build on the work.
Congratulations!
Jason ward retweeted
"You now have new surface area to expose the benefits of the silicon in new ways.
We had some really amazing results when doing this over a huge amount of different silicon. Cerebras was unbelievably good.
We could actually not only improve the speed by a significant amount in these long-context reasoning problems, we also found that we could also make them very cost-effective because we're being efficient with the tokens."
— @DanAkarca, co-founder of @CallosumAI, in conversation with our Andy Hock on the new Cerebras-Callosum partnership.
Jason ward retweeted
We put OpenAI's GPT-OSS on the @cerebras CS-4 and raced it against the competition.
The task: build the full periodic table of elements.
CS-4 finished in 4 seconds, hitting >4,000 tps
The GPU took >1 minute 40 seconds.
In AI, speed is productivity.
And at Cerebras, we build the fastest AI in the world.
Jason ward retweeted
Today, we announced a new 165 MW data center in Mikkeli, Finland, in partnership with Compute Nordic Finland.
Construction of the first 50 MW is already under way.
Two things make this project special to me:
First, we designed the Mikkeli data center to give back. The facility is designed to capture the heat our computers generate so the community can use it, and the water that cools our systems gets recirculated instead of drawn from the town's supply.
Second, it creates long-term jobs. An estimated €1.0–1.7 billion will be invested into the region and it will create up to 250 permanent technical jobs in South Savo.
Thank you to Compute Nordic Finland, the City of Mikkeli, and our network of Finnish partners for helping bring this to life.
Interesting interview, all companies are going to have to become inference companies !!
Sam Altman (CEO of OpenAI):
"You no longer need to write prompts."
In just 38 minutes, he explains how to use ChatGPT at a level that most people can't even imagine.
It's a talk he gave to Stanford students. A friend sent me the recording last night.
After watching it, I realized I was only taking advantage of about 15% of what this tool can really do.
Watch it in full and then read the guide I leave below on how to create a system that prompts itself.
Jason ward retweeted
Ten years ago, we started @cerebras around an approach many believed was impossible.
As a computer architect, it is hard for me to imagine a more exciting time. Model releases are accelerating, and hardware tapeout is compressing from multi-year roadmaps to annual launches.
Hot Chips is my favorite conference, and it’s where I launched Cerebras 7 years ago. This year’s conference was especially exciting, and so much innovation was shared. I am watching the industry recreate itself: SRAM is mainstream, DRAM is moving into the third dimension, networks are being fundamentally redesigned, and AI is helping design and program the chips themselves.
The industry has never moved faster and some of the hardest architectural questions are still wide open.
Jason ward retweeted
Congratulations @andrewdfeldman on being included in @TIME's list of TIME100 AI. 🟧🚀
