
Compiler engineer @cerebras. Compiler correctness / formal methods enjoyer. Opinions are my own, retweets are not endorsements.
Dubai
Joined May 2019
- Tweets282
- Following638
- Followers734
- Likes2.5K
Fearless Engineering: The CS-4 is our first engineering documentary.
Through lab footage and conversations with the engineers behind each breakthrough, the film follows the creation of the CS-4, our latest wafer-scale AI system.
0:00 introducing the engineering team
1:09 Wafer-scale history
2:28 Powering the wafer
4:00 The cooling problem
4:18 Designing the cold plate
5:11 Holes in the wafer
6:33 the alignment problem
7:47 The CS-4 PRD
8:35 Designing for deployability and serviceability
11:01 Removing the motherboard
11:25 Removing internal cooling loops
11:41 inventing the power rack
12:35 A one-person lab deployment
14:03 Backpack: 50% fewer components
14:55 Connecting the compute backpack to cooling manifold
17:58 shifting Cerebras’s focus from training to inference
19:09 Building the Cerebras Cloud
20:01 CS-3 vs. CS-4 performance
20:49 3D Stacked DRAM
How we deploy multi-trillion parameter models at scale
Today, let's break down the math for how @cerebras:
1. Fits the full model into on-chip memory
2. Maps the model across multiple chips
3. Handles a high # of concurrent requests
When we started @cerebras, we knew we would have to solve difficult problems in yield, power, and cooling. What became clear was that making wafer-scale work would require rethinking almost the entire system.
Today, we answered questions from the community about those engineering decisions, how we run larger models, and what it takes to program the hardware.
2:15 Why hasn't NVIDIA gone wafer-scale?
4:10 How we build a working chip out of an imperfect wafer
8:07 What happens when a model won't fit on one wafer?
11:34 Why we chose SRAM over HBM
14:26 How much of a barrier is NVIDIA's CUDA moat?
Anton Trunov retweeted
The work on Random Testing of a Higher-Order Blockchain Language has been presented by @ilyasergey at @icfp_conference in Ljubljana.
Co-authored by Tram Hoang, @falsenov, and @LeoLambro.
The paper: dl.acm.org/doi/abs/10.1145/3…
Anton Trunov retweeted
As JetBrains we condemn the attacks taking place. Our hearts are with the Ukrainian people, including our own colleagues and their families.
Anton Trunov retweeted
If you're a user, a prospective user, or a learner of the Coq proof assistant, we encourage you to participate in the Coq community survey 2022:
sondages.inria.fr/index.php/…
(by 28 February 2022, AoE).
We are grateful for all your feedback! 🐓✨👍
Anton Trunov retweeted
A biased, love letter to #seL4. Everyone knows that seL4 was the first general purpose OS microkernel whose implementation was verified functionally correct. But did you also know ... ? 1/15
Anton Trunov retweeted
"Counterexamples in Type Systems", an excellent new resource on type system corner cases by the inimitable Stedolan. Give it a look 👀
counterexamples.org/intro.ht…
Can you guess what this little cute Coq function does?
nat_rect (fun _ => nat) 0 Nat.sub
where nat_rect is the dependent eliminator for the Peano natural numbers type and Nat.sub is subtraction.
(yeah, yeah, it has quadratic time complexity, I know)
Anton Trunov retweeted
#DevZen Episode 330 — "Формальный Ко-ко-кок" devzen.ru/episode-0330/
\w @falsenov @defnull @svetlambda @afiskon @gliush
Really honored and excited to join the London Type-Driven Development meetup! Let's hope I'll live up to a fraction of the 'expert' title 😅. Btw, the picture is taken from the cover of 'Programs and Proofs' book by @ilyasergey, courtesy of Lilia Anisimova liliaanisimova.com
This quoted post is unavailable.
13
Anton Trunov retweeted
We are hiring a research programmer at Minnesota to build next-generation proof assistant technology based on HoTT and cubical type theory. favonia.org/recruit.programm…
Anton Trunov retweeted
Hi. For some time now Amin Timany and I have been developing a wiki for PL, modelled after the nLab. I've been using it as my personal notebook for my own projects. Please join and share anything you can remember: pls-lab.org

















