I’m a software engineer building high-performance kernels and compilers at Anthropic! Previously at Facebook/Meta (PyTorch, HHVM, ReDex)
Joined December 2022
- Tweets432
- Following402
- Followers2.7K
- Likes1.2K
Bert Maher retweeted
I am a proud American, and I’m proud to work at Anthropic. Labeling Claude a supply chain risk will only harm America’s lead in AI. It’s not un-American to support AI guardrails that protect our civil liberties.
A statement on the comments from Secretary of War Pete Hegseth.
anthropic.com/news/statement…
I am deeply proud to work for this company.
A statement on the comments from Secretary of War Pete Hegseth.
anthropic.com/news/statement…
Bert Maher retweeted
One of the most common flaws of math textbooks is that they present only the logic, without the intuition. They give you the later, cleaned up version of the idea, which hides the way it was discovered.
This is all true, but Soumith is also one of the most brilliant strategic thinkers in the world. Some of us just fail a lot, dust ourselves off, and keep hacking the next day ☺️
If you feel like giving up, you must read this never-before-shared story of the creator of PyTorch and ex-VP at Meta, Soumith Chintala.
> from hyderabad public school, but bad at math
> goes to a "tier 2" college in India, VIT in Vellore
> rejected from all 12 universities for US masters despite 1420 on the GRE
> fuckit.jpg
> goes to the US anyway on a J-1 visa to CMU with no plan
> applies for masters (again) to 15 universities
> rejected from all except USC and with late admissions, NYU in 2010
> finds this guy called Yann LeCun (before he was famous)
> starts getting into open source
> rejected from all jobs including DeepMind
> only job is Amazon as test engineer
> his PhD mentor helps him get a job at a small startup (MuseAmi)
> rejected from DeepMind
> couldn't get H-1B because of J-1 home return issue; gets waiver through months of approval with USCIS and US State Dept
> very low on confidence
> In 2011/12 builds one of the fastest AI inference engines on phones
> rejected from DeepMind
> emailed Yann again and joins FAIR because of Torch7 open-source work
> scrapes through bootcamp at Facebook, struggling on an HBase task
> L8/L9 engineers at Facebook struggle to get ImageNet working
> figures out numerics / hyperparam issue as an L4
> first big win!
> FAIR goes well, runs 3 person torch7 team and co-creates PyTorch
> because of politics, management wants to shut down PyTorch
> cries-at-bar.jpg, literally
> eventually some people save PyTorch and it launches in 2017
> gets a EB-1 green card!
> the rest is history...
Think about that. He went to a tier 2 college. Was rejected from all Masters programs 2x. Rejected from every single job except Amazon test engineering. Rejected from DeepMind 3x. Nearly had his baby project shut down. Struggled with visa issues. After 12 years of failures (2005-17), he eventually rose to became a VP at Meta one of the most influential people in AI!
Soumith's story is one of resilience and he's living proof that no matter how down in the dumps you are, there's always hope.
This got me thinking that both int and FP math is “emulated” via a pretty complex set of transistors. I wonder how many gates/transistors it takes to implement an int8 fma versus an fp8, e4m3 fma
I’ve heard this complaint from a couple people recently, and I’m surprised because we optimized the launch path like a year ago and got it down to ~10us. There’s a now closed GitHub issue I filed with a microbenchmark - someone should run it, profile, and bring it down
It would be kind of cool if torch.compile could be used as a context manager, like:
```
some_custom_kernels()
with torch.compile():
# do a bunch of easy pointwise stuff
more_custom_kernels()
```
I am really enjoying using Claude Code with Sonnet 4.5! It's super smart, and super fast!
Introducing Claude Sonnet 4.5—the best coding model in the world.
It's the strongest model for building complex agents. It's the best model at using computers. And it shows substantial gains on tests of reasoning and math.
ALT A benchmark table for Claude Sonnet 4.5 showing leading performance across many domains, including agentic coding, computer use, mathematics, graduate-level reasoning, and financial analysis.
Bert Maher retweeted
Introducing Claude Sonnet 4.5—the best coding model in the world.
It's the strongest model for building complex agents. It's the best model at using computers. And it shows substantial gains on tests of reasoning and math.
ALT A benchmark table for Claude Sonnet 4.5 showing leading performance across many domains, including agentic coding, computer use, mathematics, graduate-level reasoning, and financial analysis.
Good list! Simon’s and Pranjal’s matmul blogs were foundational to my understanding of GPU performance
Some perf related must-reads:
• How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: siboehm.com/articles/22/CUDA…
• Outperforming cuBLAS on H100: a Worklog: cudaforfun.substack.com/p/ou…
• Defeating Nondeterminism in LLM Inference: thinkingmachines.ai/blog/def…
• Making Deep Learning go Brrrr From First Principles: horace.io/brrr_intro.html
• Transformer Inference Arithmetic: kipp.ly/transformer-inferenc…
• Domain specific architectures for AI inference: fleetwood.dev/posts/domain-s…
• A postmortem of three recent issues: anthropic.com/engineering/a-…
• How To Scale Your Model: jax-ml.github.io/scaling-boo…
• The Ultra-Scale Playbook: huggingface.co/spaces/nanotr…
• The Case for Co-Designing Model Architectures with Hardware: arxiv.org/abs/2401.14489
lol, there is quite the explosion of kernel DSLs lately (triton, tilelang, gluon, TLX, cuteDSL, cuTile, …)
And honestly as much as I love TLX and want it to succeed, I think the next big kernel programming language might be… natural, human language
A great example of how subtle numerical issues can bite you. Fusing multiply-add into FMA strictly improves precision of that operation (the mul is computed in infinite precision). But it breaks the larger expression if other parts are not computed with the same high precision!
where have I seen something like this before
"This caused a mismatch: operations that should have agreed on the highest probability token were running at different precision levels. The precision mismatch meant they didn't agree on which token had the highest probability." see also below post by @cHHillee
Debugging subtle numerical issues in ML systems - especially in the compiler! - is really freaking hard. Very impressed by the debugging that went into discovering these
In our investigation, we uncovered three separate bugs. They were partly overlapping, making diagnosis even trickier.
We've now resolved all three bugs and written a technical report on what happened, which you can find here: anthropic.com/engineering/a-…
Since I’ve seen various discussion of Triton and Gluon bouncing around this website, I thought I’d throw TLX into the mix (from my old team!)
github.com/facebookexperimen…
I like that it gives you direct control over warp specialization and concurrency
And it generally retains a higher-level tensor-operation feel like Triton, but with more direct access to shared memory and synchronization, needed for tweaking concurrency
I found myself wondering if we might benefit from a resurgence of Halide- (or TVM) -like ideas, where you have some math to optimize, and a library of optimizations through which the machine can search to find the best outcome
What if instead of a paperclip maximizer we built and hoodie-and-bag maximizer, and what if it turns out we’re already living in its world?? 😱
