Google Developer Expert in AI/ML in JAX/FLAX | Docker Captain | Machine Learning Scientist in Quebec
🇨🇦🇫🇷🇨🇴
Joined May 2010
- Tweets2K
- Following1.9K
- Followers676
- Likes5.1K
Pinned Tweet
“Geography has made us neighbours. History has made us friends. Economics has made us partners. And necessity has made us allies."
"From the fields of Flanders to the streets of Kandahar."
Vive la Canada, Team Canada
The Maple Leaf Forever
lean.brettkoonce.com/bluepri…
My yearly comment of why lean should be a math and CS course
David Cardozo 🇨🇦 retweeted
if you have a startup about private LLM inference today is the perfect launch date
David Cardozo 🇨🇦 retweeted
New blog! With @_DivyaMakkar we present a worklog on how to build an asynchronous RL framework from scratch. It covers multi-actor systems in JAX, inference, distributed training, actor coordination, ICI weight sync and more! Additionally we share interesting JAX challenges encountered along the way and how you should think of async RL from a systems perspective!
David Cardozo 🇨🇦 retweeted
If you want to give credit, either do it like this or don't do it!
Joan and Esther, please do a course about hosptitality and how to make people feel welcome.
David Cardozo 🇨🇦 retweeted
Had the honor of presenting about optimization in flow models at the BCS Department, MIT, and it felt good!
Slides: docs.google.com/presentation…
YT: youtu.be/okgOtuRUCBs?si=g9Hr…
Thanks to the organizers for having me!
David Cardozo 🇨🇦 retweeted
Aritra started the profiling series when we were in Japan, getting addicted to eating Ramen.
But from there on, he undertook a rigorous effort to ensure the community becomes comfortable generating and reading profile traces in PyTorch.
Almost all the serious practitioners I know of who are working in this space know that profiling is an absolute basic if one wants to understand WTF is going on inside an accelerator. Why is it idle? Why is the model not fast enough? What can be optimized further?
However, the chatter around profiling seemed quite quiet, almost as if everyone knew it fully. I think we can agree that it's not the case at all.
Hopefully, the three-part blog series takes a good first step at TEACHING you the importance of profiling and how to use it in the grand scheme of things.
Rarely does the PyTorch documentation feel insufficient, but it did for profiling. So, I hope this series closes that gap!
Get started with the series here:
huggingface.co/blog/torch-at…
David Cardozo 🇨🇦 retweeted
our blessed post-training / their barbarous distillation attack
🤖 Made with AI
David Cardozo 🇨🇦 retweeted
The final part of the three part "Profiling in PyTorch" series is out now!
In Part 1 we profiled addition and multiplication
In Part 2 we profiled a torch Linear layer (and also MLP)
In this part we profile Attention.
> a naive implementation of attention
> talk about the torch sdpa API
> math backend
> xformers backend
> flash backend
> cuDNN backend
PS: This is a long read, bookmark this and read it over the weekends. 🤗
Did I just got insulted?
(I got this while pairing with an LLM)
Your instinct is actually legally sound in structure — it just doesn't survive contact with the evidence.
David Cardozo 🇨🇦 retweeted
Why are we still building LLMs with uniform-width layers?
For a decade, we've blindly copied the flat stack from the original Transformer. It turns out, uniform capacity is a massive waste of parameter budget.
Another recent paper about non-uniform layers 🧵
David Cardozo 🇨🇦 retweeted
hello Venezuelan friends, we're very heartbroken about the earthquake.
3 years ago we had a similar earthquake in Turkey where we lost 50.000 souls. to help people out we built a "needs map" where we scraped posts online and put address and name and what the person needs (food, water, saving) so others could help and we managed to get to people. we'd love to help you out, but actually work with local Venezuelan folk in ML, and at HF we are happy to cover inference. please respond to this post if you are interested 🙏🏼 the project is outdated so it would take a bit and you would need people to label a dataset shortly but other than that it should work
David Cardozo 🇨🇦 retweeted
This slide right here is the reason why I love JAX so much. Call it my bias or whatever, but I absolutely love writing JAX
