@bsaeta

Currently working on the amazing JAX team (https://xitter.cf/t.co/rWBc4ar9jy). Former: Tech lead for Cloud TPUs (TensorFlow), Swift for TensorFlow

 Google - Brain - JAX
Joined May 2009
  • Tweets1.1K
  • Following355
  • Followers2.2K
19Media
Making LLMs run efficiently can feel scary, but scaling isn’t magic, it’s math! We wanted to demystify the “systems view” of LLMs and wrote a little textbook called “How To Scale Your Model” which we’re releasing today. 1/n
25
385
77
1,912
468,316
Brennan Saeta retweeted
Andrej Karpathy
@karpathy20 Feb 2023
Late to the party but "GPT in 60 Lines of NumPy" / picoGPT is nicely done: jaykmody.com/blog/gpt-from-s… - good supporting links/pointers - flexes some of the benefits of JAX: 1) trivial to port numpy -> jax.numpy, 2) get gradients, 3) batch with jax.vmap - inferences gpt-2 checkpoints
15
134
5
1,348
222,583
Brennan Saeta retweeted
The Onion
@TheOnion11 Nov 2022
Crypto Confidence Soars After CEO Defrauds Customers Just Like Real Bank bit.ly/3UyxO7u
A closeup of Sam Bankman-Fried.

ALT A closeup of Sam Bankman-Fried.

391
6,097
702
40,124
Stable Diffusion JAX model for TPU has been released by Hugging Face 👀 Generate your own images with free access to TPUs via Colab and Kaggle ⬇️
Pedro Cuenca
@pcuenq14 Oct 2022
🤗 Diffusers 0.5.1 was released yesterday with JAX support. We've experimentally used Diffusers + JAX + TPU in our main demo Space for the past 2 weeks. Almost 50 million images were served in 14 days (more than 3 M per day) 🤯 huggingface.co/spaces/stabil… A couple of details 👇
Three example images generated by Stable Diffusion using the following prompts: "Labrador in the style of Vermeer", "Times Square under water, with fish and a dolphin swimming around", "Ancient Roman fresco showing a man working on his laptop".

ALT Three example images generated by Stable Diffusion using the following prompts: "Labrador in the style of Vermeer", "Times Square under water, with fish and a dolphin swimming around", "Ancient Roman fresco showing a man working on his laptop".

11
50
Wanted to share something I’ve made recently - a visualisation for ML number formats: thecharlieblake.co.uk/visual…… Hopefully this gives some intuition for the trade-offs involved (e.g. FP16 vs BF16). It also introduces the two newly proposed FP8 formats. Feedback much appreciated🙏
5
36
3
182
Brennan Saeta retweeted
Brad Larson@bradlarson14 Oct 2022
My team at PassiveLogic is hiring compiler engineers with LLVM familiarity to help us work on differentiable Swift and other projects: passivelogic.teamtailor.com/… . Fully remote, multiple experience levels, not limited to the US.
5
1
8
Brennan Saeta retweeted
PyTorch
@PyTorch13 Oct 2022
Cloud TPUs are powerful hardware for building large models & have enabled many research successes. We enable scaling up PyTorch models to 10B+ parameters on Cloud TPU with a new Fully Sharded Data Parallel (FSDP) interface in PyTorch/XLA.

1
15
1
74
Brennan Saeta retweeted
Alexandra R.@Alexa_MRo9 Oct 2022
France’s national rail company consulted on California’s high speed rail project and issued this absolutely damning indictment of CA politics: nytimes.com/2022/10/09/us/ca… – at Washington, DC
48
822
126
3,475
Brennan Saeta retweeted
Sundar Pichai
@sundarpichai6 Oct 2022
Two important breakthroughs from @GoogleAI this week - Imagen Video, a new text-conditioned video diffusion model that generates 1280x768 24fps HD video. And Phenaki, a model which generates long coherent videos for a sequence of text prompts. imagen.research.google/video…
53
272
43
2,195
Using both natural and artificial abilities, the human relationship with tools has drastically evolved. The best tools are powerful because they’re easy to use. This is where our skill of language and AI meet. Learn more on how conversation can power AI > blog.salesforceairesearch.co…
An arm holding a rock, representing the early stages of the human relationship with and development of tools over time.

ALT An arm holding a rock, representing the early stages of the human relationship with and development of tools over time.

14
3
53
Are you interested in learning JAX with Flax? We have translated our popular Deep Learning tutorials on CNNs, GNNs, (Vision) Transformers, and more from PyTorch to JAX+Flax, with considerable speedups for smaller models! Check them out here: uvadlc-notebooks.readthedocs… 🧵 1/12
8
205
9
991
Large language models can exhibit falsehoods, discriminatory language, and other unsafe behaviour. Introducing Sparrow: a dialogue agent that can search the internet and is trained to be more helpful, correct, and harmless using RL from human feedback: dpmd.ai/sparrow 1/
13
187
46
816
Brennan Saeta retweeted
Jeff Dean
@JeffDean23 Sep 2022
Infrastructure that underlies great research often doesn't get the attention that it deserves. This post discusses TensorStore, a (now) open-sourced high-performance, scalable array storage system. It enables @GoogleAI's connectomics work, LLMs, + more... ai.googleblog.com/2022/09/te…
6
82
4
454
What a goldmine by @_arohan_ et al. I finally finished my first reading of this paper. Although I don't work in RecSys but this paper discusses tons of problems and their solutions to keep in mind when deploying models. Moving on to a more in-depth reading with the references.
AK
@_akhaliq13 Sep 2022
On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models abs: arxiv.org/abs/2209.05310
18
1
89
Announcing a new library for #JAX! ⭐jaxtyping⭐ Type annotations (+runtime checking!✅) for the shape/dtype of JAX arrays, with a beautiful concise syntax: f32["foo bar"] means dtype float32 and shape (foo, bar). Also annotate PyTrees! GitHub: github.com/google/jaxtyping 1/2
6
41
3
334
Brennan Saeta retweeted
Jack Clark
@jackclarkSF1 Jul 2022
As someone who has spent easily half a decade staring at AI arXiv each week and trying to articulate rate of progress, I still don't think people understand how rapidly the field is advancing. Benchmarks are becoming saturated at ever increasing rates.
49
352
60
2,341
Prototyping @cleanrl_lib's DDPG with #jax, #flax, and #optax - I am seeing a 4x performance improvement 🤯🤯🤯🤯🤯 So yeah, JAX is coming to @cleanrl_lib 😃😃
6
22
152
Joining Google Developer Experts as well 🎉 Hope to help expand the JAX/Flax ecosystem.
Boris Dayma 🖍️
@borisdayma18 Jun 2022
I'm now part of the Machine Learning Google Developer Experts Program 🎉
3
3
1
51
Brennan Saeta retweeted
rohan anil
@_arohan_22 Jun 2022
A new image generation model just dropped. parti.research.google/ Great work by the team! + Auto-regressive, encoder->decoder Transformer + Classifier-free sampling. + ViT-VQGAN Really amazing results: Image from the website.
14
97
23
452
Load more