@alemi

Machine Learning Researcher

 Kissimmee, FL
alexalemi.com
Joined January 2008
  • Tweets67
  • Following1.5K
  • Followers1.2K
4Media
Alex Alemi retweeted
Anthropic
@AnthropicAIFeb 28
A statement on the comments from Secretary of War Pete Hegseth. anthropic.com/news/statement…

Statement on the comments from Secretary of War Pete Hegseth

Anthropic's response to the Secretary of War and advice for customers

anthropic.com
2,719
6,338
2,143
41,641
17,948,705
I am recruiting Ph.D. students for my new lab at @nyuniversity! Please apply, if you want to work with me on reasoning, reinforcement learning, understanding generalization and AI for science. Details on my website: izmailovpavel.github.io. Please spread the word!
16
101
5
742
115,818
Recently I've been playing around with a quarter-order-of-magnitude system for simple calculations. It gives better precision than single sig-fig calculations using only four, very intuitive, symbols. blog.alexalemi.com/quarters.…
A logarithmic dial separated into quarters. In order they are labelled 1, tau, pi and a three-legged pi.

ALT A logarithmic dial separated into quarters. In order they are labelled 1, tau, pi and a three-legged pi.

9
880
If you miss the NYTimes needle, especially one that is statistically uniform (blog.alexalemi.com/a-degree-…), you can use this page: alexalemi.com/random/electio… I whipped together to reason about the correlations between the swing states tonight as results come in.
1
18
2,843
Why don't we measure probabilities in degrees? blog.alexalemi.com/a-degree-…
4
11
1
56
6,511
In which I try to make sense of most of machine learning: blog.alexalemi.com/kl-is-all…

KL is All You Need

It's all KL under the hood.

blog.alexalemi.com
5
39
8
289
62,369
Is Kevin onto something? We found that LLMs can struggle to understand compressed text, unless you do some specific tricks. Check out arxiv.org/abs/2404.03626 and help @hoonkp, @alemi, Jeffrey Pennington, @ada_rob, @jaschasd, @noahconst and I make Kevin’s dream a reality.
6
16
2,761
Ever wonder why we don’t train LLMs over highly compressed text? Turns out it’s hard to make it work. Check out our paper for some progress that we’re hoping others can build on. arxiv.org/abs/2404.03626 With @blester125, @hoonkp, @alemi, Jeffrey Pennington, @ada_rob, @jaschasd

Training LLMs over Neurally Compressed Text

In this paper, we explore the idea of training large language models (LLMs) over highly compressed text. While standard subword tokenizers compress text by a small factor, neural text compressors...

arxiv.org
2
12
3
81
31,928
Each delivery service should use its own distinctive knock.
1
3
856
PaLM 540 Billion, Google's large language model used 4.2 moles of flops to train. 4.2 Moles!
9
1,464
Alex Alemi retweeted
Ben Poole
@poolio29 Sep 2022
Happy to announce DreamFusion, our new method for Text-to-3D! dreamfusion3d.github.io We optimize a NeRF from scratch using a pretrained text-to-image diffusion model. No 3D data needed! Joint work w/ the incredible team of @BenMildenhall @ajayj_ @jon_barron #dreamfusion
126
1,380
411
5,431
Alex Alemi retweeted
Alex Alemi@alemi16 Sep 2022
Replying to @dpkingma @poolio
To accompany the colab, I've also written a blog post blog.alexalemi.com/diffusion… attempting to make sense of the VDM Diffusion loss. In it, I try to motivate how the VDM diffusion loss is simply the joint KL between the forward and reverse process.

A Path to the Variational Diffusion Loss

Deriving the (Variational) Diffusion and VAE losses from the non-negativity of KL.

blog.alexalemi.com
2
11
1
51
Alex Alemi retweeted
Durk Kingma@dpkingma16 Sep 2022
Want to understand and/or play with variational diffusion models? - See colab.research.google.com/gi… for a simple stand-alone implementation and explanation. (Thanks @alemi and @poolio for making this)! - See colab.research.google.com/gi… for an even more basic implementation on 2D data.

SimpleDiffusionColab.ipynb

Run, share, and edit Python notebooks

colab.research.google.com
1
63
2
324
A pretty cool paper (and I also hope useful) on using pre-training models to create highly informative priors for downstream tasks. Thanks to all the collaborators, it was a lot of fun!
Andrew Gordon Wilson
@andrewgwils23 May 2022
Pre-Train Your Loss: Easy Bayesian Transfer Learning with Informative Priors. arxiv.org/abs/2205.10279 w/@ziv_ravid, @micahgoldblum, @HosseinSouri8, @snymkpr, @Eiri1114, @ylecun 1/6
2
11
1
78
Alex Alemi retweeted
Ethan Dyer@ethansdyer1 Jul 2022
1/ Super excited to introduce #Minerva 🦉(goo.gle/3yGpTN7). Minerva was trained on math and science found on the web and can solve many multi-step quantitative reasoning problems.
aitor@alewkowycz30 Jun 2022
Very excited to present Minerva🦉: a language model capable of solving mathematical questions using step-by-step natural language reasoning. Combining scale, data and others dramatically improves performance on the STEM benchmarks MATH and MMLU-STEM. goo.gle/3yGpTN7
27
504
70
2,538
We are thrilled to announce Imagen, a text-to-image model with unprecedented photorealism and deep language understanding. Explore imagen.research.google and Imagen! A large rusted ship stuck in a frozen lake. Snowy mountains and beautiful sunset in the background. #imagen
55
293
74
1,598
here are the next few days wordle answers as md5 hashes 2022-01-11 = 0b18a3d7b9c43ff1750d2baa4606b8d0 2022-01-12 = 047fb90408a79f189d51cbcea168b1a5 2022-01-13 = ab3358313efb03210a1babfb372246f1 2022-01-14 = d821e448212defd91ac1e67f9653a34d
3
2
you can verify with `echo -n "answer" | md5sum`
We are presenting our paper "Does Knowledge Distillation Really Work?" at #NeurIPS2021 poster session 2 today - come check it out! Joint work with @Pavel_Izmailov, @polkirichenko, @alemi, and @andrewgwils. Poster: nips.cc/virtual/2021/poster/… Paper: arxiv.org/abs/2106.05945
2
12
3
77
Excited to kick-start focus #SciML series on #ML meets Info theory and statistical mechanics! Amazing speaker/session chair line-up: @alemi (@wellingmax), @pratikac (Karthik), @ShoYaida (@jaschasd), @yasamanbb (@SuryaGanguli) and Elena Agliari. Details at: cmu.edu/aced/sciML.html
4
33
3
195
Load more