MF FOOM (@MF_FOOM) header
MF FOOM (@MF_FOOM) profile picture

masked attention head

Joined July 2023
Photo by MF FOOM (@MF_FOOM)Photo by MF FOOM (@MF_FOOM)Photo by MF FOOM (@MF_FOOM)Photo by MF FOOM (@MF_FOOM)Photo by MF FOOM (@MF_FOOM)Photo by MF FOOM (@MF_FOOM)Photo by MF FOOM (@MF_FOOM)Photo by MF FOOM (@MF_FOOM)Photo by MF FOOM (@MF_FOOM)Photo by MF FOOM (@MF_FOOM)
Contrast pairs are overpowered. Once you have them, you can use them to generate control vectors, and to initialize classifiers, and to do RL/DPO, and probably more
Replying to @AnthropicAI
To make the probes, we track how the model’s internal state changes between “Yes” vs “No” answers to questions like "Are you doing something dangerous?" We use this info to detect when a sleeper agent is about to misbehave (e.g. insert a code vulnerability). It works quite well:
2
5
81
11,944
‼️⚠️bad day to be a LLM⚠️‼️ @haizelabs took one of our favorite adversarial attack algorithms, GCG, and made it *38x* faster
2
14
3
65
12,357
When a student sadly tells me that the idea we've been working on for weeks was just arXived, I say: "Great! We've just gotten *strong* confirmation that our thinking was in the right direction. We've had the initial work done for us. Lets figure out how to make this 10x better"
6
12
1
229
19,259
new blog post! played around w/ representation engineering, and released a new library for training control vectors in <60s! i train some useful control vectors, some… out there… control vectors, test them for jailbreaking, test them *against* jailbreaking, and more!
39
32
21
442
170,853
New blog post: how to make LLMs go fast! Want to understand how people are making LLMs go brrrrr? This post is a survey of lots of different LLM inference optimizations, ranging from "everyone uses this in prod" to "I cooked this up last week (but it seems to work)"
12
149
11
1,001
176,928
Talk: Inverting Language Models (by @jxmnop) youtube.com/watch?v=lguThumF… Techniques for extracting text from vector databases and prompts from LLM APIs.
2
21
2
202
108,522
Just rewrote the forward function of self attention in nanoGPT (@karpathy) with einops and 🤌: For anyone working with higher dim tensors: einops.rocks/1-einops-basics…
2
5
1
109
13,507
PagedAttention, Virtual Context, Speculative Decoding, Register Tokens: the last year has seen many ideas from systems programming applied to LLMs. Not many folks live in that intersection, so I wrote an explainer post to make them a bit more accessible! charlesfrye.github.io/progra…
17
250
18
1,431
312,428
MF FOOM retweeted
the value of embeddings is not search, but search for “something like this” without having to define what “this” is
3
1
12
3,410
(1/n) Although the original model definition is not released. It's possible to reverse-engineer the model definition from torch.jit object using `model.forward.code` to get somewhat readable code. And `model.named_parameters()` to get even more details.
if you get the chance, do try decoding your Stable Diffusion generations with this decoder. You should see some improvements in text, small faces and straight lines. Made with @_tim_brooks , @DrYangSong , @model_mechanic , @txhf , @neonbjb github.com/openai/consistenc…
1
9
55
34,114
So much gold in this thread
Use flash attention 2, parallel attention and feedforward layers, rotary embeddings, pre-layer norm, and probably 8/3 h multipliers but that doesn't matter too much. Basically Mistral + parallel layers (they left a free +10% performance on the table).
2
6
1,018
Replying to @MF_FOOM
This paper did a nice exploration on it arxiv.org/pdf/2212.10562.pdf - character level tokenizer greatly improves text rendering - but comes at expense of general text understanding - alternatively a very large text model with standard tokenizer may be competitive – at Gainesville, FL
2
1
6
401
finally published our latest research on text embeddings! TLDR: Vector databases are NOT safe. 😳 Text embeddings can be inverted. We can do this exactly for sentence-length inputs and get very close with paragraphs...
42
170
54
1,269
391,557
This looks legit. Attention heads tend to use the beginning of sequence for "null attention", so maintaining those tokens at the start of the KV cache allows for better sliding-window generation of long text. Can also be combined with long context tricks. arxiv.org/abs/2309.17453
4
11
2
122
21,338
μP allows you to keep the same hyperparameters as you scale up your transformer model. No more hyperparameter tuning at large size! 🪄 It saves millions of $ for very large models. It’s easier to implement than it seems: You have to 1. Keep the initialization and learning rate on your embeddings fixed as you increase the width of the model. Do the same for all biases in the model. 2. For all linear layers, multiply the learning rate with 1/width and the initialization scale with 1/sqrt(width). 3. Scale the attention logits with 1/head_dim instead of the usual 1/sqrt(head_dim). Tip: just leave the head_dim fixed and increase the number of heads. Then you can ignore this step. 4. Multiply the final output logits of the transformer with 1/width. This is equivalent to changing the temperature of the softmax. 5. (Optional) Initialize the final output layer with zeroes for slightly better results. The above only works for Adam and can’t be used with plain SGD. These steps are equivalent to table 8 in our paper: arxiv.org/abs/2203.03466 The instructions work with both shared and distinct input/output embeddings. An additional trick that is useful is to start with a base model that is trained without μP and then to scale up by using the ratio “new_width/old_width” instead of “width” in the rules above. That way you can start with any existing base model. A nice way to verify it is to perform multiple training runs keeping everything fixed but increasing the width to be very large. The losses of the wider models must be equal or less to any smaller models at all points in time. This “wider is always better” effect is shown in figure 8 of the paper.
If you are doing LLM (>1B) training runs, you ought to do these 3 things: 1. Use SwiGLU 2. Use ALiBi 3. Use µP Why? Your training will be almost 3X faster! You can do 3 runs for the price of 1. You can go for much bigger model or train longer. There is no excuse. 1/3
10
63
2
405
109,990
Replying to @voooooogel @MF_FOOM
this technique achieves SOTA in the "minimum time to destroy your OpenAI budget" benchmark
1
16
402
Showerthought: can you mix speculative decoding and contrastive decoding? Basically leverage the cases when the draft model and main models disagree in a contrastive way. I mean, you have both sets of logits so there shouldn't be much additional computation
3
1
9
1,315
The open source community has done a great job improving run time speed and fine-tuning performance of models. However, we don't talk enough about techniques for training bigger models. One of these is hyperparameter transfer via μP. The problem is that optimal hyperparamaters such as learning rate, when tuned on smaller models, don't transfer to larger models. This means that since tuning often takes hundreds of individual runs, it can be extremely expensive to find the right HPs for large models For us GPU poor who can't train two dozen 7B models, we need something better. It turns out in "Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer" which was joint work by msft and openai you can use clever parameterization to do scale transfer! Essentially, models parameterized via μP (e.g., scaled properly) learn features in the infinite-width limit. This means their training dynamics are stable and predictable. AND are equally stable and predictable at all scales. So when you scale your small model and tune, the hyperparameters can be used on a large model of the same family The downside is the parameterization needs to be derived for each layer type. However, that work has been done, so we can `pip install mup` and go! It can't be used for every hyperparameter, so ones relating to regularization like dropout p don't transfer. But initialization weights and learning rate do!
7
43
2
416
144,082
How can we make LLMs more efficient? The dumbest thing about transformers is their static compute usage - given "The capital of France", both "is" and "Paris" consume one forward pass each! There are two approaches to allocating the compute dynamically. (1/)
3
9
2
77
13,763