MF FOOM retweeted
Contrast pairs are overpowered. Once you have them, you can use them to generate control vectors, and to initialize classifiers, and to do RL/DPO, and probably more
Replying to @AnthropicAI
To make the probes, we track how the model’s internal state changes between “Yes” vs “No” answers to questions like "Are you doing something dangerous?"
We use this info to detect when a sleeper agent is about to misbehave (e.g. insert a code vulnerability). It works quite well:
MF FOOM retweeted
‼️⚠️bad day to be a LLM⚠️‼️
@haizelabs took one of our favorite adversarial attack algorithms, GCG, and made it *38x* faster
MF FOOM retweeted
When a student sadly tells me that the idea we've been working on for weeks was just arXived, I say:
"Great! We've just gotten *strong* confirmation that our thinking was in the right direction. We've had the initial work done for us. Lets figure out how to make this 10x better"
MF FOOM retweeted
new blog post! played around w/ representation engineering, and released a new library for training control vectors in <60s! i train some useful control vectors, some… out there… control vectors, test them for jailbreaking, test them *against* jailbreaking, and more!
MF FOOM retweeted
New blog post: how to make LLMs go fast! Want to understand how people are making LLMs go brrrrr? This post is a survey of lots of different LLM inference optimizations, ranging from "everyone uses this in prod" to "I cooked this up last week (but it seems to work)"
MF FOOM retweeted
Talk: Inverting Language Models (by @jxmnop)
youtube.com/watch?v=lguThumF…
Techniques for extracting text from vector databases and prompts from LLM APIs.
MF FOOM retweeted
Just rewrote the forward function of self attention in nanoGPT (@karpathy) with einops and 🤌:
For anyone working with higher dim tensors: einops.rocks/1-einops-basics…
MF FOOM retweeted
PagedAttention, Virtual Context, Speculative Decoding, Register Tokens: the last year has seen many ideas from systems programming applied to LLMs.
Not many folks live in that intersection, so I wrote an explainer post to make them a bit more accessible!
charlesfrye.github.io/progra…
MF FOOM retweeted
the value of embeddings is not search, but search for “something like this” without having to define what “this” is
MF FOOM retweeted
(1/n) Although the original model definition is not released. It's possible to reverse-engineer the model definition from torch.jit object using
`model.forward.code` to get somewhat readable code.
And `model.named_parameters()` to get even more details.
if you get the chance, do try decoding your Stable Diffusion generations with this decoder. You should see some improvements in text, small faces and straight lines. Made with @_tim_brooks , @DrYangSong , @model_mechanic , @txhf , @neonbjb
github.com/openai/consistenc…
MF FOOM retweeted
So much gold in this thread
Replying to @borisdayma @ldjconfirmed
Use flash attention 2, parallel attention and feedforward layers, rotary embeddings, pre-layer norm, and probably 8/3 h multipliers but that doesn't matter too much. Basically Mistral + parallel layers (they left a free +10% performance on the table).
MF FOOM retweeted
Replying to @MF_FOOM
This paper did a nice exploration on it arxiv.org/pdf/2212.10562.pdf
- character level tokenizer greatly improves text rendering
- but comes at expense of general text understanding
- alternatively a very large text model with standard tokenizer may be competitive – at Gainesville, FL
MF FOOM retweeted
finally published our latest research on text embeddings!
TLDR: Vector databases are NOT safe. 😳 Text embeddings can be inverted. We can do this exactly for sentence-length inputs and get very close with paragraphs...
MF FOOM retweeted
This looks legit. Attention heads tend to use the beginning of sequence for "null attention", so maintaining those tokens at the start of the KV cache allows for better sliding-window generation of long text. Can also be combined with long context tricks.
arxiv.org/abs/2309.17453
MF FOOM retweeted
Replying to @davidad
huh... this is interesting!
I was curious, so I trained the smallest possible GPT to win at tic-tac-toe
made it work with 2548 parameters!
I think it could go smaller still
philliphaeusler.com/posts/ti…
MF FOOM retweeted
μP allows you to keep the same hyperparameters as you scale up your transformer model. No more hyperparameter tuning at large size! 🪄
It saves millions of $ for very large models.
It’s easier to implement than it seems:
You have to
1. Keep the initialization and learning rate on your embeddings fixed as you increase the width of the model. Do the same for all biases in the model.
2. For all linear layers, multiply the learning rate with 1/width and the initialization scale with 1/sqrt(width).
3. Scale the attention logits with 1/head_dim instead of the usual 1/sqrt(head_dim). Tip: just leave the head_dim fixed and increase the number of heads. Then you can ignore this step.
4. Multiply the final output logits of the transformer with 1/width. This is equivalent to changing the temperature of the softmax.
5. (Optional) Initialize the final output layer with zeroes for slightly better results.
The above only works for Adam and can’t be used with plain SGD. These steps are equivalent to table 8 in our paper: arxiv.org/abs/2203.03466
The instructions work with both shared and distinct input/output embeddings.
An additional trick that is useful is to start with a base model that is trained without μP and then to scale up by using the ratio “new_width/old_width” instead of “width” in the rules above. That way you can start with any existing base model.
A nice way to verify it is to perform multiple training runs keeping everything fixed but increasing the width to be very large. The losses of the wider models must be equal or less to any smaller models at all points in time. This “wider is always better” effect is shown in figure 8 of the paper.
MF FOOM retweeted
Replying to @voooooogel @MF_FOOM
this technique achieves SOTA in the "minimum time to destroy your OpenAI budget" benchmark
MF FOOM retweeted
Showerthought: can you mix speculative decoding and contrastive decoding? Basically leverage the cases when the draft model and main models disagree in a contrastive way. I mean, you have both sets of logits so there shouldn't be much additional computation
MF FOOM retweeted
The open source community has done a great job improving run time speed and fine-tuning performance of models. However, we don't talk enough about techniques for training bigger models. One of these is hyperparameter transfer via μP.
The problem is that optimal hyperparamaters such as learning rate, when tuned on smaller models, don't transfer to larger models.
This means that since tuning often takes hundreds of individual runs, it can be extremely expensive to find the right HPs for large models
For us GPU poor who can't train two dozen 7B models, we need something better. It turns out in "Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer" which was joint work by msft and openai you can use clever parameterization to do scale transfer!
Essentially, models parameterized via μP (e.g., scaled properly) learn features in the infinite-width limit. This means their training dynamics are stable and predictable.
AND are equally stable and predictable at all scales.
So when you scale your small model and tune, the hyperparameters can be used on a large model of the same family
The downside is the parameterization needs to be derived for each layer type.
However, that work has been done, so we can `pip install mup` and go!
It can't be used for every hyperparameter, so ones relating to regularization like dropout p don't transfer. But initialization weights and learning rate do!









































