Fat Rust Developer retweeted
Our ability to do things is dying a death of 1000 cuts.
It's nice to talk about bottlenecks and silver bullets, but my hypothesis is that stagnation is the accumulation of many small frictions.
My plan is to continuously update this thread with examples as they pop up.
Ok sure it's cool to see @karpathy 's llama2.c but what about llama2 in holy-c
Fat Rust Developer retweeted
Please remind me how we are moving forward. In this video, a machine from the year ~2000 (600MHz, 128MB RAM, spinning-rust hard disk) running Windows NT 3.51. Note how incredibly snappy opening apps is. 👇
Fat Rust Developer retweeted
GPT-4 will have the same number of parameters as GPT-3, but these parameters will be kiki instead of bouba
Fat Rust Developer retweeted
More generally, the narrative that we're in decline (in tech or otherwise) because of "wokeness" (a term used exclusively by a certain kind of people to designate everything they don't like about social progress, the way they might have used "hippies" in the 1960s) is nonsense.
Fat Rust Developer retweeted
LeCun's "5 best ideas 2012-22” are mostly from my lab, and older: 1 Self-supervised 1991 RNN stack; 2 ResNet = open-gated 2015 Highway Net; 3&4 Key/Value-based fast weights 1991; 5 Transformers with linearized self-attention 1991. (Also GAN 1990.) Details: people.idsia.ch/~juergen/lec…
