a comment from 3blue1brown's most recent video

If we can interpret frontier LLMs, the entire AI industry as it currently exists would be upended - Death of RL - RSI becomes near-trivial - Labs get enormous incentives to interpret their models (*especially* OpenAI and Anthropic) (obviously highly speculative but hear me out) Death of RL If we can interpret what individual neurons do, using RL as the primary way to improve models becomes as silly as using RL to fix bugs in a codebase. The far more efficient alternative is to *directly* observe neuron behavior and *directly* modify it. Much like directly editing your code to fix a bug once you understand it, rather than randomly trying edits and seeing what changes. RL would likely still be part of the training pipeline to bootstrap capabilities, but would become more like pretraining today: largely figured out and not a point of differentiation. RL will not be where the majority of capabilities come from. This is good: training models becomes OOM more efficient. This both makes frontier models far more powerful and lowers the threshold for new market entrants. Easy RSI If humans are able to interpret LLMs, then so can the LLMs themselves, in which case the recipe for RSI (recursive self improvement) becomes obvious. RSI is hard today largely because without interpretability, models have to be capable of doing novel AI research, which is not yet within reach. Models today suck at idea generation and innovation on the frontier. But if we have interpretable neurons, we can simply let the model observe its own neurons, modify their behavior directly, and improve outcomes. This is far easier because models won't need to come up with innovations in AI training methods, they'd simply modify weights and observe outcomes. "Vibe training", if you will. The bitter lesson wins again. More compute obliterates the need for AI research itself. Incentivized to interpret Labs today have nearly zero immediate economic incentive to interpret their models. All work, no reward (beyond good vibes). If interpretation becomes essential to the training pipeline itself (as I laid out above), however, they will have enormous incentives to interpret their models as much as possible, because it would be essential to improving capabilities via RSI. This is *especially* true of the frontier labs like OpenAI and Anthropic, who need to fight to stay ahead of each other. Not interpreting their models would be an existential mistake for them. What does this mean? - OOM faster improvement in AI capabilities - Data becomes nearly worthless - Evals become even more valuable - Frontier models get so interpretable you're tired of it - AI safety gets a *lot* easier, basically just grunt work - AI researchers get new careers as eval engineers - Goodfire gets trillion dollar valuation - Infinite zoomer abundance communism etc etc I for one look forward to that future (and seek exposure to Goodfire)

Rip it doesn’t work anymore

Testing mini apps in the timeline 👀 tap the embed and sign my guest book guestbook.stupidtech.net/

Officially, every viral AI influencer was made on Higgsfield. Make one with your face or from scratch. Then jump on viral trends with Genjutsu. Try up to 5 generations free. Available now in Higgsfield and via ChatGPT Extension. Credits: @JeanPhilMadame