Latest Webb telescope image shows the grand-design spiral galaxy

it’s been a dramatic couple of days, I didn’t expect the already clean essay to be read so widely. I've heard more interesting discussions about mechanistic interpretability – should we think mostly in features or mechanisms and the flow of data through them, how much do weak activations matter, how does scale change things, and lots of other questions – than I have in a long time and I’ve been happy to hear all the discussion! a couple things on the essay itself. it didn't have enough citations, only ones for the basic story, since it was an "informal research essay and working hypothesis" rather than a paper, and other prior work around neurons were in the technical paper we're still finishing up. but it blew up, so I've added citations to the essay as well. I also want to thank @curttigges who did a lot of the language neuron research with me finally, while there was quite a range of updates, from this being incremental and obvious to interpretability going from hopeless to fully solved, I want to try to be clear about roughly where I stand I used to think neural networks would take some big cleaning step and then we could begin the ordinary natural science of mapping the parts and how they connect, watching data flow through, mapping circuits, and so on. at various points, like late layers of early model organisms and most layers of language models years ago, this didn't really work. but now with better tools and better models of study, it works much better for smaller vision models, this normal science process worked surprisingly well, until the whole map started to be broadly filled in. For bigger modern language models, it seems promising so far, for both neurons and circuits, attribution graphs, basically everywhere we've glanced. the map isn't filling in, there's a lot of map. there's still off-target firing, neurons that do a few things, but so far it's not an incoherent mess that you can't explore and learn from, and over time we get better at learning how to see it. so maybe it keeps going, and mapping motifs and exploring actually gets you pretty far, maybe something like "all the way" to a reasonable solution. I don't think it would be so scientifically surprising if that were the case. everything in the neural network was built in some sense to be read – by its downstream readers – and if we can learn to look like a reader, we might find reading them relatively easy. many researchers I talk to outside interpretability assume the parts are already understandable, and the difficulty is just in the mapping stage! it was also my prior when I first started looking at neurons when I first joined the field. there's lots of highly interpretable ones, maybe that's just how neural networks are if you look right. even in the good world it would still take some iteration on tools, on ways of looking, neuron-attention interactions are tricky for instance. there's probably no bright red line separating "cleaning the model" from "learning to look better" but there's broad working principles, and as of now I think we should try to look relatively directly if we can it shouldn't be so hard to test, you can just start looking at things and iterating and trying to understand motifs and such and see if it just breaks down at some point. it can take some iteration, I would recommend a bias against assuming arbitrary mixtures when you don't understand something at first, but also explanations should be tested for now I'm pretty optimistic, and I'm glad there's excitement about interpretability, but too much can cause problems too. I think part might be that ai safety isn't a very hopeful field, and even moderate amounts of optimism about an important problem being solvable in the future fills a spot that has been malnourished for a while. across several safety disciplines, the researchers directly working on them are often more optimistic than the people broadcasting pessimism more broadly for neural network interpretability at least, these are extraordinarily experimentally accessible objects of science, I don't know of a better one, and we have automated scientists. we have many things going in our favor. it's important and it's fun. the circuitry of these machines can be beautiful, surprising, entertaining, and insightful. I've spent thousands of hours looking through them, asking questions and running experiments and testing things, and would do so even if it weren't so important. I think the problem is solvable, whether the simple thing works, it mostly works but we need some new things, or we need big new ideas. I think we can figure it out

genuinely an amazing day

The AI exchange is live. Trade the takeoff: mnx.fi

From a friend: “Waiting until the models get good enough to one-shot it is apparently now a viable strategy for technical debt”

IT'S HERE!!! HALO CE (XBOX) IN THE BROWSER! Features: ✅ WORKING Splitscreen Co-op ✅ WORKING Multiplayer up to 128 players (untested) ❌ NO GEARBOX MASTER CHIEF

Yes

batch four... soon

I think ambient intents are going to be a big deal. There are so many intentions we have that would make our lives better, but the cost of surfacing them to a market it too high, so they never become legible to the world. You want a better job, you want to swap your couch, you would apartment-swap with someone in your web-of-trust, you would upgrade from a two-bedroom to a three-bedroom if there were some graceful way to find the person who wants to size down, and you would love to sublet you place in New York without posting on Instagram and making 95% of you friends read a logistical errand that has nothing to do with them. Right now, the cost of expressing these intents is high. You have to remember the want, decide it is worth acting on, find the right channel, phrase it socially, tolerate the inbound, filter for trust, negotiate details, and then keep the whole thing alive in your head. So most of the long tail dies. Agents change this because they can keep the low-grade, half-formed wants running in the background. They know your calendar, your travel plans, your music, your reading, your friends, your constraints, and maybe your willingness to be interrupted. You listen to a band on repeat on Spotify and your agent notices they are playing 20 minutes from where you will be in California next month. You highlight a book you love in Readwise and it tells you that your friend is reading it too, and you will both be at the same dinner next week. You mention wanting Berlin in June and it quietly checks whether any trusted people from there want to apartment swap in New York then. The magic is lowering the cost of noticing, holding, matching, and negotiating these things. It will feel like a higher level of serendipity. This will require a web-of-trust that has yet to be built because there is an important privacy aspect to this. The dystopian version is "AI companies capture your intentions and auction them to whoever wants to manipulate you." The useful version is user-owned intents, where your agent can prove enough to match or negotiate without dumping your private life into a marketplace. Some of this already has been solved in cryptography: private set intersection for finding overlaps without revealing all non-matches, secure multiparty computation / homomorphic encryption for computing matches or scores over private inputs, zero-knowledge credentials for proving things like membership, attendance, reputation, or trust path without exposing everything underneath. If this works, a lot of modern life gets more liquid. Idea sharing, couches, apartments, reading groups, dinner plans, travel overlaps, introductions, tiny labor exchanges, borrowing a camera, finding the one person at an event who cares about the same weird thing. All the stuff that currently relies on posting into the void and hoping the right person happens to see it. The hard parts are real: consent, spam, weird incentives, agent loyalty, social context, and making sure this becomes a tool for people rather than a new ad exchange with better vibes. But I increasingly think the big unlock is giving our unexpressed intentions a safe place to live, and giving our agents permission to help them find each other. I know of @indexnetwork_ working on this. Anyone else?