Dan Swanson retweeted
If we want to continue progress in science, we must be prepared to acknowledge that the correct description of nature will eventually simply be too complicated for humans to understand. I do not think it is wise to make human understanding a requirement. AI is the way forward.
Dan Swanson retweeted
Inspired by Nick Cammarata’s interpretability essay “Neural Networks Are Already Clean”, I turned a couple of its ideas into speedups for pretraining without backpropagation. Still in the works. 2.3x speedup, cool xitter.cf/industriaalist/status/…
Replacing backprop with zeroth-order optimization is inevitable.
Introducing Noise: a speedrun for pretraining without backprop.
Last week we presented Dust, the first method to approach backprop with SGD for pretraining transformers. We think another 1,000x improvement on top of it is possible, and it will require *co-designing* credit assignment, optimizers, architectures, and eventually, hardware.
We built Noise to iterate on this faster.
Dan Swanson retweeted
it’s been an enormous privilege and joy to watch @nickcammarata cook this up over the last year. week after week, experiment after experiment, he unraveled the tangled mess of our thinking about neural nets to reveal the truth.
i never knew quite what to expect on our weekly walks. each conversation brought a torrent of new theories about the nature of AI, some conjectures teetering on the brink of madness, and others that seemed to cut through years of confusion with a single sentence.
nick is one of the great scientists of our time. his intuition and research taste are out of this world, but what sets him apart is his unmatched ability to capture the essence of what something truly is, without being lost in the details. i would highly recommend you read his entire essay if you’re curious about AI. it’s long, but written beautifully.
nick’s research makes me far more optimistic about the future of AI alignment. i hope that it can inspire the same for you.
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Dan Swanson retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Dan Swanson retweeted
Hoy lanzamos un gran update de municipiosargentinos.net. Ya mide a los 2.280 municipios y comunas de todo el país.
Abajo lo nuevo 🧵👇
Dan Swanson retweeted
Anthropic says it has reduced the data harvested from Claude in distillation attacks by around 90–99%. I congratulate them on this achievement.
It’s consistent with the point I made a few months ago (All-In Ep. 282): Anthropic, not the government, was in the best position to stop distillation of its own models. Once the horse is out of the barn, there’s limited value in after-the-fact government action.
At the time there was a major push in Washington to restrict open models on IP grounds. The threat looked serious enough that 50 companies — including Nvidia, Microsoft, Meta, IBM, Palantir, and Dell — published an open letter in defense of open-weight models. Jensen Huang promoted it in his first-ever X post. Nearly 200 startups sent a similar letter.
I argued that restricting open models risked removing the incentive for closed-model companies to solve the problem themselves — and could reward negligence by stifling competition while they continued collecting revenue from distillation. The administration did not impose those restrictions.
Now we see the result. Anthropic invested in defenses and they worked. Not every technical challenge is a reason to run to Washington for new regulation.
Dan Swanson retweeted
Aprovecho que en este momento los inmigrantes musulmanes están prendiendo fuego Francia y volviendo a secuestrar aviones para estrellarlos y matar un montón de gente en el nombre de su religión, para contarles que hace unos días el Centro Islámico de la República Argentina me denunció PENALMENTE por publicar esta imagen👇
Quieren a como dé lugar, censurar la batalla cultural de Occidente en contra del Islam para dejarnos desarmados y que nos termine pasando lo mismo que a Europa. Porque así hicieron allá: llamaron "islamofóbico" a todo aquel que los criticara y les tiraron los sistemas judiciales de nuestras democracias encima, que en muchos casos terminaron arrodillándose y convirtiéndose en cómplices de la destrucción de sus países. Ahora estos muchachos están persiguiendo ciudadanos argentinos para meterlos PRESOS literalmente por criticarlos. Ni amenazas, ni hostigamientos, ni violencia: crítica pura y dura. Eso sí, de los genocidios cristianos, de los atentados por todo el mundo, de las violaciones en Europa, de las imposiciones religiosas por la fuerza, de los casamientos con nenas, de los asesinatos de homosexuales, de las lapidaciones por adulterio, del maltrato a las mujeres y todas las BARBARIDADES cometidas en nombre de la religión de ELLOS, NUNCA DICEN NADA. Bueno acá va mi respuesta a la denuncia: COMANME BIEN LOS HUEVOS. Nunca voy a pedir perdón por decir LA VERDAD y estoy dispuesto a ir PRESO y a todo lo que haga falta para defender a la Argentina de la amenaza que representa hoy el islam para el mundo y para la humanidad entera. Y dejen de querer apretarme con sectores del gobierno que comulgan con ustedes: ME CHUPAN BIEN LA PIJA TANTO ELLOS, COMO USTEDES.
Como dijo el presidente Donald Trump ayer sobre Francia y los ataques a las escuelas: "esto no se trata de las escuelas; esto se trata de que el islam quiere apoderarse de un país que una vez fue grande". Bueno, con Argentina no podrán.
ARGENTINA NO SE PUEDE CENSURAR
ARGENTINA ES LIBRE
ARGENTINA ES FARO DEL MUNDO
ARGENTINA NO SERÁ ISLAMIZADA
Dan Swanson retweeted
"mathematicians did not ask for this work to be done"
we're going to see a lot more of this in a lot of different fields
if these people genuinely loved math, they would be throwing parties to celebrate what might be one of the biggest days for mathematics in decades. but instead they're calling for a boycott
and it makes me wonder how many of them actually love math, and how many just love being seen as mathematicians.
because for a lot of people, the point of studying math isn't necessarily to understand the world, to explain it, or to advance the field. it's to be seen as a mathematician. to have a sense of importance, to feel valued by others. their entire identity is centered around this, and their goal is to be needed and considered smart
i think a lot of people are about to realize that what gave their life meaning wasn't really the work itself, but the fact that they were the ones doing it.
and i wonder what will happen when this happens to all professions. will we finally be forced to figure out what it means to be human?
maybe people will rediscover meaning in family, relationships and things outside of their careers.
or maybe millions of people will have an existential crisis because the one thing they've built their entire identity around is suddenly no longer needed
i genuinely wonder if society is ready for it
This quoted post is unavailable.
Dan Swanson retweeted
absolutely pathetic
"Terence Tao reposts a statement from the Association for Human Mathematics urging mathematicians to stop working with OpenAI"
i keep saying tao has a massive ego (and he's earned it) but he wants a small group to control the entire field. thankfully that's not going to happen anymore
Mathematicians are coping so hard they might abolish irrational numbers again and turn the field into a guild
terrytao.wordpress.com/2026/…
Dan Swanson retweeted
🔥🔥🔥No pasarán, fuera de Argentina y toda America. El Islam es incompatible con occidente.
Dan Swanson retweeted
TOCÓ LIBERAR A ESPAÑA
🇦🇷🇪🇸‼️ | LO ÚLTIMO — Según reportes de TN, el Presidente Javier Milei confirmó que viajará a Madrid el 19 de octubre para respaldar a la derecha frente al mandatario socialista Pedro Sánchez antes de las elecciones del 29N. En su séptima visita a España, Milei busca extender su influencia global e impulsar a sus aliados de Vox y el PP en la contienda, tras consolidar victorias electorales aliadas en la región.
