software engineer and aspiring knower of things! words @TheAtlantic @reboot_hq @kernel_magazine @logic_magazine

San Francisco, CA
Joined November 2017
My lexical call-to-arms is now out in The Atlantic: Let’s abandon the term “lab” as a term for the AI industry’s most powerful corporations. We should reserve the word for organizations whose primary activity is scientific research. Gift link: theatlantic.com/technology/2…
13
65
21
373
67,605
You Can See Everything provides a sobering reminder of the unrelenting societal chaos that arises from misunderstanding the forall/exists duality
3
19
1,190
the key to interpretability is a Singaporean Chinese neuron
4
133
the interpretability experts can duke it out on the merits of the claims, but as a work of scientific communication, what a beautiful artifact & a testament to changing your mind after years of dedication to a particular area of study
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
10
270
nothing could’ve prepared me for the experience of watching this movie. marketed as a comedy but in actuality one of the most gruesome action movies I’ve watched with some of the most beautifully filmed Chinese cooking scenes I’ve ever seen in my life. watch and be transformed
6
307
Matthew Sun retweeted
it's incredible that you can be late to a meeting taking place inside a computer you're already staring at
10
68
4
1,965
23,628
there’s an odd cognitive dissonance when elites in the AI industry insist that they want to “save democracy” and “beat authoritarianism” while repeatedly ignoring the concerns of everyday people in the name of pursuing profit & winning a purported race
(somewhat belated) blog version of this paper is out, thank you to @ahall_research for letting me crash the @freesystemslab newsletter!!! freesystems.substack.com/p/b…
3
4
27
687
from the viewpoint of someone whose town official signed an NDA to build a data center, or an artist who can no longer find work, or a SWE affected by an AI-related round of layoffs, none of this feels democratic or fair. even if you used “constitutional AI” for model training
1
2
94
read the latest NYT on the American microdrama industry and have learned about such cursed terms as “the fat-wife counterattack subgenre”
1
4
156
lmao
how the American taxpayer helps back Meta's A.I. data center expansion by @kashhill @JesseDrucker @elitanjourno and me nytimes.com/2026/09/30/techn…
2
5
382
also learned about this concrete incentive for tech companies to portray their commercial activity as “research” for tax purposes!
3
61
very glad to see someone articulate this argument in an eminently reasonable way! discourse about AI is definitely more ~ exciting ~ when it traffics in sci-fi tropes, but that language tends to cloud our thinking, rather than clarify it.
In @NewYorker today, I write about the "But China" problem—evoking China as the reason to avoid pausing or slowing down AI development and regulation. The argument, rooted in Western philosophy and its thought experiments, takes a sensible premise like self-improving AI and slips into sci-fi absurdity—killer robots, ethnically targeted bioweapons, supersoldiers—while ignoring basic realities of diffusion: how tech burrows into societies and organizations. As @jjding99 shows, the AI “race” is a marathon, not a sprint. Even god, if it is to mean something in this world, must come down to earth and grapple with all its messiness. newyorker.com/news/the-lede/…
1
11
592
unfortunately some things that are true (we need bureaucracy, oversight, better application of security principles that have existed for a long time) just aren’t that sexy! none of this underplays the risks from agents running rampant, just points to rather mundane solutions
36
Matthew Sun retweeted
Check out our RSI data transparency tracker! Current scores: OAI (2/8), Ant (1.5/8), GDM (0.5/8) Hoping the labs will hill-climb this as fast as they have every other benchmark :) This is based on our recent econ of RSI paper @ElasticityInst 1/2
[1/5] The 8 most valuable data points labs should share to help measure RSI: First, RSI would likely accelerate growth in AI capabilities. Thus, companies should report performance on diverse benchmarks for the latest internally deployed models.
6
24
1
135
15,131
enjoyed this thoughtful reflection about how AI has exacerbated the underlying structural contradictions in the American university system by @ibogost in The Atlantic. (the disentanglement of intellectual labor and social interaction is one that shows up in the workplace, too!)
1
4
211
Matthew Sun retweeted
great to see this — government mandated independent auditing / evals could be one of the highest impact policy goals of our generation. it’s absolutely untenable for voluntary disclosures to remain our only window into what’s happening in the labs.
I think government mandated independent oversight of the labs is a good idea with broad bipartisan public support and that failure to act quickly could have grave irreversible consequences
2
12
79
4,577
imo, public skepticism about AGI / extinction is not reducible to “dumb public refuses to believe genius scientists.” utopian *and* dystopian claims about social media / crypto / metaverse / etc have proved not credible. if the idiom of sci-fi is not persuasive, stop using it!
1
1
4
240
i just think that if you were actually invested in AI safety, you would do the work of trying to earn political popularity. why spend millions of $ on yet another safety org when you could try accepting that the alarmist language is turning people off & try another comms strategy
1
1
127
whether AI employees believe themselves to be sincere isn’t that interesting to me. but if you really wanted to be effective in a democracy, wouldn’t you want to speak in a way people actually understand? doesn’t the burden of proof lie on experts to communicate competently?
62