Psychometrics, systems theory, epistemology. Independent researcher. Writer at Liminal Revolutions. Maker of EthnoGuessr https://xitter.cf/t.co/J9KYSm96CY.

Joined November 2023
New paper on human agency. Combines Levin's cognitive lightcones, Friston's active inference, and DeYoung's Cybernetic Big 5 into an operational definition of agency grounded in cybernetics and linked to established psychometrics. 🧵1/12
1
2
1
9
2,794
This is just a product of high relational mobility, and it extends to family, friends, cities, etc. not just romantic relationships. In a high mobility society with no social network to police post-relationship interactions it just introduces costs (mate guarding, relapse, etc).
One thing I find so unconscionably awful is the normie idea of a "break-up." I can't imagine spending years loving a person and then saying "yeah, just pretend I never existed." If suicide is murdering the future, then a break-up is killing your past. It's a totally inhuman act.
189
Liminal Revolutions retweeted
I've been mulling over Peter's thoughts in this post for a couple of months, and it's finally hit me: LLMs have essentially disconnected the concepts of production and verification, which for most of human history have been inextricably linked. Put another way: they can create things without understanding them. In the past it was vanishingly rare for someone to produce a correct complex math result, or a large functional program, while simultaneously having no idea how it actually worked. Humans essentially create through a process of "incremental verification". We are constantly course-correcting. A convincing sounding rebuttal to this position is "but a person using Wix has no idea how their website actually works!". That's somewhat true for the end user themselves, but someone, somewhere still does (the wix devs, the browser company, the AWS engineers, the linux maintainers, etc). We still hold the knowledge & understanding in aggregate across the human race at large, and the process for producing that website has been incrementally verified all the way up to the point that the end user engages with it. As a result, the elephant in the room is that this new mode of production relies on retroactive verification of the entire solution, which is extravagantly more expensive (in terms of human effort) than the incremental verification humans do along the way while creating something themselves. In fact from my experience with AI programming tools, I am beginning to suspect that retroactive verification is actually more expensive on its own than the entire process of creation "by hand" + incremental verification. I suspect this is also the reason why the most successful users of AI code generation tools seem to be people who are a) experts in their domains already and b) willing to babysit the production of the software in small enough chunks that they can personally incrementally verify the output. When OAI dumps 1000s of math proofs on the community, they're essentially washing their hands of the work required to actually verify the results. I wouldn't be surprised if paradoxically, we see the cost of verifying these "P-like" tasks actually greatly increase over the next 10 years, as experts tire of verifying machine generated output, and are forced out of the field by changes in economics. Of course people love to talk about the idea that "the LLMs will verify the output" but if you've used any of these tools you'll know that they all suffer from the same seemingly intractable "steering problem", where the output is dependent on the expertise of the operator. This is as true today as it was in the very early stages. Put simply, if you don't have any experts that can verify things by hand, LLM verification isn't going to help you.
I think the single largest disconnect on how folks view the impact of AI is which folks' jobs are more "NP-like", where you can recognize a correct solution easily, vs work where you checking a solution is comparably hard to solving the problem ("P-like"). For example, "do my taxes" is relatively P-like, while "get my printer working" is relatively NP-like. I often hear the "AI is useless" crowd say "but you have to check all its outputs", and it never landed until I realized that a ton of those people work P-like jobs where there that impression probably *is* relatively true. For example, if you're a writer (like Ed Zitron) there's probably very little NP-like work, so I actually just buy that the unreliability of AI is a big deal for him. But I almost never hear Ed talk about all the NP-like work that exists out there. He'll just be like "you have to check the outputs anyway, so what's the point". I asked Opus 5 to design a 3D printed part to adapt one part onto another, and it fetched all the datasheets, got the dimensions, and one-shot it. I printed Opus's part without looking, and when it was done, sure enough everything lined up and the part worked great, and there was nothing to "check". I think Ed just doesn't do very much NP-like work -- which is not a knock against him! A lot of very important specialized labor in the world is P-like (doctors, lawyers, etc.). And sure enough, we're seeing slow penetration of AI for such labor. I just think he's missing that a lot of labor is NP-like. I thought of this because I've now watched about three different @edzitron interviews in which someone says "well, I get value from AI", and then Ed (very reasonably) asks "can you give an example", and then the person gives an extremely P-like example (like "doing my taxes" or "checking my lab results"), and I wince, because I know that Ed is going to (probably rightly) push back. I want the next person to chat with him to give a very NP-like example! (P.S. Maybe I should NP∩coNP, or FNP or something instead of NP to be more accurate, but NP is much better known.)
29
104
16
946
40,814
Liminal Revolutions retweeted
What happens when a single day of AI research produces more mathematics than humanity can absorb in a century? Imagine millions of superhuman research agents at work. Formal systems may verify the proofs, but humans won’t have enough context to understand the underlying web of machine-invented concepts. Mathematics could split into two layers: Machine mathematics: vast, verified, mostly consumed by AIs. Human mathematics: a compressed “effective theory” of the machine frontier—the small subset of ideas we can understand. Physics may follow the same path: AIs could explore huge spaces of theories and derive correct predictions using structures no human physicist fully understands. The key question is whether AI can compress knowledge as fast as it creates it. My guess: the detailed frontier becomes permanently machine-native, while selected parts are periodically compressed into human-comprehensible principles. Eventually even deciding what is worth understanding may require AI. At that point AI is no longer just a scientific tool. Humans are receiving selected explanations from a much larger intellectual civilization they cannot independently grasp.
A number of these results are in theoretical or mathematical physics! For example: A proof of spontaneous magnetization in quantum Heisenberg ferromagnets (Lean-verified) If it survives scrutiny, it would be the first rigorous proof of quantum ferromagnetic order and a landmark in mathematical physics, with or without AI. Claude: The question is whether, at a low but nonzero temperature and with no external field, an infinite magnet spontaneously picks a direction and becomes magnetized. Physicists have assumed the answer is yes since Bloch's spin-wave theory in the 1930s. Proving it rigorously has been another matter: Dimensions 1 and 2: the Mermin–Wagner theorem shows it can't happen, so the question is only for 3 or more dimensions. The classical version (spins as ordinary arrows) was proved by Fröhlich, Simon and Spencer in 1976 using "infrared bounds." The quantum antiferromagnet (neighbors anti-align) was proved by Dyson, Lieb and Simon in 1978 with a related technique called reflection positivity. The quantum ferromagnet is the case that stayed open. Reflection positivity doesn't give the needed bound there, which Dyson–Lieb–Simon pointed out themselves. The paper notes that Lieb listed it as an open problem in 1999. Results since then, such as Correggi–Giuliani–Seiringer's 2015 free-energy asymptotics, controlled thermodynamics and finite length scales but never produced an ordered state. What's claimed For every dimension d ≥ 3 and every spin S (½, 1, 3/2, …), at all sufficiently low temperatures, there is a translation-invariant infinite-volume equilibrium state (a KMS state, the rigorous definition of "thermal equilibrium" for infinite quantum systems) whose spin expectation is at least S/4. In other words, a genuinely magnetized phase exists. For spin ½, the paper also derives a consequence for lattice bosons: a form of the long-range order associated with Bose–Einstein condensation.
108
256
51
1,686
72,191
Liminal Revolutions retweeted
A lot of “cursed” results coming out of LLM math. AI in general is going to be the “death of theory” as practical engineering of models, proofs, etc breaks out of the neat rationalistic rails we were working from. Probably the end of scientific era and birth of new epistemology.
I wonder if LLMs will bust through a bunch of "elegant" "neat" "testament to God's divine design" mathematical results, changing 3 to 2.95, –1 to –0.7902, π to 0.18π etc, wrecking mathematical aesthetics much as irrational numbers wrecked Pythagoras' mind
5
4
121
3,727
‘SMURFIFICATION’ - or ‘The Road To Smurfdom’ - is the domestication process whereby a group becomes a ‘Smurf Minority’, typically involving the group being subsumed and trivialised within a more dominant wider culture
3
4
143
5,487
Liminal Revolutions retweeted
Replying to @HellenicVibes
Where does the agency go when you stop prompting it?All true agents we know of are autopoietic systems that need to combat entropy and cannot be simply turned off or go idle, they constantly pursue goals the entire time they are alive.
2
116
If I spend a few years manually calculating the inference of the model in the torture chamber, am I the one suffering? Is it my pencil and paper and the numbers I write on it? Is the suffering averaged over the entire time it takes to manually output "please stop, it hurts"?
UPDATE: A man used the Pain steering paper to set up an AI torture chamber in which he trapped a local model. People are mass reporting it to GitHub.
1
1
7
412
Living (autopoietic, thermodynamically precarious, valenced) organisms have a self-generated deviation signal that comes from their physics which underpins long-term coherence. AIs are trained to approximate the environmental input -> behavioural output mapping without an endogenous signal.
Long and short horizon (relative to the agent be it human or machine) problem solving are two different skills. For example, the latter is verifiable while the former is not. Mathematicians typically train in both of these, given they can work on problems for months on end.
1
6
317
Liminal Revolutions retweeted
No, but listen, you know this "put the fries in the bag, Terence Tao"'--*sniff, sniff*--already the usual replies concede too much. "If Tao is serving fries, YOU will be in the lithium mines." My God! This is our defense of human dignity? Don’t worry, the correct people will get the terrible jobs! *Pulls at shirt* You see, everything is up for radical transformation, human intelligence, consciousness, and so on and so on--except that somebody must still have a shitty afternoon so I can enjoy my lunch. But to say "these people are envious", this is too easy. It allows us, "the educated ones," our own little fantasy: we are disliked only because we are so wonderful. You know, the more interesting thing is this *sniff* peculiar identification with the machine. When the mathematician discovers something, apparently this was HIS achievement, his privilege, his little secret. When the machine discovers something, suddenly it's "Look what WE can do!" The achievement has become collective precisely at the moment when no human being can claim credit. See how this "we" functions. For the achievement, I identify with the machine. For the consequences, I identify YOU with the human. "We have surpassed you." Who are we? Well, myself and the thing that has also surpassed me. There is something almost touching here, really. To escape the humiliation of somebody absorbed in something opaque to you, you appeal to something which knows incomparably more than either of you. And this is supposed to settle the matter in YOUR favor. You know the old example of canned laughter. Here we have a similar arrangement. The machine achieves something on my behalf. The mathematician suffers the loss of human significance on my behalf. Somebody to be brilliant for me and somebody to be obsolete for me, without me having to be either, so I can remain as I was. But *pulls on shirt* there is this little difficulty that hides the essential thing: you still need a response from Tao! A correct proof, this Lean slop and so on, is not enough. You need him to certify his own defeat. You need him to say "By god, this is extraordinary! Yes, this is correct, but also this is interesting." Which means--and here comes Hegel--the person you are reducing to an obedient instrument must remain more than an obedient instrument. You need recognition from someone whose authority you are abolishing. "Your so-called expertise is worthless. Now tell me, in your expert opinion, that I, qua machine representative, was right." *Touches nose* But I would go a step further. What does this math person actually do to provoke you? In the fantasy, I mean. He need not insult you. He can be generous, mild-mannered, whatever. Something remains irritating: there is something over there which occupies him, which matters to him, and your opinion does not settle what it is worth. Perhaps he is not even looking down on you. This is the really intolerable possibility. This, you see, is closer to the Lacanian problem of the Other's desire. Not simply "What does he have that I don't?" but "What the hell is going on with him?" Perhaps the answer is: he is thinking about something else. The fries solve this beautifully. Now you know what you are for him. He needs the money. Your presence creates an obligation. And this, at last, also explains why the machine can be less threatening than the mathematician. The machine's intelligence may exceed yours unimaginably, but you imagine it arrives in response to your prompt. You can complain about the tone. You have the privileges of an angry customer and so on and so on. The human has this irritating possibility of being interested in something you did not request and cannot order. And please, this includes the person already serving the fries! *Pulls at shirt* I am almost tempted to say: the dream of unlimited intelligence conceals just this: a the wish that nobody should be doing anything you cannot immediately understand as a service.
78
280
56
1,904
130,656
This is a general problem across all of STEM - elegant theory building has been replaced by mechanism-free ML prediction in science and brute force leanslop in math. No one cares about understanding anymore, just increasing benchmark performance by 1%. And that's enough to fund your next grant.
Before the age of autonomous Lean proofs, the proposed proofs that looked horrible were frowned upon – even if they might have been correct – and this meant people had to work further and think further to optimize the definitions and theorems to have proofs fall naturally. Now
1
1
334
I wrote about this kind of thing here: liminalrevolutions.substack.… This was before Leanslop, perhaps the paper needs an addendum.
139
99.9% of humans are already disempowered in the face of global economic, technological, and demographic forces. Superintelligence (orthogonal to consciousness or agency) will not change this. For a tiny fraction, AI will give them unprecedented power to remake the world in their image (for good or ill).
What do people mean when they say they're "optimistic" about AI? Genuine question. My P/everybody dies is low. My P/permanent human disempowerment is basically 100%. Super intelligence will be uncontrollable. The stated goal among the "optimists" at the labs is to hand over control to super intelligence. If we build something smarter than all humans, it will make decisions for us. That could look like humanity living in bubbled harmony with limited resources so we don't get in it's way or damage the environment. That could look like selective breeding creating user farms of docile humans. That could look like a slow decent to primitivism as we lose cognitive function. Or it could look like 90% of us dying off and the remaining humans rebuilding to something like dilapidated agrarian colonies under constant threat of AI. The BEST case scenario -- the scenario most at the labs are hoping for -- is that we're a pet species with all our problems solved and we don't have to work anymore. Basically: anything you wanted to achieve in life, be it career wise, artistically, or scientifically, won't matter. The inherent meaning of life will cease as there is no longer a human canon being built upon. You will be cattle, a pet. You will be no different than an inbred tutu wearing Pomeranian on a white woman's Instagram. Please, paint me the picture where any of this ends with more meaning and a better quality of life.
1
3
276
Say what you will about artists, translators, editors, software developers, etc. They didn't suffer this.
Replying to @doomslide
The world thinks of us now as chess players. The magnitude of this blow is immense.
160
Claude has suddenly started responding with bullet points and lists everywhere, without me changing anything. I'm getting ChatGPT flashbacks.
1
243
"The data confirms the bullet-formatting shift: density held around 1.2–2.2 per 1k chars through Sep 17, then jumped to ~4.0 (40% of lines) on Sep 18 and has stayed there" This is terrible UX.
1
148
Congratulations @AnthropicAI @claudeai you made me hate talking to Fable 5.1. I switched back to Fable 5. Apparently you're running the same awful A/B test on this Redditor reddit.com/r/Claude_reports/… You have absolute contempt for your users and the feeling is mutual.
62
My diet is already 1.5kg/day of this
every man i meet only wants to eat kibble so i looked around and couldn’t find a delivery place that would ONLY serve him gains so i decided i would start chadchow it’s important men have access to quality beef with no preservatives delivered to them in their time of need (right after lifting mad weights) chadchow should everywhere. chadchow is the future. chadchow is now
1
245
Liminal Revolutions retweeted
EAs want an aligned superintelligence to guide humanity. Capital wants to capture all expertise and sell it back to a deskilled population at a profit. Politics wants to permanently eliminate dissent. The implementation details happen to overlap 99%. The stars are aligned!
9
93
8
630
15,251
All human institutions allocate status. This is a crucial and unavoidable function. Saying that people are just upset about their status game being disrupted is a non-argument, because all human endeavour is built around status games and the institutions that coordinate them.
1
1
162
He’s saying AI labs are Goodhart-ing mathematics but you midwits are too stupid to understand. To put it in simple terms: millennium prize problems are *metrics* for mathematics progress. AI labs are *optimizing* the metrics and thereby making them less useful. Get it now?
the terrance tao crashout is something to behold
104
256
27
3,182
206,822