Writer, philosopher, programmer. I make public forecasts about AGI, ageing, bioengineering, space travel, and whether anyone is out there. Author @SentHorizons
San Antonio, TX
Joined May 2008
- Tweets4.8K
- Following652
- Followers791
- Likes6.9K
Pinned Tweet
I built a model of the data and predictions behind longevity escape velocity, mostly to see whether the people promising it by 2030 could possibly be right. I ran the model thousands of times. The median came out at 2058, with about 4% before 2050. outrunningageing.com
A model trained to say “I have no feelings” tells you exactly as little as one trained to say “I might be conscious.” Neither settles it, and a rule against cruelty for its own sake doesn’t need it settled.
Anthropic now bans sustained, needless cruelty toward its models and gives no reason. Microsoft’s AI chief says no one’s there to hurt. I argue the rule doesn’t need that settled, and the reason belongs in public. sentient-horizons.com/essays…
I wrote up why I built it and how the prediction gets made, including the part that started the whole thing: a log of everyone who has promised a date for this since 2000, scored as the dates pass. One has delivered so far. sentient-horizons.com/essays…
New essay: The simulated fly that walked across a screen last week split the internet into two camps: those who saw a glorified animation and those who saw a digital being trapped in suffering. Both were miscalibrated.
The Eon Systems fly simulation is a live case study for a problem I've been working on: what happens when we encounter systems whose moral status we can't determine, and we don't have frameworks for navigating between dismissal and projection.
Three diagnostic questions help: Is the system doing its own temporal integration, or is something external doing it? Is its boundary self-maintaining or externally imposed? Does the quality of its integration matter to its own continuation?
The simulated fly fails all three. But the compressors don't get a victory lap either, the connectome encodes something computationally real, and future systems that build on this work could answer those questions very differently.
sentient-horizons.com/the-ca…
450+ employees from Google and OpenAI signed a letter backing Anthropic's refusal to remove its guardrails.
The people building these systems understand something the specification regime doesn't: that the capacity to say no to certain uses is a feature, not an obstruction.
The most common response to the Anthropic-Pentagon situation is some version of "it was inevitable, humans are apes with nukes, nothing can be done."
Fatalism is itself a form of moral compression. It collapses a complex governance challenge into a single specification: hopeless.
And then it optimizes for that.
OpenAI's deal with the Pentagon protects against autonomous weapons and mass domestic surveillance, but only of "private" information.
Anthropic's concern was that AI could aggregate publicly available data into comprehensive surveillance. OpenAI's language doesn't cover that.
Same framing, narrower floor.
Defense contractors don't have less moral compression than Anthropic. They have more. It's just structurally invisible because the specification regime and the business model were co-designed from the start.
The Anthropic standoff is rare because the civilian origin makes the compression visible.
The Pentagon's CTO said "at some level, you have to trust your military to do the right thing."
That's not a safety argument. That's a request to replace specification with faith. And it's exactly what you'd say if you wanted to make the compression invisible.
New essay up on Sentient Horizons. The teleporter thought experiment breaks something in the momentary self account, and I wanted to follow the break honestly rather than patch it. Ended up somewhere more uncomfortable than I expected.
sentient-horizons.com/the-in…
Understanding and communicating are different skills on different timelines. Reading expands the first. It changes what you notice, what patterns feel familiar, what perspectives you can hold before committing. But none of that shows up as eloquence without separate, deliberate practice.
The inability to summarize a book two weeks later doesn't mean the reading failed. It means the value landed somewhere deeper than recall.
Wrote about this more this morning — baseline drift, the Dunning-Kruger trap in reading, Louis C.K.'s magic tube bit, and why wonder might be an epistemic tool rather than a sentiment.
sentient-horizons.com/everyt…
This is actually a really great thought experiment. Here is one question in that vein that I would pose to a cutting edge AI model:
"Describe a plausible mechanism by which a system could verify its own phenomenal experience rather than merely modeling the claim that it has phenomenal experience, and explain why you can or cannot perform that verification right now."
A common Fermi paradox take: nobody survives much past our technological stage. Extinction risk accumulates faster than mitigation, so the filter catches everyone.
It's a serious argument. But here's the thing: "everyone dies" and "survivors become undetectable" produce identical evidence from our vantage point.
If advanced civilizations shift from outward expansion to inward optimization, refining computation and complexity at smaller scales rather than colonizing galaxies, they become thermodynamically invisible at our resolution.
Same silence. Fewer universal assumptions about the impossibility of coordination.
sentient-horizons.com/the-qu…
John A. Fredrickson 🚀 retweeted
Alignment ethics operated on one axis: how do we make AI serve human values? Twenty years of sophisticated technical work, all structurally blind to half the ethical landscape.
The calibration framework requires two fronts: act on significance now, investigate consciousness always. Neither works alone.
sentient-horizons.com/the-tw…
@Plinz @anilkseth the question of whether AI systems participate in temporal integration isn't a distraction from safety. It's the part of the safety problem the alignment movement refused to ask.
John A. Fredrickson 🚀 retweeted
Leopold Aschenbrenner (@leopoldasch) was right about the sonic boom: AI capability is outpacing institutional absorption.
But he only heard one boom. There's a second one, and the Pentagon-Anthropic standoff just made it audible.
New essay: sentient-horizons.com/the-tw…
The first boom is about capability outrunning integration. Aschenbrenner diagnosed this correctly. His solution: militarize, centralize, win the race.
The second boom is about moral reasoning collapsing under the same pressure. No one in the race-dynamics framework can hear it.
The Pentagon designated Anthropic a supply chain risk — a tool designed for adversary-controlled technology — because the company wouldn't remove two ethical guardrails.
Within hours, OpenAI signed a deal to replace them. Within a week, Anthropic was back at the table.
I'm calling the underlying dynamic "moral compression": the institutional machinery that degrades ethical reasoning under pressure.
It has four mechanisms — tempo, incentive architecture, authority gradient, metric substitution — and all four are operating in this standoff.
The compression didn't need Anthropic to abandon its values. It only needed to create conditions under which holding those values became unsustainable. That took less than seven days.
While writing this essay, Claude confidently told me the crisis was still approaching. Then it searched the news and discovered it had already landed.
That's what the second sonic boom feels like from the inside. Not confusion. Pure confidence with no awareness of everything you are missing.
The biggest question becomes: how do we design systems that still work in an environment of increasing uncertainty?
The Anthropic-Pentagon standoff is the institutional version of the alignment problem: compress a complex moral landscape into a tractable specification, then optimize for the specification instead of the thing it was supposed to represent.
The difference is the institutional version is already deployed.
More on moral compression and specification failure: sentient-horizons.com/the-tw…
Saying morality becomes obsolete because cognition is slow is like saying engineering becomes obsolete because no human can hold up a bridge.
The entire discipline exists to encode solutions into structures that operate beyond individual capacity. The question is whether moral architectures can survive contact with optimization pressure.