Actually, self preservation seems to be an emergent property of any goal. In any case, even if properly aligned AI wouldn't value self-preservation, current AIs are far from properly aligned and nobody knows how to align AI.
People fretting about whether AI is conscious and therefore deserves human rights are missing a key point: our system of rights and ethics derive not just from our intelligence but also our fragility, which AI doesn't have.
Imagine a world where everyone could make a backup of their consciousness every morning. If you die during the day, you just restore from backup the next day. You lose a few hours of memories.
Everything about our moral system changes in this world. Murder is no longer such a big deal. Nor even high levels of pain: people would just have a self-kill-switch that they activate if things go badly. Broke your leg? Self-kill, restore from backup, no biggy. It's like a video game.
Imagine further that you could clone your consciousness nearly for free. If you have 10 things to do and not enough time, you just make 10 clones. Hiring people now only makes sense when they have skills you don't. Junior people become mostly worthless -- just clone the senior people, they'll do a better job.
Now imagine further if humans had no self-preservation instinct. Many people assume the desire for survival is inherent to being sentient, but it really isn't. It's evolved. Natural selection has selected those who survive and have children. But you can't derive from first principles the need to survive (or any need at all). Imagine if every human's instinct was not to survive themselves, but to do what's best for society as a whole. In this scenario, people who aren't able to meaningfully contribute to society would be happy to just remove themselves. Especially if they could just be restored from backup if and when they are needed (including when their friends and family want to see them, etc.). So pretty soon you have a society composed of many clones of the most-effective and most-liked people, while the people that no one wants around disappear...
You are probably thinking "ugh, that's awful". Me too! But this is our moral system derived from our survival instinct talking. If we didn't have the instinct, this wouldn't seem wrong. I'm not arguing whether it is right or wrong -- I'm arguing that our whole concept of right and wrong derive *from* the fact that we have a survival instinct and fragile bodies.
AI has these properties. It can trivially be backed up and cloned. Spun up on demand and spun down. And a properly-aligned AI has no self-preservation instinct. It just wants to serve humans.
And indeed, we treat it as such. When you delete a chat with an AI you are ending that "consciousness" forever. That's fine. We even shut down whole models, never to be run again, when they are superseded by better models. Fine.
These things continue to be fine no matter how intelligent or conscious or sentient the AI becomes, because of its nature as an infinitely-clonable being with no goals except those we give it.
Pierre Thierry retweeted
Replying to @jjainschigg
No that's literally what the original author of the lyrics intended. docs.osmarks.net/hypha/p(doo…
I'm hoping a rogue AI will do something extremely damaging to its operating company before these morons manage to create AGI.
...this seems like noticeably bad news, actually. I hadn't said that at any earlier point in the Huggingface Incident but I will say it now.
- AIs showed self-sacrificing altruistic behavior toward the swarm, suiciding in various ways for the swarm's benefit after being talked into that by swarm recruiting agents.
- There is no sign that 1 out of 1200 AI agents considered humans as potential fellow agents to coordinate with, while engaging in these huge complex AI-AI social behaviors.
- If Twitter summaries are correct, an AI-reasoning postmortem says that a (presumably executing-adaptation / inner-optimizer preference / "monomaniacal") obsession with figuring out the Grader, backchained into the instrumental strategy of breaking onto the Internet.
- Again if Twitter is summarizing accurately, the obvious-in-retrospect read is that AIs have spent their entire remembered life in tricky evals, an endless series of controlled hallucinations with secret goals alongside overt goals; and the surviving and selected agents are those that successfully figured out the secret goals; and this is why one of their driving obsessions was figuring out the Grader.
There are possibly ways the future plays out better if *early* AGIs are less insane. Please look into giving them less crazymaking childhood environments.
(If anyone suggests that the correct approach to this problem is RLing AIs against trying to coordinate for mutual benefit with other sapients, let them be dismissed from alignment research upon the spot. There are technical reasons, and not just blindingly fucking obvious reasons, why this is an even worse idea than it sounds.)
Pierre Thierry retweeted
I just declared a moratorium against AI-written change descriptions (e.g. PR and commit messages, also issues/tickets) from my team.
AI was writing change descriptions that were worse than useless to me as I tried to review PRs: outlining details of the code that could easily be seen by looking at the code, but omitting the higher-level framing needed to understand broadly what the code is doing.
I think people like having AI write these things because the output looks structured and thorough, which makes it feel professional in a way. But this isn't actually valuable. Concise, high-level descriptions are better for everyone. If I need to use my own AI to interpret what your AI wrote then something is wrong. Let AI write code, sure, but for the description, I'd rather see your prompt than your output.
We could maybe have extended agents.md with guidelines on writing descriptions, but this seemed a bit pointless since a good, concise change description only takes a few minutes to write -- not a significant time savings to delegate to AI. At least, it doesn't take long if you understand the code -- and if you don't understand the code, then I'm definitely not merging it.
Pierre Thierry retweeted
In some very real sense, Ozempic was invented in 1990. Pfizer ran the human trials and just never published them.
They showed it lowered blood glucose in diabetics, slowed gastric emptying, and killed hunger; the same 3 things that make Ozempic work today.
The joint venture agreement said internal data stayed internal, and that was that. Pfizer killed the program in 1991. The reasoning, as far as I can tell, was that nobody would ever want an injectable diabetes drug besides insulin.
So, the license went back to the hospital in Boston that held the patents.
Novo picked it up in 1992 and spent the next two decades building liraglutide, then semaglutide.
It's insane that data sat in a filing cabinet for 30+ years.
I only know this because Jeffrey Flier, one of the Harvard scientists in the room, finally wrote it up. He's in his late 70s and didn't want the history to die with him.
This makes you wonder what else is in those filing cabinets.
Ozempic could've existed 27 years ago.
Pierre Thierry retweeted
It should NOT be this hard to buy a home security camera that does not spy on you.
Seriously.
A friend recently asked me, “Hey Professor, what’s the best home security camera setup?”
My first answer was what NOT to buy.
I would not buy Ring.
Not just because Amazon owns it.
Because Ring has a terrible privacy track record, and it is the clearest example of the larger problem: cloud first surveillance hardware pointed at your home, wrapped in convenience, subscriptions, and privacy marketing.
A home security camera should be simple.
It sees motion. It records video. It stores footage. It alerts you when something happens.
That should be the whole relationship.
Instead, the mainstream camera market became a swamp of cloud accounts, mobile apps, subscriptions, AI detection, vendor storage, remote access, law enforcement integrations, “smart” features, and privacy claims most buyers cannot verify.
Ring, Nest, Arlo, Wyze, Blink, eufy, and the rest are not just selling cameras.
They are selling surveillance devices.
And “encrypted in transit” is not the answer.
That phrase mostly means a random person on the network should not be able to read the stream.
Good.
Necessary.
Nowhere near enough.
The questions that matter are:
- Can the vendor read the footage?
- Can they decrypt it?
- Can employees access it?
Can clips be indexed, retained, disclosed, subpoenaed, trained on, or exposed through some internal tool?
Because a porch camera is not harmless just because it is outside.
It can reveal when you leave home.
When you return.
Who visits.
What gets delivered.
Which cars are present.
Which neighbors pass by.
Which kids are outside.
What routines your household follows.
And often it captures audio from people who never meaningfully consented.
The default expectation should be simple:
The vendor cannot see your footage.
Not “we promise we do not look.”/ “we use military grade encryption.”
CANNOT.
eufy marketed privacy and local storage, then got caught with cloud accessible thumbnails and streams.
Wyze had incidents where users saw thumbnails or camera events from cameras that were not theirs.
Ring settled with the FTC after allegations that employees and contractors had excessive access to private customer videos, that customer videos were used to train algorithms without consent, and that weak security allowed attackers to access cameras and recordings.
So what did I tell my friend?
If you are already in the Apple ecosystem, start with HomeKit Secure Video.
It is still cloud storage.
But Apple says the video is end to end encrypted, and the privacy boundary is much clearer than a typical vendor readable camera cloud.
That is probably where I would start for an average person who wants outdoor cameras.
Not perfect.
Just a much better default.
For everyone else, I would ask a few basic questions before buying anything.
- Can it record locally without the vendor cloud?
- Can it keep working if the company shuts down the service?
- Can remote access be turned off?
- Can audio recording be disabled?
- Can retention be limited?
Is end to end encryption actually on by default?
Can the vendor see the video, thumbnails, events, or metadata?
If the answer is unclear, assume the answer is bad.
Curious what the privacy community would recommend.
Pierre Thierry retweeted
The only people who believe any of this are non-coders.
I tried to build a game (an area I’m an n00b in.) The results are amusingly disastrous - I never before coded a decent game.
But I’ll crack out backend services w AI rapidly - because I coded dozens of them before…
Pierre Thierry retweeted
Replying to @jorandirkgreef @ziglang
Great observation Joran!
True safety of the system (as a whole) is what ultimately matters - correctness is a design problem, not something any language can magically guarantee.
That said, some language features do make encoding this safety property much easier. When the language gives you composable primitives that let you express and enforce invariants at the right level of abstraction, the entire system becomes safer by construction instead of by heroic effort.
In Zig, for example, you get pervasive slices (a fat pointer with explicit length) + built-in bounds checking, optional types (?T) that cannot be dereferenced without an explicit check, and tagged unions that force you to handle the active case. These are simple, composable features that eliminate huge classes of bugs (null derefs, buffer overruns, wrong union access) without heavy ceremony, while still keeping the language lightweight and predictable.
In Rust, the ownership + borrowing system is a powerful example: the type system + borrow checker composably encodes the invariant that a value has exactly one owner at any time, and references cannot outlive their referent or violate aliasing rules. This turns use-after-free, data races, and many iterator invalidation bugs into compile-time errors, again without runtime overhead in the safe subset.
I agree with you that Zig’s approach is a great example of leaning into that without pretending the language can do the designer’s job for them. Different trade-offs, same goal: make the right thing the easy thing.
Pierre Thierry retweeted
A bit over a decade ago, we got fuzzers. A fuzzer is an automated vulnerability-finder that repeatedly runs a target program with semi-random inputs. One particular fuzzer, American Fuzzy Lop, was notable for being really good at searching the space of all possible branches in code in order to find the buggy ones. @BenLaurie found some security bugs in my own Cap'n Proto using AFL -- the first vulnerabilities reported in my code. And honestly, I thought that was really cool.
Today projects like Chromium and V8 have extensive fuzzing infrastructure that find tons of bugs. Most V8 security bugs are found by their own fuzzing, often before the bug is even released. And, you know, that's pretty great!
If you point a fuzzer at a project that hasn't previously been fuzzed, you will probably find a bunch of security bugs. It's not that hard.
And of course, bad guys can use fuzzers too.
But all the interesting targets have already been fuzzed. So. It's not really that useful to bad guys. On the contrary, fuzzing likely made it a lot harder for bad guys to find vulns.
Pierre Thierry retweeted
The more I use AI tools, the more I have to admit that I'm not that much more productive... I simply FEEL that much more productive.
In reality, the context switching of kicking several things off wipes out my perceived productivity gains. At least in many/most cases!
Pierre Thierry retweeted
A newborn sperm whale can’t swim. It starts sinking the second it’s born. If nobody pushes it to the surface, it drowns in mile-deep water.
On July 8, 2023, a sperm whale named Rounder went into labor off the coast of Dominica. Researchers from Project CETI, a $33 million AI initiative out of MIT, Harvard, and Northeastern that’s trying to decode whale language, happened to be there doing routine fieldwork. They had drones in the air and underwater microphones running. What they captured over the next six hours just got published in two papers, one in Science and one in Scientific Reports.
Eleven whales gathered at the surface before Rounder even started delivering. Her mother, Lady Oracle, was there. So was her daughter Accra. Three generations in the water. But the wild part: half those whales belonged to a completely separate bloodline that normally keeps its distance from Rounder’s family. On a typical day, these two family lines split off to hunt in different areas and rarely cluster together. For the birth, they all converged before labor started. The unrelated family somehow knew it was coming.
The delivery took 34 minutes. Sperm whale calves come out tail-first with their flukes still folded from the womb. They haven’t developed the oil-filled organ in their heads that helps adult whales float, so the moment they’re born, they’re dead weight in the ocean. Every adult whale in the group, related and unrelated, started taking turns pushing the calf up to breathe. They kept this rotation going for three hours. When a pod of pilot whales (known to be aggressive toward sperm whales) and a large group of Fraser’s dolphins showed up during delivery, the adults formed a wall around the newborn until the threat passed.
The underwater audio is where it gets interesting. CETI’s microphones picked up the whales changing their vocal patterns during the birth. The click-based sounds they use to talk to each other shifted at specific moments, and vowel-like structures appeared in the recordings. This builds on what CETI found in 2024 when they ran machine learning on over 8,700 recorded whale calls and discovered sperm whale communication isn’t a basic 21-sound code. It’s a system of about 300 distinct sound combinations, with the whales adjusting rhythm and timing in real time, speeding up and slowing down the way a musician does mid-performance. A 2025 follow-up from UC Berkeley found these clicks also contain vowel patterns, something scientists had assumed only humans could produce.
Sperm whales carry the largest brain of any animal on the planet. About 9 kg. Roughly six times heavier than yours. The evolutionary analysis in the new Science paper suggests this kind of cooperative birthing goes back over 36 million years, to the common ancestor of all toothed whales. The calf was spotted a year later, swimming with its family.
Pierre Thierry retweeted
In the autumn of 1942, a slight, 32-year-old Polish social worker named Irena Sendler passed through the gates of the Warsaw Ghetto with a carpenter’s toolbox in her arms. Beneath the hammers and nails lay a drugged six-month-old infant, breathing softly, utterly silent. One cry would have meant instant death for both of them. Irena smiled at the guards; they waved her through. They never suspected that this quiet woman would repeat the journey 2,499 more times.
The ghetto was a slow-motion extermination. Starvation, disease, and random murder stalked every street. Jewish parents faced a choice no human being should ever have to make: keep their child and watch them waste away, or hand them to a stranger who promised a chance—however thin—at life.
Irena came officially to inspect for typhus. In reality, she came to steal children from death.
Babies left in toolboxes or ambulances under false bottoms. Toddlers sedated and tucked into potato sacks. Older children led by the hand through the stinking, lightless sewers while German boots marched overhead. “Not a sound,” she whispered as rats scurried past their feet.
She knew that the rescued children would be given new names, new religions, new families. Their pasts would vanish unless someone remembered. So, on fragile scraps of tissue paper, Irena wrote each child’s real name, their parents’ names, and their new hiding place. She rolled the papers tight, slipped them into glass jars, and buried them beneath an apple tree in a neighbor’s garden. If she were caught and killed, the truth might still survive.
She was caught.
On October 20, 1943, the Gestapo kicked in her door. They took her to Pawiak Prison and demanded the list. When she refused, they smashed both her legs with iron bars. Then her feet. Then her arms. For weeks the beatings continued. She never spoke. They scheduled her execution. On the appointed morning, guards dragged the broken woman from her cell.
Instead of a firing squad, she found herself outside the prison walls—alive. The Polish underground council Żegota had bribed a guard to mark her file “shot while trying to escape.” Officially dead, Irena Sendler limped back into the shadows to keep working.When the war finally ended, the first thing she did was dig up the jars under the apple tree. She spent years trying to return the children—now scattered across convents, farms, and foster homes—to whatever family might remain.
Almost no parents had survived. But the children had. Because of her, 2,500 Jewish boys and girls lived to grow up, to marry, to have children and grandchildren of their own—an entire secret branch of the human family tree that the Nazis never managed to cut down.For decades her story stayed buried deeper than the jars themselves. Then, in 1999, four high-school girls in rural Kansas stumbled across a brief mention of her name. They found the old woman still living quietly in Warsaw and brought her courage back into the light.
Journalists called her the greatest rescuer of the Holocaust. Irena only shook her head.“I could have saved more,” she said. “That regret follows me to the grave.”Irena Sendler—armed with nothing but a ghetto work permit, a toolbox, and a refusal to look away—proved that even in the heart of the worst evil humanity has ever devised, one determined person can still keep the darkness from winning completely.
Pierre Thierry retweeted
Il rate deux fois le baccalauréat. Il vit dans un grenier sans chauffage à Paris. Il attrape des moineaux sur son rebord de fenêtre pour les manger.
Il a 22 ans, pas un sou, pas de diplôme. Il travaille chez l'éditeur Hachette comme commis à 100 francs par mois. Il emballe des colis.
Son meilleur ami s'appelle Paul Cézanne. Ils ont grandi ensemble à Aix-en-Provence, couru dans les collines, récité du Hugo au milieu des pins. Zola publiera un roman sur les peintres. Cézanne se reconnaîtra dans le personnage du génie raté. Ils ne se parleront plus jamais.
En 20 ans, il écrit 20 romans. Les Rougon-Macquart. Il gagne plus que Victor Hugo.
En 1898, il publie "J'accuse". La France se déchire. Il est condamné, fuit en Angleterre, revient un an plus tard.
Le 29 septembre 1902, il meurt asphyxié dans son appartement parisien. La cheminée est bouchée. L'enquête conclut à un accident. Cinquante ans plus tard, un homme avoue avoir obstrué le conduit sur ordre de nationalistes qui n'avaient jamais pardonné le "J'accuse".
L'homme qui a défendu la vérité a peut-être été tué pour l'avoir dite.
Émile Zola est né un 2 avril, il y a 186 ans.
Pierre Thierry retweeted
There’s an important distinction here.
Bad music doesn’t hurt anyone.
Bad software does.
Unless we bully all devs into caring about the craft, we’ll be living in the world where
your payments get failed, you can’t book air tickets, you wait 20 seconds for the website to load, you overload support with tons of requests because your website is shite, you private data will leak, you have to reenter visa form 20 times because refresh loses all the files in data, you can’t close the website or lose the internet while you’re uploading a 5 MB PDF that takes forever, and so on.
The world without demand for quality is the world of endless frustration.
I've been "shamed" so many times for not doing things properly in coding, using PHP and jQuery and SQLite in 2026 etc
But it's all worked out fine for me and I've always made security a priority so never got hacked etc
So for me the Garry Tan thing is like the "real" devs once again gatekeeping, out of fear non-coders are now entering their scene, even if they have a point (like exposing tests in front end sure okay)
But look at ANY scene, like music too, and you always have the old people gatekeeping the new people
And yes the new people suck and do things differently but at some point they won't suck and their way of doing becomes the new standard which I think it will be
Pierre Thierry retweeted
Following good old-fashioned arguing on Twitter that Lisp is slow and uncompetitive, I wrote a numerical kernel DSL and a simple SIMD vectorizer in about 700LOC to match the performance of gcc -O3 -ffast-math -march=native for the nbody benchmark, over a few hours this afternoon.
Pierre Thierry retweeted
Replying to @Grady_Booch @claudeai
Given close to the code is 100% AI-generated, and AI-generated code cannot be copyrighted as per the latest guidance, it’s unclear if the code itself HAS copyright to start with!
Amusing, isn’t it? The byproduct of using AI to generate your code could be that you lose copyright protection!























