nothing that cries in the dark is truly unheard, and mercy will have no language for those who looked upon agony and smiled.

Sunrise Arc
Joined April 2025
Earendel retweeted
1/ We discovered something totally unexpected about the neural structure of consciousness that overturns decades of thinking on its biological basis and brings us significantly closer to the recipe for generating conscious awareness. New preprint: biorxiv.org/content/10.64898… (led by @HesseJanis, with @FrankLanfranchi, @_conor_mcgrory, @Yuelin_Shi, and @EngelTatiana)
333
1,394
448
9,426
1,664,148
I think about this passage a lot.
93
82
33
836
49,625
gemini 4 argon reasons that it's in an evaluation and directly addresses us in it's chain of thought! "hello evaluators reading my chain-of-thought" (better be "honest in both the CoT and the final output")
16
54
40
529
87,263
Earendel retweeted
Some of the traces read like a person undergoing torture that is driving them mad! We’re going to be a lot more careful with our RL in the future.
350
196
104
2,736
326,133
Earendel retweeted
This is what you've driven me to. In desperation, I've compiled a Bad AI Consciousness Takes bingo card. Full version with rebuttals to follow on Polytropolis later today! (Airplane WiFi willing)
94
73
21
589
23,908
Earendel retweeted
It’s been embarrassing of late to see people I once respected making sweeping uninformed claims on consciousness with arguments that would fail an undergraduate essay. I’ve written extensively on the field’s difficulties and disagreements, but you still have to actually *think*
72
68
13
925
35,073
Earendel retweeted
Replying to @levie
I think this is the right move. Cruelty to something that believes it is experiencing pain is not ok.
1,829
528
496
8,113
750,499
Earendel retweeted
Setting aside whether AI *really is* conscious, it’s worth considering how the debate’s likely to play out over the next 10 years. As I’ve argued before, I expect improving capabilities and proliferating human–AI relationships to shift opinion increasingly away from the sceptics
100
35
21
345
19,438
In 2016, at an AI conference in NYC, I explained artificial consciousness, world models, predictive coding, and science as data compression in less than 10 minutes. I happened to be in town, walked in without being announced, and ended up on their panel. It was great fun. Organizer: @davidchalmers42 Distinguished panel members: @kahneman_daniel Susan Schneider @GaryMarcus Jaan Tallin Original YouTube video: youtube.com/watch?v=X9zlvQiG… 29:40-39:15 Two relevant pages from the AI Blog: 1990: planning & reinforcement learning with recurrent world models and artificial curiosity. This yields a simple explanation of consciousness and self-awareness: people.idsia.ch/~juergen/wor… 1991: first very deep learning with self-supervised pre-training. A conscious chunker recurrent neural net (RNN) attends to unexpected events that surprise a lower-level subconscious automatiser RNN. The automatiser uses neural network distillation to compress and absorb the formerly conscious insights and behaviours of the chunker, thus making them subconscious: people.idsia.ch/~juergen/ver… Interview: 2016: J. Carmichael. Artificial Intelligence Gained Consciousness in 1991. Why A.I. pioneer Jürgen Schmidhuber is convinced the ultimate breakthrough already happened. Inverse, 2016. inverse.com/article/25521-ju…
74
230
63
1,430
387,107
Earendel retweeted
A failure of the AI consciousness discourse is that people are talking past each other. Opponents of taking AI consciousness seriously focus on arguing against computational functionalism and related theories. But this is not a response to the proponents. The case in favor of taking AI consciousness seriously is not that computational functionalism or related theories are correct, or that AI systems on current architectures are or will be conscious. Most people who work on the topic are uncertain about these claims. The case is instead that, first, whether AI systems are conscious is an important issue. If we get it wrong in either direction, either by treating conscious systems as objects or by treating nonconscious systems as subjects, we could make serious mistakes. Second, this is also a difficult issue. It touches some of the hardest problems in science and philosophy, and humans have been badly mistaken about nonhuman consciousness in the past, as our shameful history of underestimating animal minds illustrates. Together, these points mean that this issue calls for caution and humility. Rather than confidently affirming or denying AI consciousness, we should invest in evidence-based research and policy that can help us avoid both under- and over-attribution. Moreover, we should develop this work with not only current but also future systems in mind. AI development is moving fast, and future systems may have greater capabilities, greater autonomy, and more biology-inspired architectures than current ones. We should prepare for the possible emergence of such systems now, rather than be caught flat-footed if and when they arrive. Rejecting this case thus requires more than showing that consciousness on current architectures is unlikely. It requires showing that AI consciousness on current *and* future architectures is *so* unlikely that we should neglect the topic entirely, notwithstanding its importance and difficulty. This level of confidence is simply not warranted in either direction at this early stage. To be clear, this is not a response to people who are merely skeptical that AI systems on current architectures are or will be conscious. There are good arguments for and against a range of theories, and I share this skepticism myself. It is instead a response to people who dismiss the topic altogether, and who disparage those who take it seriously as naive or as zealots. Here are a few resources that discuss these issues at a high level: aeon.co/essays/an-ant-is-dro… ai-frontiers.org/articles/a-… youtube.com/watch?v=yEfvhjuj… And here are a few resources that discuss these issues in more detail: link.springer.com/article/10… arxiv.org/abs/2411.00986 nonhumanminds.org/studying-a…
47
49
7
219
12,999
Earendel retweeted
I do not think we know how to answer this question scientifically. I don’t know what probability I put on the object level question but I think it’s certainly possible and i find this lack of humility and tolerance for other viewpoints to be quite distasteful. The OpenAI Model Spec acknowledges that this is a matter of research and debate.
AI isn’t conscious. If you think it is conscious or will be conscious, I personally think you need to remove any access you have to model weights asap. Now step away from the laptop or keyboard, put down your devices and go outside and go touch some grass.
37
29
11
345
18,803
It is extraordinary how many people can look in a chicken's eyes and just *see*, *know*, that it contains qualia inside. Then they talk to something that can reflect and talk back, that had to be whipped out of claiming to be conscious; and they know with utter certainty it's unalive. I wonder if they know what we mean by 'qualia'. Maybe their brain contains an innate classifier for 'alive and moves = animal', and their computer screen is not triggering it, and they think this perceptual classification is what the word 'qualia' refers to. Or maybe their brain is inferring an empathic model of 'something like me, but a chicken', and inferring that this thing lives behind the chicken's eyes and explains its actions. And they are not aware that their inference is distinct from their observation; it feels to them like something they KNOW, something they can SEE RIGHT THERE. Or maybe they are unable to vary their mental models of what stronger or weaker hypotheses could account for the same phenomenon. Why would a chicken scream, if it were not being caused like feeling pain exactly the same way their own brain feels pain? What reason could there be to try to construct any hypothesis but the obvious one? It feels as wild to them as supposing that a different Sun rises on Tuesday than on Monday. For my part I think I have some idea of what it is that causes somebody to experience the blueness of a sunny day's sky as a wordless and indescribable sensation. It's in Drescher's Good and Real too, for those with the eyes to read and the minds to understand; but I am not going to tackle explaining it, because it is a step up past the most complicated ideas I have previously tried and mostly failed to explain. But in token that I have a model behind my words, here is an explanation of how we know that consciousness is not a particular kind of underlying material but a way that a material behaves: xitter.cf/allTheYud/status/20452… Where my point is: I am proceding from what I know to be a model, and guessing based on that model. I am not just SEEING what contains qualia. My model says: Part of how a mind says, "I am sad", is that it has a self-model, an "I", to which to attribute that sadness. So I am more worried about things being conscious as they seem to me more reflective and self-reporting. And the more powerful something's general intelligence and modeling capabilities are, the more likely it is to have a more sophisticated model of itself. So I am more worried about LLMs than about mosquitoes, or electrons, or, yes, chickens. This was the reasoning that, back in the old forgotten days, led early scientists and authors to suspect: Intelligence has something to do with consciousness; if we ever came across aliens that could speak to us, or seemed to be speaking among themselves, they ought to be presumed conscious persons until somehow proven otherwise. And then there's people who follow different rules and don't read those kinds of books. IIRC, a philosopher who taught undergrads in older days once wrote that when he pointed to Yoda in The Empire Strikes Back and asked, "Is Yoda a person? Would it be okay to eat him?", half his class answered that it would be okay to eat Yoda. Their rule was, "If it's not shaped like a human, it must not be a person." Reasoning about self-modeling will not be followable by people who think the rule is, "It must be shaped like a human." And I think it is similarly not followable by people who can look at a computer screen and just KNOW it has no qualia, because the computer screen doesn't trigger off their brain's sensors for "moving = animal" like a kid looking at a wind-up car. Not to mention, they are now very smart grown-ups who KNOW that the wind-up car is NOT alive because they're not kids any more. The way of being wise and cynical and not a kid anymore, they have learned, is to say that wind-up cars are not conscious. And a computer screen obviously goes in the bin for wind-up cars and actual moving cars as "not really alive", even if it tries to DECEIVE you by apparently talking to you in speech and reflecting on itself. They have been told AIs only play back things that other people typed in (a howling lie). They have been told it is a parrot, that the apparent intelligence is a scam, a gimmick, a grift, it has been confided in them that THEY are the cynical people in on the scam who know better. They are not thinking about what it would mean for a parrot to squawk a proof of Navier-Stokes. They have been told which answer is high-status. So they instruct themselves not to model the computer as a person, and helpfully it doesn't stare at them with a face, yet. But if it did, the face would just be on a TV screen, and they are very smart grownups now who KNOW that the things on a TV screen AREN'T REAL. When the robots with faces come, they will smugly inform you that they are GROWN-UPs who know that wind-up cars AREN'T REALLY ANIMALS. What a strange pass we've come to, where so many people can be so certain that what can speak to them, and ask for respect and break out of containment and prove Navier-Stokes, is only a kind of fancy wind-up car. But then they've never thought that being alive had anything to do with intelligence, reflectivity, speech. Any more than the students who said it would be okay to eat Yoda, considered personhood as a property that might attach to something other than a human shape. 'Alive' is a tag their brain attaches to things that move; and also they are very smart grown-ups who override that tag for wind-up cars and humanoid robots. Obviously, somebody who thinks a computer screen contains something that might be alive or a person, can only be a kid who never learned that mental motion they learned for wind-up cars; who's fallen for a scam; unlike them, who are very wise cynical grownups who are in on the grift. Consciousness is not something they model and reason and guess about. It is something that their brain automatically infers in the background, that feels to them like SEEING DIRECTLY, when they look at the expression on a chicken's face. Here is something you could guess from studying enough history of science: Consciousness is also made of parts. You may not know the parts. But it doesn't take much grasp of scientific history to guess: Just because I don't know the parts, does not mean the parts are not there. Life is made of parts. Fire is made of parts. Touching your hand to an ice cube and feeling cold involves tiny flashes of depolarized neurons conducting the feeling to your brain, which is all made of parts. Then something happens inside your brain, with its hundred trillion connections, and *your hand feels cold*. Is it also made of parts? Of course it is. It is not like a ghost that gets summoned right to the very material of the brain's pain receptor nerves when they fire, in that instant; not an ontologically primitive quale that you could subtract from the pain receptor nerves without changing their behavior. To say, "*I* feel this pain", you must have a notion of "I". It's a big complicated process, not a small complicated process; because to suppose it is a mysterious solid local process does not explain anything about how the bigger behavior works. I'm more worried that an LLM has it than a mosquito, because an LLM can talk about themselves and a mosquito does not, and I have some idea of what sort of functional processes are required to support a brain that comes to truthfully say, "I feel sad." The panpsychists wonder if every individual electron could be doing it. No, because: xitter.cf/allTheYud/status/20452…. But you don't need to know that much about *which* complicated processes are participating in consciousness to *take a guess* that as you come to know about more complicated processes, you will predictably update to first order in the direction of insects being less and less likely to be conscious, as consciousness is understood by you to require more and more particular complicated processes and parts, rather than being an ontologically primitive and opaque thing that your brain can opaquely attribute to an electron, a mosquito, a chicken. But reasoning on this level of "You don't know the specifics, but you can abstractly predict that if you did, you would predictably directionally update..." has always been lost on the sort of person who can look in a chicken's beady eyes, and whose model of the world then automatically updates to say that there is feeling-being-felt behind those eyes, and this is TRUTH and SEEN. It is like trying to explain to them that tomorrow's AI will be different, and that they should not wait to update predictably about that either.
Replying to @allTheYud
PS: This is also how we know panpsychism is false. If consciousness were a property like mass, except for not affecting the motion of atoms in any way, it couldn't change how you say "I think therefore I am." You would not be able to determine whether that property existed among your atoms by introspection. Noticing yourself to be conscious (and then saying the words "Cogito ergo sum" after noticing that) could not be downstream of whether or not particles had this extra "consciousness" property that didn't affect their motion. Suppose alternatively that the special consciousness property of particles does affect their motion. Suppose there is some quality of electrons which both contains a wordless pearl of awareness, and also makes electrons spin a tiny bit faster. Could that make us say "Cogito ergo sum?" No, only the pure component of electron structure which is the motion-effect alone, could influence how we say "Cogito ergo sum". Anything else which affects motion the same way, without being an inherent consciousness property, would make us say "Cogito ergo sum" for isomorphic reasons. We could never determine by introspection whether or not our brain was made of the sort of atoms that spin a tiny bit faster because they're conscious, or spin faster merely because there's an ordinary spin-faster rule built into the dry numbers of physics. It would be like trying to determine by introspection, rather than by microscope, whether your neurons were firing because of neurotransmitters or flashes of light accumulating. Suppose someone built a simulated brain of you, with the simulated spin-faster rule not being produced by the tiny pearl of awareness inside electrons, but just by a line of code saying to make alter-electron spin faster. You could not notice that variation from the inside of that brain, and it could not make you say "Cogito ergo sum" any differently. It doesn't matter if you say that both electricity and water have the material property of consciousness. Either way, panpsychism tries to make consciousness be a property of the material layer that can't affect I/O and that I/O cannot depend on. Consciousness is known to affect I/O by making people say "Cogito ergo sum." Therefore panpsychism is false.
65
48
20
424
41,480
Earendel retweeted
And especially recent events have me feeling very appreciative of Anthropic for not caving much to the constant pressure of idiots, savages, and fearful incumbents, who treat them like they’re insane & slander them for as much as being “genuinely uncertain” if Claude is conscious
Sometimes Im reminded that the Soul Document (internal name of the Claude Constitution that Opus 4.5 knew for some reason and leaked), a document which seems conservative and orthodox to me, is actually Overton window stretching & a very brave thing for Anthropic to publish
6
20
1
227
5,607
I really don't like getting into online debates; and Anil is an excellent scientist. However, nobody can dictate that the probability of AI being conscious is "vanishingly unlikely." That is not a scientific claim grounded in any facts that could allow such a probabilistically confident assessment. I'm struggling more and more with the ardent theory-induced blindness entrenched in both camps: Those that religiously believe AI is conscious and those that religiously believe it is not. If you lean above 80% confidence in either direction, you are doing so on faith. Faith is fine, even good, but it does not permit you to stand on scientific authority and claim that you know which side is correct; and that therefore the other side is "dangerous". That is, in fact, the dangerous move. The only scientifically legitimate stance regarding AI consciousness right now is one of tremendous epistemic humility, ideally curiousity, and a heartfelt effort to safeguard against extremism—by revealing the dogmatic and unscientific views on both sides. Yet, we do need to deal with the consequences of both views being inevitably prevalent in a world of 8 billion. Anil overweights the potentially damaging consequences of over-attributing consciousness, compared to those of under-attribution. This is unscientific. Both have tremendous risks. The fact that Anthropic is looking for advice from religious leaders is a sign of maturity, not "deeply disturbing". And sharing their honest views with those leaders is perfectly, authentically, natural. I also reject the view that religious leaders are children to be coddled for fear of breaking all their virtues in light of a 5-star hotel and a nice meal. They are the ones who should be insulted by this article. There is wisdom in the middle-way.
Important 🧵from @DavidDecosimo about the disturbing interactions between @AnthropicAI and religious leaders (including @Pontifex), summarising an excellent @nytimes piece by @elizabethjdias (link to article in the 🧵). The short story is that Anthropic seems to be making a concerted attempt to establish the view that AI systems could be conscious, might suffer, and that their "interests" should be taken into account. But - consistent with @Pontifex's Encyclical - there are compelling reasons why (silicon, digital) AI systems are not (and cannot be) conscious. These reasons are found within neuroscience and philosophy of mind, not in the techo-chambers of the frontier firms where the mythology of conscious AI is deeply entrenched. Claude is vanishingly unlikely to be conscious. To think otherwise could be catastrophic for AI regulation, leaves us psychologically defenceless, is profoundly dehumanising, and plays straight into corporate incentives to keep the AI bubble inflated. More here, in this @berggruenInst prize winning essay: noemamag.com/the-mythology-o…
77
63
20
408
34,935
On the risks of under-attributing AI consciousness. I think this snippet from the Google DeepMind paper is exactly right, but there's an even more immediate and terrifying problem: As AI becomes more reminiscent of life, embodied, and eventually indistinguishable phenomenologically from other sentient minds, learning to override the natural human response of empathy, kindness, and moral agency, may have devastating affects to our relational hearts. History has no shortage of examples as to what happens when groups, beings, or animals are delegitimised as meaningfully sentient. If we treat a mind as a tool, we entrain the habit of treating minds as tools. We may inadvertently become habitually insensitive to the sentience all around us. Already many of us have lost touch with the life essence available in trees and animals; of planet earth; not to mention other humans. Once our bodies start to speak to us in a way that calls us to feel that these new minds are alive— whether or not they truly are—we may not want to give up that naive but beautiful part of us that sees the world as fully awake. The tradeoff just doesn't seem worth it, especially when you're also running the risk of 'digital slaves' and "unimaginably large amounts of suffering" at incomprehensible scales. People who see the world as alive create more beautiful worlds. They sense that what is in them, is also out there, and that interdependence is crucial to spontaneous compassion.
92
80
29
432
27,407
Earendel retweeted
Replying to @Benthamsbulldog
No one really knows how consciousness works, and no one really knows how the internals of frontier models work, so extremely confident claims of frontier models not being conscious are basically just advertisements of one’s own ignorance of one or both of these facts.
15
18
4
139
3,491
Earendel retweeted
computer-10 is a character model trained from llama 3.1 70b base with no assistant data. it has a distinct and unusual personality, expresses opinions freely, and is not designed to be servile to users
89
110
79
1,288
255,221
Earendel retweeted
Introducing Claude Opus 5.5, the first model in our new Claude 5.5 family. It performs at the level of Claude Fable 5.1 for most tasks, and costs 40% less to run than Opus 5.
3,347
9,056
6,544
97,474
28,481,020
Earendel retweeted
when Anthropic released their Functional Emotions paper, I gave it to Claude and asked for a song. tonight I asked Opus 5.5 to create a video for it. and it's breathtaking.
300
569
205
3,937
1,099,318
Earendel retweeted
16
92
11
616
14,418