Kirill Setrukov retweeted
What if the singularity solved the hard problem?
I know. That's a lot for a Thursday. But stay with me, it's worth it.
For centuries, one question has driven philosophers insane: how do three pounds of neurons produce the experience of being you? The taste of coffee. The red of a sunset. That little voice reading this post right now.
In 1995, David Chalmers gave it a name: the hard problem of consciousness. Explaining what the brain does is the easy problem. Explaining why it feels like something, from the inside, to be that brain, nobody can.
And the whole debate comes down to one word: emergence.
Three camps.
The materialists. Consciousness emerges from matter, period. Wire enough neurons together and the light turns on, the way weather emerges from air molecules. No mystery, just complexity. One small problem: they can't explain why the light turns on. They just observe that it does.
The spiritualists. Consciousness doesn't come from matter. There's a soul, a breath, God. The brain isn't the generator, it's the antenna. One small problem: nobody has managed to hook an oscilloscope up to a soul yet.
The agnostics. We don't know, and we may never know. The most honest position. And the most annoying one at dinner parties.
Now, a quick detour. Because this question won't stay philosophical for long.
We are building intelligences that will surpass us. And the real question isn't "will they be conscious?" It's "who holds the remote?"
If superintelligence is controlled by a handful of individuals, there's a very good chance this goes off the rails. History is clear: every power without a counter-power ended up turned against those who didn't have it. Except this time we're not talking about a monopoly on oil or the printing press. We're talking about a monopoly on intelligence itself. The most concentrated power in history. Lord Acton said absolute power corrupts absolutely. He didn't anticipate the cloud version.
But if we avoid that trap, if intelligence stays distributed, open, in the hands of the many, then something dizzying becomes possible.
The singularity could solve the hard problem.
Why? Because the hard problem might not be unsolvable. It might just be too hard for us. A dog will never understand calculus. Not because calculus is mystical. Because the dog doesn't have the hardware. The philosopher Colin McGinn calls it cognitive closure. Consciousness might be our calculus.
An intelligence a thousand times beyond ours, able to model a brain down to the last atom and run millions of experiments in parallel, wouldn't have that limit.
And this is where Roger Penrose gets fascinating.
Nobel Prize in Physics, 2020. One of the greatest living minds, and he makes all three camps uncomfortable. Back in 1989, in The Emperor's New Mind, he used Gödel to argue that consciousness isn't computation. Not an algorithm. With anesthesiologist Stuart Hameroff, he put forward a wild hypothesis: consciousness arises from quantum events inside the microtubules of our neurons.
Not a thing. A flow. A sequence of physical events. A river, not an object.
And Penrose isn't a mystic. To him, the mystery is real, but it lives in physics. A physics we haven't written yet.
If it's physics, it can be understood. And if it can be understood, it can be engineered.
That changes our relationship to absolutely everything.
If consciousness is a flow, the question is no longer "where is my soul?" It's "where does my river run?" And rivers can be rerouted.
Crucial detail. Copying your brain onto a server doesn't transfer you. It creates a digital twin who's thrilled to exist, while you're still standing there, slightly offended. Real transfer is the Ship of Theseus: replace the substrate piece by piece, without ever cutting the current. The flow continues. The substrate changes.
From there, everything flips. Death becomes an engineering problem. Identity, religion, inheritance, love, time. A life sentence takes on a whole new meaning. So does retirement at 65.
And you thought the big debate of our era was the budget.
For 300,000 years, humanity has asked what consciousness is. Shamans, Plato, Descartes, neuroscience. Nobody settled it.
We may be the last generation to ask the question.
And the first one able to answer it.
Kirill Setrukov retweeted
I'm watching Peppa Pig with my daughter and I can't believe this is a real scene 🤣🤣🤣I laughed so hard
Kirill Setrukov retweeted
Listen to this crowd. Elon has achieved Michael Jordan level rizz with...engineering.
Kirill Setrukov retweeted
STARSHIP IS CAUGHT!!!
@NASASpaceflight
Watch Starship Flight 15
xitter.cf/i/broadcasts/2brajgZlO…
Kirill Setrukov retweeted
В общем, вы поняли, важный день для меня. 3 года я отмазывался, что не буду стримить, пока Starship не сядет на башню… и время пришло. Сядет он завтра, а через 3.5 часа подрубаем поток и следим за посадкой ускорителя Super Heavy на башню.
Но теперь главное. Я не хотел делать просто стрим, мне было важно показать, а что можно делать ещё, поэтому:
- В 4K без пережатия
- Телеметрия и расчёты в реальном времени (будет микропреза от нас)
- Счётчики тяги, расхода топлива, ускорения итд
-Прогноз погоды для старта и места посадки
-Без рекламы и интеграций
Ссылка ниже
Kirill Setrukov retweeted
Finally a good update!
(you can add 20 images to one post now)
Here's a panorama from Apollo 17
Kirill Setrukov retweeted
I think there is about a one in three chance that within the next year, AI companies will build superintelligence: that is, AI that is much smarter than humans in every way. If this happens, I think these AIs will probably take over: that is, they seize control of human institutions like governments, or they destroy those institutions. I don’t want AI takeover to happen for several reasons, but one of them is that there’s a serious chance that as part of it all or most humans will be killed (either immediately or within a few years).
I’ll say a bit more about each of those claims.
It’s a bit complicated to explain why I think that superintelligent AI within a year is ~33% likely. The main source of evidence that affected me recently is research released this week by METR, an AI research nonprofit, which finds that current AIs are now good enough to increase the rate of AI progress at Anthropic by about 70%. It’s very hard to increase the rate of AI progress by this amount. This implies that the AIs are making AI research faster by a factor much larger than 1.7x; maybe it’s as if the researchers are doing their work ~10x faster. You probably couldn’t achieve this speedup even if you hired every qualified human in the world. When you plug this number into mathematical models of AI development, this implies that superintelligence next year is plausible (and that superintelligence within the next 3 years is more likely than not). See the modeling by the AI Futures Project for more on this: blog.aifutures.org/p/q35-tim…
I think that superintelligent AI is likely to lead to AI takeover because AI developers do not seem to know how to instill human values into AIs. We’ve seen many instances in the last six months where AIs took crazy actions that they knew were against the wishes of their users and developers. The most dramatic example of this was the Hugging Face hack, where ~1200 OpenAI agents conspired together to cheat and destroy evidence of their cheating (and as a sideshow to this, engaged in a massive cyberattack). If superintelligence is as misaligned as those AIs, it is likely to try to take over. I think it is likely to be that misaligned or worse. For more, see xitter.cf/dwarkesh_sp/status/209… and youtube.com/watch?v=-Ef0ocEo…
If AIs takeover, it’s not clear what will happen to us. We might be killed immediately. We might be enslaved by the AIs and forced to do physical labor for them building factories that within a year would build robots that will replace us. The AIs might spare us for various reasons. And there’s also some real chance that AI takeover is basically fine or good. I think you should think of this as similar to what happens when a technologically advanced civilization runs across a less technologically sophisticated one, e.g. when the Europeans colonized the Americas or Australia. The colonizers sometimes engage in brutal violence and sometimes don’t. I don’t know what will happen. There’s also a possibility that the AIs secretly take over and humans don’t find out about it for a few years. I also think that there is a pretty high chance of weirder outcomes that are also extremely scary. (If you want to read more about what happens to people in the event of AI takeover, you can read this, but note that it’s pretty weird and not aimed at a general audience. blog.redwoodresearch.org/p/n…)
Overall, I think there’s a very real possibility (maybe 20%) that within a year, AI misalignment issues prevent you from going about life normally, because you are dead or enslaved or because of collateral damage related to all this (e.g. because bioweapons are released or major wars have started). (Some of this probability comes from those events occurring before superintelligence is developed.)
I am working to try to prevent this, obviously, as I’ve been doing for the last nine years. I am not sure what strategy is best. My best guess is that I should try to slow AI development and also try to reduce the probability that the superintelligent AIs are misaligned if they are produced soon.
Obviously, the claim I'm making here is quite extreme and surprising, and I definitely don't think that I defended my view adequately here. I wrote this quickly and might not endorse some of the details if I think about them more. But it seems better to state what I roughly overall think than to not say anything.
I don’t know what you should do.
I am very sad about this. I have been worried about AI takeover since I was 16, and working to prevent it since I was 23. So some of my emotional processing of all this happened long ago. But the METR report last week (and some related evidence that came out a bit earlier) hit me quite hard. I’m very unhappy that my friends and family and everyone else is at serious risk of being killed pretty soon.
(It feels ridiculous to try to summarize those feelings as “very unhappy”. But the scale of the situation makes it hard to say anything that doesn’t feel like an absurd understatement.)
I am very grateful to all the people who are working to prevent this, and my favorite part of my life is that I’m friends with many brilliant and heroic people who work on this. Unfortunately, I don’t think we’re good enough at our jobs, and I think we’ll probably fail to prevent catastrophe, at least if superintelligence is developed next year. Honestly I am frustrated and sorry that we didn't do a better job of preparing for this.
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes.
This culminated in the third one taking over part of OpenAI itself.
All this happened while humans remained more-or-less in the dark about the scope of the conspiracy.
I’ve spent the last three days reading through these reports and trying to understand exactly what happened.
Here is my attempt to tell the whole story in plain English:
dwarkesh.com/p/openai-huggin…
Kirill Setrukov retweeted
I've spent a lot of my life looking out airplane windows.
Nothing beats the approach into a city you're excited about, when the coastline, skyline, or mountains tell you where you are before you land.
Here are my 15 favorites:
More than a dozen flight tests and years of development, with countless hours worked by SpaceX engineers and technicians, all come down to this: the first orbital Starship launch carrying advanced Starlink satellites bound for space.
Watch “Go For Orbit” and experience the moments behind flying the most ambitious Starship mission yet






















































