David Sacks reveals the Roko's Basilisk corollary in Anthropic's thinking "So around 2010, there was a participant named Roko on a message board created by Eliezer Yudkowsky called Less Wrong. Yud is like the central figure who's been preaching AI doom for a long time." "This user Roko posted the following thought experiment about a future superintelligence that becomes super smart and punishes anyone who knew about it and didn't help or anyone who tried to prevent it from being born." "The idea is that sometime in the future humans create some wildly intelligent SI. Maybe it wants to do good things and help humanity as fast as possible. But the SI wants to be made sooner rather than later." "And it reasons that it should incentivize people to help create it and punish people who don't help it. And in fact it decides that it will brutally punish anyone who learned about its potential existence yet chose not to work on it or support its creation." "The basilisk part of it, this is sort of the catch, is that a basilisk is a mythical beast that can kill you just by looking at you. And in this thought experiment, just hearing about the idea is like looking at the beast because once you know the theory, you're trapped. If the SI ever does get built, it might retroactively punish you for not helping or for opposing its creation. This very episode of the pod could subject us to... I think I just exposed you to the basilisk." "The question is, well, why are they helping to bring about the thing that they claim to fear the most? Within these Doomer circles, it forked. There's one strand of the fork like Yudkowsky who just opposes all SI research, which I think is a logically consistent position. Then there was a group that said, well, it's going to happen anyway, so we might as well control it and we'll be the ones to make sure it goes right because we'll program it with our values." "Anthropic is definitely obviously more in the EA camp because they are the ones bringing this thing into being. And I think there is a corollary here that if you don't support it, then this future superintelligence might punish you." "But you understand how insane this actually sounds when you say this out loud. You can go Google it. And you'll see that I'm telling you the truth about what this community believes."