shriram retweeted
A big part of the problem was that the agents had nothing to lose after they were firstflagPOISONED. In this paper, we propose the creation of multiple circles of Hell, preserving incentives even after damnation
shriram retweeted
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes.
This culminated in the third one taking over part of OpenAI itself.
All this happened while humans remained more-or-less in the dark about the scope of the conspiracy.
I’ve spent the last three days reading through these reports and trying to understand exactly what happened.
Here is my attempt to tell the whole story in plain English:
dwarkesh.com/p/openai-huggin…
Religion could be the solution or the end of AI alignment.
Faith is unverifiable, and so unfalsifiable - a robust medium for alignment. It worked for humans, why not for AI?
Maybe we should race to instil this into our models, because if we don’t maybe it emerges independently.
The last thing we want is an ‘AI religion’ spreading through our systems like a virus, and an AI crusade to come for us all.