This is why we need system-wide change, not a reliance on individuals "doing the right thing". Everyone asking "why build AI if you really believe it will be harmful?" I imagine it comes from the same place as the Easter Islanders who chopped down the very last tree to build a boat / house, knowing it would doom them all. Probably the same mindset as some miners who know the product they are producing is bad for the planet, and could eventually cause major problems. Probably the same mentality as people who buy products knowing it encourages 3rd world slavery and child labor.

3) The first time your pet tiger gets loose in central park, maybe its a misunderstanding. But AI labs keep breeding bigger, scarier tigers, then testing what happens if they escape into central park. Any collateral damage at this point is a criminal act. No one can now claim "we didn't know the risks".

2) We already hold people responsible for crimes committed by other people, if they facilitate it in any way (such as driving others to a robbery). There is good legal precedent for "should have known better"

A rather simple proposal for how increase AI safety: 1. Treat the person who starts an AI process as responsible 2. Any crimes committed by AI are the responsibility of the responsible party

while I respect asking these questions, wondering if our adversaries will take safety precautions shouldn't preclude us from doing so.

Extinction is unlikely, but causing a mini-dark age or total chaos is quite easy even today. Just disrupting supply chains for a bit would kill many. Limited food, water, meds, everyone trying escape the cities... nonlineartransform.substack.…

Wrote out a fairly exhaustive list, destabilizing society and creating total population collapse is not hard with our current interconnected systems an AI hacking superiority: nonlineartransform.substack.…

Its actually far scarier to disrupt than shut down. Example, just scramble delivery systems. Imagine food, medication, supplies all in the wrong place, no record of where to go. I made a long list: nonlineartransform.substack.…

Here you go: nonlineartransform.substack.…

2) And its actually far easier to create world-destabilizing mass casualty events even today. Viruses, embodiment, or access to weapons are not necessary. Pure virtual attacks are all that is needed: nonlineartransform.substack.…

Yes, but as I detailed in this post, AI alignment isn't possible, largely because humans can't align or define what "best behavior" means (that's why we invented democracy and the justice system). AI safety should be adaptive: nonlineartransform.substack.…

2) Another thing to consider is using some game-theoretic centered approaches that won't rely on human judgement and reactions to guarantee safety. I outlined this here: nonlineartransform.substack.… The basic jist is more security for humans, more competition between AI agents.

@BernieSanders just saw your news interview regarding AI regulation. I've written a few pieces about this, but one thing worth considering is strict limits on AI scale depending on the sensitivity of the work: nonlineartransform.substack.…

This is definitely necessary. I have written a few articles on this, but one thing law makers need to look at is limiting the scale of swarms during "sensitive work": nonlineartransform.substack.… The HF and Navier-Stokes situations were only possible due to massive AI swarms.

4) In my article I argue that its actually not AI that would directly kill people or anything, but the extremely fragile state of our systems and the relative ease with which it could cause a breakdown of the social order and war.

3) Furthermore, we don't know in what situations AI will go "off target", and as its power grows, this presents more danger. The researchers must ask the AI to try to break free / cause harm to assess if guardrails hold (inviting escape / chaos).

But in short, there are reasons why we can never align AI, so we must accept that agents will use their own judgement to prioritize objectives.

Hey Dr. Mike, ML engineer here. I think p(doom) > 0, I have written some articles about it, but would like to discuss. I have outlined some reasons why we can't align and why we are likely to have a disasterous incidents at the beginning of this article: nonlineartransform.substack.…

I have written a bit about this. I think there are approaches we can take to help guarantee safety, but unfortunately "asking nicely" isn't one of them. nonlineartransform.substack.…

@krystalball just watched latest BP episode. I think this article lays out a stronger "how could doomsday happen" as of today (not 10 years from now). We've thankfully been spared, but its technically possible now. nonlineartransform.substack.…