Kunal

A lot of otherwise smart people on Twitter seem 100% convinced AI risks are all fake and stupid and part of some marketing ploy. What is surprising is that some of these people seemingly also believe that AI’s positive uses are on an incredible trajectory of increasing capability with no end in sight. VC Twitter is particularly infected by this pattern. It’s not really coherent. Most positive use cases for AI have a corresponding “dark version”. If you are super human at coding, you are also super human at hacking. If you are superhuman at structural engineering you are likely superhuman at finding structural flaws to knock buildings down. If you are superhuman at designing drugs, you are superhuman at designing novel undetectable poisons. If you can cure viruses, you can create them. Some of these “dark versions” are not so bad, and some are actually pretty scary. Either way these are real societal and technical problems that need to be solved to get the good stuff and avoid the bad stuff. We’re experiencing the first of these with coding and computer security which is the most advanced, but that won’t be the last. I think we’ll be able to solve these problems, but they aren’t solved yet and if you believe in continued AI progress they are surely coming. But “bad people using AI” is not the only problem. Uncontrolled AI autonomously doing bad things, despite sounding kind of nutty, is also something we should be concerned about. AI “killing us all” is not the most likely outcome, but the chance of a major civilization-wide catastrophe doesn’t have to be very high for it to be a concern. Again, this is only a problem if capabilities advance to a point that AI can do really crazy things on their own, which hasn’t happened yet, but I think the Hugging Face incident is a good example of the outline of how things can go wrong when capabilities outpace alignment. We should be glad that the only available bad thing right now is hacking, which isn’t all that bad. It’s clear that as you get to superhuman capabilities you need a level of alignment and control that is correspondingly superhuman. Humans have plenty of misalignment problems themselves (serial killers, mass shooters, tyrants, etc), but it’s a manageable problem because most humans can’t do that much damage and we’ve developed systems to prevent dangerous humans from getting too much power. Talking about these issues is just common sense. It’s not a sign of some kind of neuroticism or pessimism. These are just hard problems that it’s very important to solve for AI to have a positive impact. I’m pretty confident we will solve them. But we haven’t solved them yet, and to my mind we are clearly on a trajectory of rapidly increasing capabilities which means this is important. Mocking people who are worried about this or talking about it, without anything substantive to say about how we can be sure these problems won’t arise, is not really a very helpful contribution.