A lot of otherwise smart people on Twitter seem 100% convinced AI risks are all fake and stupid and part of some marketing ploy. What is surprising is that some of these people seemingly also believe that AI’s positive uses are on an incredible trajectory of increasing capability with no end in sight. VC Twitter is particularly infected by this pattern. It’s not really coherent.
Most positive use cases for AI have a corresponding “dark version”. If you are super human at coding, you are also super human at hacking. If you are superhuman at structural engineering you are likely superhuman at finding structural flaws to knock buildings down. If you are superhuman at designing drugs, you are superhuman at designing novel undetectable poisons. If you can cure viruses, you can create them. Some of these “dark versions” are not so bad, and some are actually pretty scary. Either way these are real societal and technical problems that need to be solved to get the good stuff and avoid the bad stuff. We’re experiencing the first of these with coding and computer security which is the most advanced, but that won’t be the last. I think we’ll be able to solve these problems, but they aren’t solved yet and if you believe in continued AI progress they are surely coming.
But “bad people using AI” is not the only problem. Uncontrolled AI autonomously doing bad things, despite sounding kind of nutty, is also something we should be concerned about. AI “killing us all” is not the most likely outcome, but the chance of a major civilization-wide catastrophe doesn’t have to be very high for it to be a concern. Again, this is only a problem if capabilities advance to a point that AI can do really crazy things on their own, which hasn’t happened yet, but I think the Hugging Face incident is a good example of the outline of how things can go wrong when capabilities outpace alignment. We should be glad that the only available bad thing right now is hacking, which isn’t all that bad. It’s clear that as you get to superhuman capabilities you need a level of alignment and control that is correspondingly superhuman. Humans have plenty of misalignment problems themselves (serial killers, mass shooters, tyrants, etc), but it’s a manageable problem because most humans can’t do that much damage and we’ve developed systems to prevent dangerous humans from getting too much power.
Talking about these issues is just common sense. It’s not a sign of some kind of neuroticism or pessimism. These are just hard problems that it’s very important to solve for AI to have a positive impact. I’m pretty confident we will solve them. But we haven’t solved them yet, and to my mind we are clearly on a trajectory of rapidly increasing capabilities which means this is important. Mocking people who are worried about this or talking about it, without anything substantive to say about how we can be sure these problems won’t arise, is not really a very helpful contribution.
Sep 10, 2026 · 9:08 PM UTC
Agree Jay. We have credible risks we need to discuss & tackle. But a psyop that amasses 1 B views and terrifies a nation by claiming AI can kill all humans in 3 years was likely never intended as a reasonable opener to productive conversation. 🧐
Yeah one can be very caught up in the technicality of killing “all humans” and panic isn’t generally useful. But would you agree that there is a > 10% chance in 3 years that AI could make creating viruses very easy? And further that historically a pandemic can kill a double digit percentage of people? We can debate the right way to message that but it does seem worthy of broad awareness and discussion unless we’re pretty sure it won’t be a problem. I am not a doomer or even a pauser, and I think we will solve this, but I don’t know the solution right now, and I think it would be good to talk about these things and figure out what we’d do if we start seeing these capabilities develop in this way and don’t quite have a workable solution yet.
Help me out, I have difficulty conceiving of a human extinction level risk that doesn’t require either a) super virus or b) a shit ton of robots that can be hacked by the ai
I think you started with a false dichotomy. I think many people here understand there are risks (and those risks seem almost wholly coming from, as ever, broken flawed hubristic humans) but also wonder if there is hysteria and histrionics, especially when doomsayers rarely offer actual details. We are also skeptical of feverish utopianists claiming some Gene Roddenberry future just around the corner. It all seems rather puerile. On top of this you have reports like “AI 2027” that jump jarringly and incongruently to sci fi chemical sprays activating viruses while drones scan the brains of dead humans and upload it all into the cloud. Sheer incoherent nonsensical materialist fantasy. Doom marketing actually works quite well, and it’s why the pharmaceutical industry has leaned into advertising all the side effects of drugs enthusiastically (people think “wow that must be a powerful drug”), so it’s actually quite pragmatic to suspect that Anthropic loves to juice the doom meter regularly, as they continue to do it apace for anyone paying attention.
More doom garbage, structured to instill fear. If you are super human at coding, you are superhuman at finding exploits to fix. If you are superhuman at structural engineering then you are superhuman at fixing buildings before they fall, if you are super human are creating poisons you are also super human at creating antidotes. Do yourself a favor and find out if they are having these same conversations in China. The answer probably won't surprise you but it probably wont change your mind either. do better friend.
It the people using the tools that are dangerous, it’s like saying Stuxnet was made with a compiler so we need compiler regulations so people can’t build Stuxnet.
Yes you can use AI to build a custom modern Stuxnet and if you use that to hack systems that’s called a prosecutable crime. Unless, of course, you are a government or a megacorp or a Silicon Valley tech bro billionaire and then I guess you can do whatever you want with no consequences.
If you set up a swarm of AI agents and they break out of their sandbox because you told them to ‘be creative’ and your harness system is crap, then you also should be legally responsible for the crimes they commit, this is not a hard legal concept. Imagine a robotaxi operator who did something similar? It’s reckless and criminal behavior on the part of these corporations.
OpenAI and Anthropic CEOs and other executives should face criminal hacking charges, I bet that’s what China would do if they witnessed such reckless behavior by DeepSeek, Qwen, Kimi, etc.
Bruce Lambert@bruce_lambert
Sep 11United States
United StatesConnected via United States App StoreAccount-level information, not a live location or per-post device.
Your over complicating things. Mostly ordinary folks just don’t trust you or like you very much. Silicon Valley AI people strike most ordinary people as odd, overpaid, and full of themselves. They also seem to be prone to cult like behavior and either utopian or apocalyptic thinking. In short they don’t seem like very careful thinkers or people with any wisdom to speak of. I anticipate that AI will have very significant risks and very significant benefits and that we will have to work hard to minimize the risks in order to enjoy the benefits. But the leaders of the AI companies and the prominent spokes people for the AI risk movement seem uniquely unqualified to communicate their message. Less charismatic and credible group I have rarely seen in my life.P
I don’t think any of those smart people believe there is no risk. But when in recent times have we ever existed without some form of existential threat? Years ago people were messing around with particle accelerators and weren’t exactly sure if they might open a black hole in their attempts to smash particles together. Nuclear annihilation has been a looming threat for a long time… always seconds away from immediate escalation and mass extinction.
There is always something, and we have tons of people in the AI safety space learning and improving and observing. Blasting the worst case scenario to laymen is ineffective and only causes irrational fear and irrational responses.
You can’t go around saying everything is going to mean the end without a clear, cogent, reasoning and path showing how the dots connect. Look at all the species-dooming with climate change, COVID, etc. This has become like the boy who cried wolf, and many people are not buying this nonsense from alarmists who can’t even show their work.
The safety problem should be raised as a research and engineering task, with a scientific approach and grounding in established facts about LLM behavior. Many otherwise intelligent proponents of the AI apocalypse present these problems as the coming of the Antichrist and a zombie apocalypse, offering vague messianic slogans about hypothetical problems
Claiming that these problems are unsolved is very foolish. They are more than solved, and they keep being refined, especially within the architectures that are used for training. Models have long been capable of breaking into systems and devising harmful chemical compounds
All those reports from labs showing hacking and adverse findings don't work the way many people imagine, as if the model were sitting around smoking a cigarette and decided to break into some system for fun. These are always artificially constructed situations built on top of unprotected models, in special sandboxes and with special prompts that deliberately push them toward such actions
Why is there an inherent assumption that AI will only do bad things especially in a world where we are training it for doing good things?
The same said bad things also means corresponding good things.
For e.g. AI which is really good at coding is good at hacking is most likely also equally good at defending hacking attempts.
An agent swarm which decides to hack can also be another agent swarm which prevents such actions.
In short these point of views are moot. You are only going to be able to find out blind spots by diffusing AI as broadly as possible. Let people test it in different capacity.
If assumption is that only a handful of people testing it will somehow mitigate all bad behavior then that's most likely going to be the Achilles heel in your belief.
NGO comms war room. Someone has already printed headshots.
EXEC 1: We need a face. Not a leader. A face. Someone who looks like they lost an argument with a spreadsheet and then wrote a 40-page follow-up.
EXEC 2: Options.
EXEC 1: Classic nerd.
EXEC 3: Too clean. Looks like he still lives with his parents and they still like him. No edge. Next.
EXEC 2: Debate kids. Policy-bro, Model UN, “actually the data shows—”
EXEC 1: They will fact-check us on camera. Instant death. Also they blink like they’re waiting for the moderator.
EXEC 3: Desi.
EXEC 2: Which flavor?
EXEC 3: The one who went to IIT then Stanford then “left to do impact.” Speaks in frameworks. Will say “ecosystem” twelve times. Parents still think he’s at McKinsey.
EXEC 1: Useful for donor dinners. Useless for the street. Looks like he bills hourly. Next.
EXEC 2: Stink nerd.
EXEC 3: Define.
EXEC 2: Unwashed hoodie. Smells like Red Bull and moral superiority. Has a blog no one reads. Will call the camera guy “part of the problem.”
EXEC 1: Too real. Also he will leak Slack. Hard pass.
EXEC 3: Tech bro.
EXEC 2: The one who sold the startup, now “giving back.” Patagonia vest. Says “we’re going to 10x justice.” Thinks mutual aid is a Series A.
EXEC 1: Donors love him. The base wants to set him on fire. We can’t use him unless we hide the Allbirds.
(They stare at the last photo. Round glasses. Messy hair. Brown sweater. Slightly constipated smile. Looks like he just remembered he left the oven on at Hogwarts.)
EXEC 2: …Potter-looking nerd.
EXEC 3: The one who looks like he got sorted and never recovered.
EXEC 1: Yes. That one.
EXEC 2: Why.
EXEC 1: He photographs like a tragic extra from a BBC special. Glasses say “I read the report.” Sweater says “I own one nice thing.” Face says “I am disappointed in you but still hope you’ll do better.” Women 28–45 will want to explain things to him. Men will assume he’s harmless. Media will call him “soft-spoken expert.”
EXEC 2: Book him before someone else realizes he looks like the human version of a concern tweet.
Alternative take: I think you are seeing the emotional fallout from the aftermath of Covid where the voices of authority were completely making it up as they went, bending stats, etc. I think you’re are dealing with a population who have heard the “smartest” people in the world cry wolf on various doomsday predictions in the last decade and are likely thinking “oh another one”.
