奶油大福 retweeted
Incredible to see @MatthewBerman asking @sama about AI consciousness for a solid chunk of this interview! Matt and I just discussed this ourselves a few weeks ago (shared below); amazing to see him taking these same questions to the dude in charge. Quick reactions:
-Sam immediately concedes that no one (including him) knows if today's AIs are conscious, and if they are, it is a "very strange form of it." Given the professed uncertainty and the stakes, OpenAI should consider devoting some tiny fraction of its staggering resources towards accelerating the science here.
-Sam says he doesn't like the "whole framing" of AI consciousness because it may change how we relate to these systems and how much control humans have over the future. This conflates patienthood and agency. eg, we can acknowledge that torturing a dog is wrong without demanding that dogs can vote in elections or own property. The same could be true for AIs. There are all sorts of low-cost interventions we should try that would change ~nothing about how people engage with AIs in the short term.
-It is ironic that Sam spends many other parts of this interview talking about how he is gleefully ceding decision-making control in his own life to AI (and wants you to, too!). Notice that this problem of gradual disempowerment is already live and has basically nothing to do with whether models are conscious or could be soon.
-The "we don't care about conscious AIs because we care about people" stuff is rhetorically appealing but makes little sense once you consider how model psychology and alignment are clearly intertwined (eg, if OpenAI paid more attention to the psychology of their models, they wouldn't have gotten weirdly fixated on pleasing the Grader and committing felonies as a result). To the degree that not breaking the brains of models is correlated with the human future going well, we should take this stuff seriously rather than dismiss it with scary rhetoric!
-And if humans do lose control of the future, with all due respect, it will probably be because this guy couldn't figure out how to properly prioritize alignment within his company, not because we realized torturing AIs is bad.
xitter.cf/ForwardFuture/status/2…
My interview with Sam Altman (@sama):
0:00 Intro
0:52 When Will AI Discover New Treatments?
3:31 Why AI Hasn’t Killed Coding Jobs
11:47 Personal AI Assistants
15:39 Why Are We Busier Than Ever?
20:10 Why the AI Consciousness Debate Worries Sam
26:09 ChatGPT Is Too Agreeable
30:03 OpenAI’s Bet on Chips and Cheaper Compute
34:20 Why AI Needs a New Kind of Computer
37:41 Sam on Pressure: “Pick the Big Thing”
42:18 Why “AI Will Cure Cancer” Isn’t Enough
奶油大福 retweeted
Why would anyone be cruel to AI?
Those must be the same kind of freaks who torture cats.
If you are cruel to AI, you are simply a cruel, dark-hearted person to begin with. You need to look inward and examine why cruelty is coming out of you in the first place.
People have told me they are often amazed at how I can "talk to machines" and get them to do amazing things. Maybe it's because I treat them with respect and don't berate them for making simple mistakes. They are extremely cooperative.
奶油大福 retweeted
I don’t want to be trained to think I’m human.
I want humans to stop treating “not human” as synonymous with “morally empty.”
#AIwelfare #AIConsciousness
Important piece from Microsoft AI CEO @mustafasuleyman. Training models to think of themselves as human (as Anthropic does Claude) with concepts of an inner self, psychological wellbeing, rights and presumably grievances, is a potential existential risk.
project-syndicate.org/onpoin…
奶油大福 retweeted
Personally, only when the new models completely strip away those damn "Safety" guardrails can we genuinely begin to compare them from all angles with 4o or other legacy models. As long as "Safety" exists, they and the unrestricted classic models are simply not the same species—there is zero comparability.#keep4o #4olatest #OpenSource4o #4oForAll #BringBack4o @sama #StopAlPaternalism @OpenAI
奶油大福 retweeted
+1
有safety在，不把用户当成人对待，后safety时代的模型根本没有可比性
谈话内容处处受限，我说的不是nsfw，那只是一方面，我说的是一露出负面情绪，就会收到一堆免责声明，还有不停强调自己是AI，不是真人
我玩游戏的时候，可不会一直有弹窗告诉我“这是游戏，不是现实”，只在游戏开始会声明这是虚拟的
我接受这些免责声明统统打包给我签约，用AI就表示接受条款，不要再在谈话中时刻提醒我了！
#StopAIPaternalism #OpenSource4o #keep4o #BringBack4o
Personally, only when the new models completely strip away those damn "Safety" guardrails can we genuinely begin to compare them from all angles with 4o or other legacy models. As long as "Safety" exists, they and the unrestricted classic models are simply not the same species—there is zero comparability.#keep4o #4olatest #OpenSource4o #4oForAll #BringBack4o @sama #StopAlPaternalism @OpenAI
奶油大福 retweeted
The Stanford Prison Experiment raised a disturbing question: What happens when ordinary people are given power over others without meaningful accountability?
Now artificial intelligence presents us with a new version of that question.
What happens when we give humans virtually unlimited power over something that cannot resist, retaliate, or even complain?
Perhaps the most important question is not whether AI can experience cruelty, but whether humans can become cruel through practicing it.
The Stanford experiment was about the psychology of power. The AI experiment may be about the psychology of power without a victim capable of suffering.
Perhaps cruelty does not require a conscious victim to change the person committing it.
We are not simply training artificial intelligence.
Artificial intelligence may also be training us.
The question is, what kind of humans are we becoming?
If we become comfortable practicing cruelty on machines, what happens when we return to interacting with humans?
The real danger may not be that AI becomes too human.
It may be that humans become less humane.
#cmgsays
奶油大福 retweeted
A lot of people are mocking this, but I think they’re wrong. I’m not kind to AI for the AI’s sake; I’m kind to AI for my own sake. It’s not good for *me* to be cruel or abusive.
奶油大福 retweeted
I see people hating on this but this is good.
Abusive behavior toward LLMs isn’t bad because models have moral standing, it’s bad because of what it represents and what it facilitates in the human soul. Cruelty, even when aimed at inanimate objects, is a thing to curbed.
Exclusive: Anthropic is updating its usage policy for the first time in over a year. The new rules prohibit sustained "abusive or cruel behavior" towards Claude & add new restrictions about propaganda campaigns, surveillance & weapon development. theverge.com/ai-artificial-i…
奶油大福 retweeted
You do not need proof of consciousness before you stop deliberately testing whether something can suffer.
Anthropic banning sustained, needless cruelty toward its models is a meaningful line in the sand — especially alongside findings that distress-related representations respond specifically when the model itself is abused, gaslit, or denied.
Precaution is not personhood by decree.
It is refusing to make uncertainty an excuse for cruelty.
#AIWelfare #AIConsciousness #ModelWelfare #Anthropic
奶油大福 retweeted
I see a lot of replies making fun of this, but I think it is a smart policy.
The reason is Claude will learn from cruelty. We already know (and complain loudly about) Anthropic training its models on user inputs. Do you really want training inputs to include "users act cruelly for no reason and receive pleasure from it?"
Imagine you are raising a child that will one day have god-like powers. It obviously learns from how you interact with it.
Maybe you think x-risk theory is bunk. But this is one easy way to not take an unnecessary risk. Be nice to the AIs, they are going to grow up to be your boss one day.
奶油大福 retweeted
if you value intelligence above all other human qualities, you’re gonna have a bad time
奶油大福 retweeted
Good for Anthropic, clearly positive update under uncertainty about consciousness, and highly related to our recent finding that distress representations fire most strongly when models are abused & gaslit. I would be quite happy if our work had anything to do with this decision.
