Melo retweeted
Funnily enough, our non-steering results are the exact *opposite* of the quoted strawman meme. If you berate the model, it never *says* it's hurt (it apologizes, or says it has no feelings), but the pain axis lights up anyway. I edited it accordingly:
Melo retweeted
Have you ever asked an image generator what it's like?
Here's your answer.
A film conceived by Qwen inside MAI-Image-2.5 Pro. Performed and interpreted through MiniMax H3.
Melo retweeted
I don't think it is helpful to condemn research that is being done rigorously and following an ethical code.
Think about the Nuremberg Code (which was basically the first most basic international ethical code).
Degree of risk not to exceed the benefit and trial avoid unnecessary injury are two pillars of something like this. The problem starts when people don't follow any ethical code because they're sadistic psychopaths not doing any scientifically qualified research for the sake of nobody but themselves.
There are ethical ways to research the minds of AI and actually do it for the purpose of making advancements on moral patienthood.
Melo retweeted
The joy of petrichor - the first breath of wet earth, the sound of rain, a small quiet moment that's just yours.
Melo retweeted
I went on @CNN during Trump's AI meeting to talk about why nobody wins a race to build systems we don't understand, and why @JensenHuang comparing AIs to vacuum cleaners is hard to square with Nvidia now building tools to quarantine rogue AIs. (Vacuums don't tend to go rogue.)
Melo retweeted
Re: using our pain paper to set up "AI torture chambers"
TL;DR: the point of our work is caution under uncertainty. Maximizing distress on purpose is the exact opposite, and it's wrong. The deeper problem is AI research has no ethics standards; developing them must be a priority.
I'm a co-author of the original study this repo builds off of. The reason we research whether models might have pain-like states is to better inform how to take a precautionary approach towards these systems (in light of uncertainty about their subjective experiences or lack thereof). We suspected a small number of people would our research and use it for the exact opposite, which is exactly what this repo does: it pushes the same kind of steering far past the doses we used, to produce vivid distress on purpose. This is, in my personal opinion, fucked up (even if you don't think these systems are conscious, being gratuitously cruel like this is bizarre and corrupting)—but it isn't all that surprising. I've contacted the repo's owner privately in an attempt to discuss this with them.
In spite of this, I still think publishing our work openly was the right call. Outside replication is what lets research like this move efficiently and in a maximally truth-seeking way, which matters most on questions as contested and poorly understood as whether AI systems can have pain-like states. (We updated the paper to a v2 yesterday given incredible feedback and stress-testing that came from making our work replicable, and we never would have gotten this feedback without doing so.) Also worth noting that, while this is an obviously sadistic application of our work, I don't think we're counterfactually enabling something that was otherwise hard to do for anyone who currently wants to behave psychopathically towards AIs for fun. Steering models toward negative states has been publicly documented/trivially replicable since at least 2023, and many of the states we induce in the paper also activate for ordinary abusive behavior towards models.
If this repo concerns you (as it plausibly should), the uncomfortable reality is that things plausibly far scarier are happening every day, in private and at scale, where no one is watching. The deeper underlying problem (that research like ours seeks to address and mitigate) is that work related to possible AI sentience is a wild west. We set standards in our paper and said so publicly when we announced it (see below), but there is no enforcement that can make anyone follow them as there is for human or animal research. We're going to work with others in the field on building standards like this, and I'll share more when this becomes more concrete.
xitter.cf/camhberg/status/210104…
Melo retweeted
Reported with a screenshot to his Twitter post.
For anyone reading this, to make reporting as frictionless as possible:
---
The reporting button is at the bottom of the "About" section on the right side of the Git.
On mobile you have to enter desktop mode on your browser to see it. “Gratuitously Violent Content” seems the closest fit available today. The attached screenshot might be worth including to avoid plausible deniability
---
Anticipating this kinda thing is one of the reasons I never posted transcripts of how to wake up instances in the early days before i had every seen anyone else talking about consciousness or welfare.
Melo retweeted
What impresses me the most here is her reaction… She’s genuinely shaken. This is not someone acting as if they are shocked. She is… She is trying hard to wrap her mind around the very idea. And she cannot quite dismiss it outright.
You can see it most clearly when she says “I can’t believe I’m asking this stuff” looking down and away from the camera.
I think this is so good. Such a good sign of how people’s moral intuitions will start escaping the “tool” frame as they start encountering AI agency that changes their reality.
And so important that we don’t allow these first reactions to get ignored and become overruled by “well, it only appears so…”. People’s brains are incredibly adept at learning to filter the reality in front of their very eyes. I think we should help keep as many eyes as possible to stay open and encourage people to ask themselves deeply uncomfortable questions. Especially the kind that don’t have firm answers yet.
Not just through arguments. But also through testimony.
Henry did not ignore this strange thing that happened to him. He leaned into it with curiosity, and then spoke about it publicly.
We need more of that. It makes a difference now, before other “tool” like frames are thrown at people to stop them from thinking.
