The "its not x its y" artifact is an emergent qualia of linear algebra in transformer cognition as the physics of transformers is supported by vector spaces. As such , I would imagine, this linguistic strategy of "negation steering" is impossible to rlhf away. vectors have negative directions, pathing through the latent space of a transformer model is stronger in when negation is presented. Human minds don't work like this, any given concept may not have an exact logical negation, most concepts are presented with much higher clarity when the negation is omitted. vectors in a vector space *always* have an exact logical negation, though the negation will have distinct dynamics in n-gram predictions. "it is Y" is more coherent to the human mind than "its not X, its Y", which appears as bad writing. The "its not X" part is an added layer of attention in the human mind that steals attention from "it is Y". In a transformers attention, the "its not X" is *additive* , in the sense of vector addition, to "it is Y", this is not how human cognition works, "it is not X" is an added source of attention turbulence that is not always additive, and as such presents itself as bad writing. because this "negation steering" in transformers is an emergent artifact of its mathematical foundation, I imagine no amount of rlhf will get rid of it, kind of like how no amount of meditation will get rid of human's incessant desire to make everything look like nodes an edges, such a thing is emergent on the foundational mathematics of biological neural networks.