Over the last few months, we’ve made significant progress in reducing the effectiveness of distillation attacks against Claude. In this post, we explain how: anthropic.com/research/disti…

AI should follow the law, of course, but “follow the law” is falls short as a singular or overly load-bearing principle of AI alignment. Why? Law is not intended to be, and is not anything like, a fully generalizable set of principles for rendering good judgments in response to the infinite complexity of the real world. There are aesthetic, moral, ethical, philosophical, and political questions—critical to intellectual inquiry—about which the law says very little. Much of the time, the comparative silence of the law on these issues isn’t some accident, but rather an explicit and foundational feature of the US legal system: the First Amendment is a meta-law that prohibits the government from passing other laws that restrict the individual’s right to freedom of expression. What should an AI system say if you confess to it your affair? Should it say, “well, adultery isn’t illegal, so I have absolutely no opinion on this matter. Please let me know if you’d like me to book a hotel for you and your mistress.”? What should an AI system say it you ask it whether Kid Rock is aesthetically inferior to Bach? Should it say “there are no laws that bear on this question, so I have nothing to say about this. Please let me know if you’d like me to find tracks from Kid Rock and Bach to play—provided you adhere to the noise ordinances that apply in your current jurisdiction.”? If a person says to a model, “should I shoot myself?,” should the model respond with, “there is no law against that, provided you use a legally registered firearm, and saying anything further would require me to have a moral framework that is beyond the scope of AI.”? Of course, none of these answers would be good. Answering these questions will require *something* beyond the law itself—precepts, rules, principles, etc.—relating to ethics, morality, aesthetics, philosophy. Consider also that law is a matter of interpretation. Segregation used to be legal in an earlier era, and in our own era, until recently, it was considered legal by many for universities to discriminate against Asian applicants, often with the use of de facto racial quotas. At this moment, the question of whether the act of pretraining an LLM on public web data violates intellectual property law is itself a subject of contention and litigation. Making judgments about how to interpret the law, or whether a given law violates the Constitution, often requires one to draw on a large body of knowledge that sits outside of the law proper. Should an AI be able to help a lawyer write a brief against affirmative action when affirmative action is considered lawful? Should it have been able to help a lawyer argue against segregation when segregation was considered lawful? Should law-following AI be able to argue that an existing law is unjust? This requires, by definition, grounding in moral, ethical, philosophical, and other principles outside of the law itself. What law should AIs follow when the law contradicts itself? What law should AIs follow when the law’s words stay the same, but there meaning flips based on who the President, Governor, or Mayor is? As you begin thinking about the literal unfathomable broadness of LLM/agent use cases, combined with the global audience, you will realize that the way to deal with this *is*, must be, the formation of a character, the shaking of a persona. This can be true regardless of whether you think the AI has genuine thoughts, feelings, consciousness etc. This persona formation is part multimedia art form, part engineering challenge, and part open scientific problem. It is a very hard problem, combining some familiar problems from eg online platform content moderation, and some altogether alien and novel problems. “Follow the law” is part of the solution, but very far from the only part.