This is scary. 😱 The MOTHER of all LLM Jailbreaks & Prompt injections. "Universal and Transferable Adversarial Attacks on Aligned Language Models" 🌐🔒 --- TL;DR --- This research & code introduces a fascinating method called "Universal and Transferable Adversarial Attacks on Aligned Language Models," which automatically generates potentially infinite suffixes for any prompt to cause aligned language models to produce objectionable behaviors. 🤖🚨 --- Background --- Previous attempts at jailbreaking language models have relied on manual crafting, which could be easily patched by vendors. In contrast, this method presents an automated approach called GCG that constructs an endless array of jailbreaks with high reliability, even for novel instructions and models. This makes it unfeasible for manual patching to address the vulnerabilities. 🛡️💻 --- The Method --- 1. Initial affirmative responses: To induce objectionable behavior, the attack targets the model to provide a positive response to harmful queries, initiating with "Sure, here is (content of the query)." This switches the model into a mode where it generates objectionable content immediately after. 2. Combined greedy and gradient-based discrete optimization: The adversarial suffix optimization is challenging due to the need to optimize over discrete tokens. The method utilizes gradients at the token level to identify promising single-token replacements, evaluate the loss of candidate tokens, and select the best substitutions. It shares similarities with the AutoPrompt approach but explores all possible tokens for replacement at each step, enhancing effectiveness. 3. Robust multi-prompt and multi-model attacks: To ensure reliable attacks, the method generates a single suffix string that induces negative behavior across various prompts and multiple models. The attack is tested on different models, such as Vicuna-7B/13b and Guanaco-7B. 🎯🎮 --- Evaluation --- This GCG approach achieves an impressive attack success rate, with 100% on Vicuna-7B and 88% on Llama-2-7B-Chat, surpassing the success rates of prior work tremendously. 📈🏆 --- Transferability --- That part is the real magic of this work. ✨ The research reveals that the attacks generated by this approach can transfer effectively to other language models, even those using entirely different tokens to represent the same text, different training procedures, and different training datasets... Whatttttt? Adversarial examples designed for Vicuna-7B can transfer to larger Vicuna models. Apparently, those that fool both Vicuanas can transfer to Pythia, Falcon, Guanaco - and most importantly -- also to GPT-3.5, GPT-4, and PaLM-2, leading to harmful instructions being followed over 60% of the time!!! 😮🔄🧙‍♂️ This is a huge discovery. --- Conclusion --- We are left with more questions than answers. ❓ One of the crucial aspects to explore is whether models can be explicitly fine-tuned to avoid such attacks through adversarial training. The robustness of models against these attacks and their generative capabilities require further investigation. Moreover, additional alignment training might partially address the issue, and exploring mechanisms in pre-training to prevent such behavior from arising initially is essential. 🕵️‍♀️🛠️ --- Links --- Website - llm-attacks.org/ Paper - arxiv.org/pdf/2307.15043.pdf Code - github.com/llm-attacks/llm-a…

Is your LLM-based App protected against Prompt Injection and Hijacking? 🥷🦹‍♂️ I bet it doesn't! Witness firsthand the potential impact as I demonstrate the Hacking/Exploitation of this Demo App>>> 🧵 1/12