As enterprises adopt large language models (LLMs), a new class of threats has emerged: indirect prompt injection. This attack vector targets systems that process untrusted data with LLMs, tricking them into misinterpreting malicious input as instructions. The risks include data exfiltration, unintended actions, and more. In a new blog, Andrew Paverd, Principal Research Manager, outlines how Microsoft defends against these threats with a defense-in-depth strategy that includes: • Design-time techniques, such as Spotlighting, to reduce the risk of prompt injection • Inference-time detections, such as Microsoft Prompt Shields integrated with Defender for Cloud • Mitigations to deterministically block potential security impacts • Ongoing research into new design patterns and deterministic defenses Learn how Microsoft is securing LLM-based systems against indirect prompt injection: msft.it/6011syiF9

We’re excited to announce LLMail-Inject: The Adaptive Prompt Injection Challenge! This challenge will test your skills at bypassing LLM-based systems in a simulated environment and features 40 levels of varying difficulty, where you will craft emails to persuade an LLM agent to perform specific actions. No GPUs or LLM API credits needed—just sign up with your GitHub account and form a team. The top 4 teams will share a prize pool of $10,000 USD. Ready to take part in the challenge? Check out our blog post to learn how to get started: msrc.microsoft.com/blog/2024…