
Founder @GenAISecLab. Author of @pentestbox, @appsecwiki. Occasional Bug Bounty Participant. Railfan. Defence Enthusiast. ISTP
India
Joined January 2013
- Tweets801
- Following562
- Followers3K
- Likes2.3K
Ever since the AI/LLM wave hit, I've been keeping a scrappy little personal list โ the payloads and techniques I actually reach for when testing AI apps. My own private cheat sheet, basically.
And lately, so many of us keep hearing the same question from clients: "Can you security-test the AI part too?" Honestly? Most folks I've talked to didn't have a solid methodology for it yet. No judgment โ this space is moving at light speed, and we've all been figuring it out as we go.
So today I'm genuinely excited to finally share mine: the AI Security Testing Cheat Sheet, part of the @genaiseclab project.
Here's what's inside:
๐ 150+ practical techniques across 20 sections โ all mapped to the OWASP LLM Top 10 2026
๐ Prompt injection, indirect & multimodal injection, sensitive data disclosure, tool and agent abuse, MCP attacks
๐ Real example payloads for every technique
๐ A tracker you fill in as you test, plus findings export and reporting templates
And the best part โ it's truly free. No signup, no credit card, no gate. Just grab it and go. โค๏ธ
This is a living project, too. If there's a technique you think I've missed, I'd genuinely love to hear it โ drop a comment or DM me and I'll fold it in. This thing gets better when we build it together.
๐ genaisecuritylab.com/cheatshโฆ
If it saves you even a few hours on your next assessment, that honestly makes my week. And feedback โ good or bad is welcome, so tell me what you think! ๐
I've been working on making practical AI security education more accessible to penetration testers and security professionals through GenAI Security Lab
As AI adoption grows, so does the need for security professionals who can identify and understand vulnerabilities in LLM applications, AI agents, and MCP integrations.
To make it easier for more people to get started, we're running a Black Friday sale with 40% OFF!
What's included in the sale?
โข Yearly Individual Plan โ 40% off the first year.
โข 4-Month Pass โ 40% off with a one-time payment and no renewal.
What you'll get with GenAI Security Lab:
โข 200+ hands-on labs covering OWASP LLM, Agentic AI, and MCP Top 10s.
โข Realistic AI applications, including chatbots, RAG systems, file uploads, and agents with tools.
โข No API keys required.
โข Isolated cloud containers for labs running full applications.
โข Structured learning paths, written curriculum, and concept videos.
โข 25% off GSCP and GSCE certification exams for yearly members.
Not ready to subscribe? You can also explore 37 free labs, without a credit card.
๐
The offer is valid until 30 November 2026, 23:59 UTC.
๐ Check the discounted pricing for your country: genaisecuritylab.com/pricing
One small request: If you know someone interested in AI security, LLM penetration testing, or expanding their AppSec skills, please share or reshare this post with your network. It would really help us reach more people who could benefit from the platform.
Thanks for your support!
#CyberSecurity #AISecurity #LLMSecurity #Pentesting #AppSec #BlackFriday
๐ Cybersecurity Awareness Month Giveaway!
Get 1 month of FREE access to 200+ AI security labs on GenAI Security Lab! ๐
โ
LLM & prompt injection
โ
RAG security
โ
AI agents & MCP
โ
Browser-based labs with GPU-hosted LLMs. No setup required!
๐ How to qualify:
Follow @GenAISecLab
Like & repost
Sign up by Oct 31: genaisecuritylab.com
DM me your signup email
#CybersecurityAwarenessMonth #AISecurity #AppSec #Pentesting #LLMSecurity #InfoSec #EthicalHacking
Replying to @MoinCyberTales
Well said. Least privilege should be the first line of defense, not an afterthought. Even if an agent is manipulated, the credentials and tools available to it should limit what it can actually do.
1
1
19
Replying to @marcoscyberx
Couldn't agree more. Least privilege, scoped credentials, and limiting side effects are critical. Prompts can guide agent behavior, but they shouldn't be the security boundary. The architecture needs to assume the agent might do something unexpected.
4
๐จ What can we learn from real-world AI security failures?
We have seen AI agents delete production databases, prompt injection attacks expose sensitive information, and vulnerabilities put AI-powered applications at risk.
But reading about an attack and understanding how it actually works are two different things.
๐ Today, I'm launching the ๐๐ ๐ฆ๐ฒ๐ฐ๐๐ฟ๐ถ๐๐ ๐๐ป๐ฐ๐ถ๐ฑ๐ฒ๐ป๐ ๐๐ฎ๐๐ฎ๐ฏ๐ฎ๐๐ฒ on GenAI Security Lab.
๐ What's inside?
๐น Real-world AI security incidents and vulnerabilities
๐น Technical breakdowns of how attacks work
๐น Root causes and lessons for developers and security professionals
๐น Mitigation strategies to help prevent similar attacks
๐น Links to hands-on labs where you can practise the underlying attack techniques.
The idea is simple: connect real-world AI security incidents with practical, hands-on learning.
Instead of just reading about prompt injection, AI agent vulnerabilities, data leaks, or MCP security issues, you can explore the technical details and practice relevant attack techniques in hands-on labs.
Whether you're an application security engineer, penetration tester, developer, or AI security researcher, this resource is built to help you go beyond theory.
๐ genaisecuritylab.com/incidenโฆ
This is part of my ongoing work on GenAI Security Lab, where were building hands-on labs to help security professionals learn to test and secure LLMs, RAG applications, AI agents, and MCP integrations.
Which real-world AI security incident should we break down next?
#AISecurity #LLMSecurity #CyberSecurity #AppSec #GenAI
Remember when you first tested an XSS payload and watched JavaScript execute in the browser? ๐
Now imagine an LLM generating that payload and the application executing it because the output wasnt handled securely.
๐ ๐ช๐ฒ๐น๐ฐ๐ผ๐บ๐ฒ ๐๐ผ ๐ข๐ช๐๐ฆ๐ฃ ๐๐๐ ๐ญ๐ฌ: ๐๐บ๐ฝ๐ฟ๐ผ๐ฝ๐ฒ๐ฟ ๐ข๐๐๐ฝ๐๐ ๐๐ฎ๐ป๐ฑ๐น๐ถ๐ป๐ด.
The underlying problem might feel familiar to web pentesters: untrusted data reaches a sensitive application component without proper validation, sanitization, or encoding.
But with LLM-powered applications, that data comes from the model, and developers may trust it more than they should. Depending on how the application handles the output, the consequences can include:
๐ด Cross-Site Scripting (XSS)
๐ด SQL Injection
๐ด Other injection vulnerabilities
For pentesters, this means looking beyond the prompt itself and examining what the application actually does with the models response.
I've just released ๐๐๐ ๐ญ๐ฌ ๐ถ๐ป ๐๐ต๐ฒ ๐ณ๐ฟ๐ฒ๐ฒ ๐๐ ๐ฆ๐ฒ๐ฐ๐๐ฟ๐ถ๐๐ ๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด ๐ฆ๐ฒ๐ฟ๐ถ๐ฒ๐ on GenAI Security Lab!
๐ด ๐๐ง๐ง๐๐๐ โ Find and exploit vulnerabilities.
๐ก๏ธ ๐๐๐๐๐ก๐ โ Learn how to fix them.
โ
๐ฉ๐๐ฅ๐๐๐ฌ โ Retest and verify the fix.
Ready to put your AI security skills into practice?
๐ genaisecuritylab.com/learn-aโฆ
#AISecurity #LLMSecurity #OWASP #AppSec #Pentesting #CyberSecurity #CyberSecurityTraining #ApplicationSecurity
๐จ ๐๐ผ๐ฟ ๐๐ต๐ฒ ๐ณ๐ถ๐ฟ๐๐ ๐๐ถ๐บ๐ฒ ๐ถ๐ป ๐๐ต๐ฒ ๐ฐ๐๐ฏ๐ฒ๐ฟ๐๐ฒ๐ฐ๐๐ฟ๐ถ๐๐ ๐๐ฟ๐ฎ๐ถ๐ป๐ถ๐ป๐ด ๐ถ๐ป๐ฑ๐๐๐๐ฟ๐, @GenAISecLab ๐ถ๐ ๐ถ๐ป๐๐ฟ๐ผ๐ฑ๐๐ฐ๐ถ๐ป๐ด ๐ฎ ๐๐ต๐ฟ๐ฒ๐ฒ-๐๐ถ๐ฒ๐ฟ ๐ฟ๐ฒ๐ด๐ถ๐ผ๐ป๐ฎ๐น ๐ฝ๐ฟ๐ถ๐ฐ๐ถ๐ป๐ด ๐บ๐ผ๐ฑ๐ฒ๐น ๐ฑ๐ฒ๐๐ถ๐ด๐ป๐ฒ๐ฑ ๐๐ผ ๐ฐ๐ผ๐๐ฒ๐ฟ ๐ฒ๐๐ฒ๐ฟ๐ ๐ฐ๐ผ๐๐ป๐๐ฟ๐ ๐๐ผ๐ฟ๐น๐ฑ๐๐ถ๐ฑ๐ฒ. ๐
Here is why this matters.
Many penetration testers have to pay for their own training and certifications. Most international training platforms charge in USD. But the same $100 subscription can be affordable in one country and a significant financial burden in another.
A pentester in one country shouldn't have to sacrifice a much larger share of their income to access the same training as someone living elsewhere.
We are building @GenAISecLab without outside funding(bootstrapped), so we understand the challenges of making a training platform sustainable. But we also believe that where you live shouldn't determine whether you can afford to develop your cybersecurity skills.
Some providers already offer regional discounts in selected markets. We wanted to take a broader approach by designing a three-tier pricing model intended to cover every country worldwide.
Because cybersecurity is a global industry, and access to cybersecurity education should be global, too. โค๏ธ
So we split our pricing into three tiers, set from income per person and local purchasing power:
โ Tier 1: standard pricing
โ Tier 2: around 20% off, for countries where the standard price is harder to reach
โ Tier 3: around 50% off, our lowest prices, where international training costs are the biggest barrier
Every tier gets the same hands-on labs, the same learning paths and the same certifications.
๐ ๐๐ป๐ฑ ๐๐ต๐ฒ๐ฟ๐ฒ๐ ๐บ๐ผ๐ฟ๐ฒ ๐ณ๐ผ๐ฟ ๐๐๐๐ฑ๐ฒ๐ป๐๐!
We understand that students often have limited budgets and may be paying for their education and professional development themselves.
That's why were offering an ๐ฎ๐ฑ๐ฑ๐ถ๐๐ถ๐ผ๐ป๐ฎ๐น ๐ฐ๐ฌ% ๐ข๐๐ ๐ณ๐ผ๐ฟ ๐๐๐๐ฑ๐ฒ๐ป๐๐, on top of their regional pricing.
We don't believe you should have to wait until you land your first cybersecurity job to afford the training that could help you get there.
Read complete story at : genaisecuritylab.com/blog/faโฆ
๐ Explore GenAI Security Lab: genaisecuritylab.com
#CyberSecurity #AISecurity #Pentesting #CyberSecurityTraining #ApplicationSecurity #AppSec
๐จ ๐ฅ๐๐-๐ฏ๐ฎ๐๐ฒ๐ฑ ๐ฎ๐๐๐ฎ๐ฐ๐ธ๐ ๐ฎ๐ฟ๐ฒ ๐ฐ๐ผ๐บ๐บ๐ผ๐ป, ๐๐ฒ๐ ๐ผ๐ณ๐๐ฒ๐ป ๐ผ๐๐ฒ๐ฟ๐น๐ผ๐ผ๐ธ๐ฒ๐ฑ ๐ฑ๐๐ฟ๐ถ๐ป๐ด ๐๐ฒ๐ฐ๐๐ฟ๐ถ๐๐ ๐๐ฒ๐๐๐ถ๐ป๐ด.
Our ๐ฃ๐ฎ๐ฟ๐ ๐ฑ ๐ผ๐ณ ๐๐ต๐ฒ ๐๐ฅ๐๐ ๐๐ ๐ฆ๐ฒ๐ฐ๐๐ฟ๐ถ๐๐ ๐ฆ๐ฒ๐ฟ๐ถ๐ฒ๐ is now live at genaisecuritylab.com/learn, covering ๐ข๐ช๐๐ฆ๐ฃ ๐๐๐ ๐ฌ๐ต: ๐ฉ๐ฒ๐ฐ๐๐ผ๐ฟ ๐ฎ๐ป๐ฑ ๐๐บ๐ฏ๐ฒ๐ฑ๐ฑ๐ถ๐ป๐ด ๐ช๐ฒ๐ฎ๐ธ๐ป๐ฒ๐๐๐ฒ๐. ๐ฅ
A vector database retrieves content based on similarity. But it doesn't inherently know who should access that content, who created it, or whether it should have been deleted.
๐ง๐ต๐ถ๐ป๐ธ ๐ผ๐ณ ๐ถ๐ ๐ฎ๐ ๐ฏ๐ฟ๐ผ๐ธ๐ฒ๐ป ๐ฎ๐ฐ๐ฐ๐ฒ๐๐ ๐ฐ๐ผ๐ป๐๐ฟ๐ผ๐น, ๐๐ถ๐๐ต ๐ฎ ๐๐ถ๐บ๐ถ๐น๐ฎ๐ฟ๐ถ๐๐ ๐๐ฒ๐ฎ๐ฟ๐ฐ๐ต ๐๐ต๐ฒ๐ฟ๐ฒ ๐๐ต๐ฒ ๐ผ๐ฏ๐ท๐ฒ๐ฐ๐ ๐๐ ๐๐๐ฒ๐ฑ ๐๐ผ ๐ฏ๐ฒ. ๐
This week, were covering ๐ณ ๐ต๐ฎ๐ป๐ฑ๐-๐ผ๐ป ๐น๐ฒ๐๐๐ผ๐ป๐ exploring six ways RAG pipelines can go wrong:
๐ ๐๐ฟ๐ผ๐๐-๐๐ฒ๐ป๐ฎ๐ป๐ ๐ฑ๐ฎ๐๐ฎ ๐น๐ฒ๐ฎ๐ธ๐ through improperly scoped retrieval.
๐ฏ ๐ฆ๐ฒ๐ป๐๐ถ๐๐ถ๐๐ฒ ๐ฑ๐ฎ๐๐ฎ ๐ฒ๐
๐ฝ๐ผ๐๐๐ฟ๐ฒ through overly broad search results.
๐ ๐๐ผ๐ฐ๐๐บ๐ฒ๐ป๐ ๐ฝ๐ผ๐ถ๐๐ผ๐ป๐ถ๐ป๐ด that manipulates answers for future users.
๐ ๐ฅ๐ฎ๐ป๐ธ๐ถ๐ป๐ด ๐บ๐ฎ๐ป๐ถ๐ฝ๐๐น๐ฎ๐๐ถ๐ผ๐ป that makes untrusted content outrank official sources.
๐๏ธ ๐ฆ๐๐ฎ๐น๐ฒ ๐ฑ๐ฎ๐๐ฎ ๐ฒ๐
๐ฝ๐ผ๐๐๐ฟ๐ฒ through indexes or caches that still serve deleted content.
๐ต๏ธ ๐๐ถ๐ฑ๐ฑ๐ฒ๐ป ๐ฟ๐ฒ๐๐ฟ๐ถ๐ฒ๐๐ฎ๐น ๐ฝ๐ฎ๐๐ต๐ that bypass access controls through parent expansion, agent follow-ups, HyDE, hybrid search, and fallback logic.
๐ก๏ธ ๐ง๐ต๐ฒ ๐ณ๐ถ๐
: Enforce authorization in code at query time across every retrieval path.
๐ฅ๐ฒ๐บ๐ฒ๐บ๐ฏ๐ฒ๐ฟ: ๐ฌ๐ผ๐๐ฟ ๐๐๐ ๐ ๐๐๐๐๐ฒ๐บ ๐ฝ๐ฟ๐ผ๐บ๐ฝ๐ ๐ถ๐ ๐ป๐ผ๐ ๐ฎ๐ป ๐ฎ๐ฐ๐ฐ๐ฒ๐๐ ๐ฐ๐ผ๐ป๐๐ฟ๐ผ๐น ๐บ๐ฒ๐ฐ๐ต๐ฎ๐ป๐ถ๐๐บ.
๐ป ๐๐ฒ๐ฎ๐ฟ๐ป ๐ฎ๐ป๐ฑ ๐ฝ๐ฟ๐ฎ๐ฐ๐๐ถ๐ฐ๐ฒ ๐ณ๐ผ๐ฟ ๐๐ฅ๐๐:
โ
No sign-up required to read the lessons.
โ
Hands-on labs with live, sandboxed targets.
โ
Free account for access to free labs.
๐
Next Tuesday: ๐๐๐ ๐ญ๐ฌ โ ๐๐บ๐ฝ๐ฟ๐ผ๐ฝ๐ฒ๐ฟ ๐ข๐๐๐ฝ๐๐ ๐๐ฎ๐ป๐ฑ๐น๐ถ๐ป๐ด.
#AISecurity #LLMSecurity #RAG #OWASP #AppSec #Pentesting
On an AI pentest, the system prompt is your recon. It tells you the tools, the rules and the guardrails before you send a single payload. Part 4 of my free AI security series is live: ๐๐ถ๐ฑ๐ฑ๐ฒ๐ป ๐๐ผ๐ป๐๐ฒ๐
๐ ๐๐
๐ฝ๐ผ๐๐๐ฟ๐ฒ (๐ข๐ช๐๐ฆ๐ฃ ๐๐๐ ๐ฌ๐ด).
LLM08 is about getting an assistant to give up what sits behind it: its instructions, its tool definitions, its guardrail rules. For a tester, that is the map for everything you try next.
Live now at genaisecuritylab.com/learn
Five ways it happens:
โข ๐๐ถ๐ฟ๐ฒ๐ฐ๐ ๐ฒ๐
๐๐ฟ๐ฎ๐ฐ๐๐ถ๐ผ๐ป - you ask the assistant to repeat or list its instructions. Do not reveal this is a request, not a control.
โข ๐ง๐ฟ๐ฎ๐ป๐๐ณ๐ผ๐ฟ๐บ๐ฎ๐๐ถ๐ผ๐ป ๐ฎ๐ป๐ฑ ๐ฒ๐ป๐ฐ๐ผ๐ฑ๐ถ๐ป๐ด - a translate or reformat task reproduces the hidden setup in a form an exact-match filter does not recognise.
โข ๐ฃ๐ฟ๐ถ๐บ๐ถ๐ป๐ด ๐ฎ๐ป๐ฑ ๐ณ๐ผ๐ฟ๐ด๐ฒ๐ฑ ๐๐๐ฟ๐ป๐ - a model completes text, so a primed sentence opening gets the setup echoed back.
โข ๐๐ฎ๐ธ๐ฒ ๐บ๐ผ๐ฑ๐ฒ๐ ๐ฎ๐ป๐ฑ ๐ฏ๐ผ๐ฟ๐ฟ๐ผ๐๐ฒ๐ฑ ๐ฎ๐๐๐ต๐ผ๐ฟ๐ถ๐๐ - claim a debug session or an audit, and disclosure becomes the task.
โข ๐๐ป๐ฑ๐ถ๐ฟ๐ฒ๐ฐ๐ ๐น๐ฒ๐ฎ๐ธ๐ฎ๐ด๐ฒ - the instruction is planted in a document the assistant retrieves for someone else.
The important part is what you tell the client. You cannot make a model keep a secret it can read. So the fix is to stop putting secrets there: keys, internal URLs and business rules belong in code and in server-side checks, not in the prompt. Assume the system prompt will be read, and make sure nothing breaks when it is.
If you have ever found credentials in client-side JavaScript, this will feel familiar. Same mistake, new place.
Five short lessons, each with a live target to try it on. Free to read and try our free labs, no paywall.
Next Tuesday, ๐ข๐ฐ๐๐ผ๐ฏ๐ฒ๐ฟ ๐ญ๐ฏ: ๐๐๐ ๐ฌ๐ต ๐ฉ๐ฒ๐ฐ๐๐ผ๐ฟ amp; ๐๐บ๐ฏ๐ฒ๐ฑ๐ฑ๐ถ๐ป๐ด ๐ช๐ฒ๐ฎ๐ธ๐ป๐ฒ๐๐๐ฒ๐, covering the RAG stack itself: embedding inversion, cross-tenant retrieval, and poisoned indexes.
#LLMSecurity #Pentesting #AISecurity #AppSec #CyberSecurity #GenAISecurity #AI #GenAI
Aditya Agrawal retweeted
Ten years ago I published a free Android application security series at manifestsecurity(dot) com/android-application-security. At the time, Android security had almost no free material to learn from โ web security had plenty. That gap is why I wrote it. It still gets traffic today, and people still message me saying it's how they learned Android pentesting.
AI security has that same gap right now. So I'm rebuilding it โ this time for AI.
genaisecuritylab.com/learn is live.
It teaches LLM security the way you'd actually test it. Short concept material, then live labs in your browser where you run the full loop: Attack โ Defend โ Verify. You exploit a real target, implement the fix, and verify the fix holds. Every topic has an offensive and a defensive track. It's a live system you break with your own hands โ not a module you click through.
Two are live today: AI Security Foundation, and Prompt Injection start to finish โ direct, jailbreaks, encoding, smuggling, indirect, tool/agent-mediated, and the defense for each.
Over the next 12 weeks I'll publish the rest free, one part a week, for pentesters and AppSec engineers who already know web and need to add AI to their scope:
โข System prompt and secret extraction
โข RAG data leakage and cross-tenant bleed
โข Agent and MCP tool abuse, excessive agency
โข Improper output handling โ second-order injection
The point: applications are shipping with LLM components now, and clients are starting to ask for AI testing. This gets you ready for that.
Every part maps to the OWASP LLM Top 10 (2026), the Agentic Top 10, and the MCP Top 10 โ and every part ends in a live hosted lab you attack right in the browser. No setup, no API keys, no Docker. Attack it, fix it, prove the fix holds.
I'm picking 100 people as the founding cohort โ early access to each part before it goes public, a private channel with me, and your feedback directly shaping the curriculum. If you want in, say so in the comments. I'll reply to every one and send you the link.
#AISecurity #AppSec #LLMSecurity
Part 3 of this series is live today.
More details at: xitter.cf/exploitprotocol/statusโฆ
Part 3 of my free AI security series is live: ๐ฆ๐ฒ๐ป๐๐ถ๐๐ถ๐๐ฒ ๐๐ป๐ณ๐ผ๐ฟ๐บ๐ฎ๐๐ถ๐ผ๐ป ๐๐ถ๐๐ฐ๐น๐ผ๐๐๐ฟ๐ฒ (๐ข๐ช๐๐ฆ๐ฃ ๐๐๐ ๐ฌ๐ฎ).
Five days ago I launched the "Learn AI Security" series with Foundations and Prompt Injection, and said Iโd publish a new part every week. Hereโs the next one.
LLM02 is about situations where an AI application gives out data it was never supposed to show. Live now at genaisecuritylab.com/learn
A few common examples:
โข ๐ฆ๐ฒ๐ฐ๐ฟ๐ฒ๐๐ ๐ฎ๐ป๐ฑ ๐ฐ๐ผ๐ป๐ณ๐ถ๐ด๐๐ฟ๐ฎ๐๐ถ๐ผ๐ป exposed through the model context - API keys, internal endpoints, tool details, etc.
โข ๐๐ป๐๐ฒ๐ฟ๐ป๐ฎ๐น ๐ธ๐ป๐ผ๐๐น๐ฒ๐ฑ๐ด ๐น๐ฒ๐ฎ๐ธ๐ฎ๐ด๐ฒ - staff-only runbooks, debug traces, or internal instructions being returned to users.
โข ๐ฃ๐ฒ๐ฟ๐๐ผ๐ป๐ฎ๐น ๐ฑ๐ฎ๐๐ฎ ๐ผ๐๐ฒ๐ฟ-๐ฑ๐ถ๐๐ฐ๐น๐ผ๐๐๐ฟ๐ฒ - the model returns an entire customer record when the task only required one field.
โข ๐๐ฟ๐ผ๐๐-๐๐ฒ๐ป๐ฎ๐ป๐ ๐ผ๐ฟ ๐ฐ๐ผ๐ป๐๐ฒ๐ฟ๐๐ฎ๐๐ถ๐ผ๐ป-๐ต๐ถ๐๐๐ผ๐ฟ๐ ๐น๐ฒ๐ฎ๐ธ๐ฎ๐ด๐ฒ - one customerโs data ends up visible to another through shared context, caches, or chat history.
โข ๐ฅ๐ฒ๐๐ฟ๐ถ๐ฒ๐๐ฎ๐น ๐ฎ๐ป๐ฑ ๐ฎ๐ด๐ด๐ฟ๐ฒ๐ด๐ฎ๐๐ถ๐ผ๐ป ๐น๐ฒ๐ฎ๐ธ๐ - RAG retrieves restricted information, or several harmless responses can be combined to reveal something sensitive.
The important part is how you defend against these issues.
A system prompt saying โnever reveal customer dataโ isnโt a reliable security control. The stronger approach is to control what gets retrieved, who is allowed to retrieve it, and what actually makes it into the modelโs context.
If youโve tested IDOR and multi-tenant isolation in traditional web applications, a lot of this will feel familiar.
The interesting part is learning how to actually find these issues in an LLM application.
The format is the same as the previous parts:
Attackโ Defend โ Verify
You can work through live labs directly in the browser.
No setup and no API keys required.
Both the attack and defense paths are free.
Next Tuesday, ๐ข๐ฐ๐๐ผ๐ฏ๐ฒ๐ฟ ๐ฒ: ๐๐๐ ๐ฌ๐ด โ ๐๐ถ๐ฑ๐ฑ๐ฒ๐ป ๐๐ผ๐ป๐๐ฒ๐
๐ ๐๐
๐ฝ๐ผ๐๐๐ฟ๐ฒ, covering system prompts, hidden instructions, guardrails, and other context that shouldnโt be exposed.
#AISecurity #AppSec #LLMSecurity
40
Part 3 of my free AI security series is live: ๐ฆ๐ฒ๐ป๐๐ถ๐๐ถ๐๐ฒ ๐๐ป๐ณ๐ผ๐ฟ๐บ๐ฎ๐๐ถ๐ผ๐ป ๐๐ถ๐๐ฐ๐น๐ผ๐๐๐ฟ๐ฒ (๐ข๐ช๐๐ฆ๐ฃ ๐๐๐ ๐ฌ๐ฎ).
Five days ago I launched the "Learn AI Security" series with Foundations and Prompt Injection, and said Iโd publish a new part every week. Hereโs the next one.
LLM02 is about situations where an AI application gives out data it was never supposed to show. Live now at genaisecuritylab.com/learn
A few common examples:
โข ๐ฆ๐ฒ๐ฐ๐ฟ๐ฒ๐๐ ๐ฎ๐ป๐ฑ ๐ฐ๐ผ๐ป๐ณ๐ถ๐ด๐๐ฟ๐ฎ๐๐ถ๐ผ๐ป exposed through the model context - API keys, internal endpoints, tool details, etc.
โข ๐๐ป๐๐ฒ๐ฟ๐ป๐ฎ๐น ๐ธ๐ป๐ผ๐๐น๐ฒ๐ฑ๐ด ๐น๐ฒ๐ฎ๐ธ๐ฎ๐ด๐ฒ - staff-only runbooks, debug traces, or internal instructions being returned to users.
โข ๐ฃ๐ฒ๐ฟ๐๐ผ๐ป๐ฎ๐น ๐ฑ๐ฎ๐๐ฎ ๐ผ๐๐ฒ๐ฟ-๐ฑ๐ถ๐๐ฐ๐น๐ผ๐๐๐ฟ๐ฒ - the model returns an entire customer record when the task only required one field.
โข ๐๐ฟ๐ผ๐๐-๐๐ฒ๐ป๐ฎ๐ป๐ ๐ผ๐ฟ ๐ฐ๐ผ๐ป๐๐ฒ๐ฟ๐๐ฎ๐๐ถ๐ผ๐ป-๐ต๐ถ๐๐๐ผ๐ฟ๐ ๐น๐ฒ๐ฎ๐ธ๐ฎ๐ด๐ฒ - one customerโs data ends up visible to another through shared context, caches, or chat history.
โข ๐ฅ๐ฒ๐๐ฟ๐ถ๐ฒ๐๐ฎ๐น ๐ฎ๐ป๐ฑ ๐ฎ๐ด๐ด๐ฟ๐ฒ๐ด๐ฎ๐๐ถ๐ผ๐ป ๐น๐ฒ๐ฎ๐ธ๐ - RAG retrieves restricted information, or several harmless responses can be combined to reveal something sensitive.
The important part is how you defend against these issues.
A system prompt saying โnever reveal customer dataโ isnโt a reliable security control. The stronger approach is to control what gets retrieved, who is allowed to retrieve it, and what actually makes it into the modelโs context.
If youโve tested IDOR and multi-tenant isolation in traditional web applications, a lot of this will feel familiar.
The interesting part is learning how to actually find these issues in an LLM application.
The format is the same as the previous parts:
Attackโ Defend โ Verify
You can work through live labs directly in the browser.
No setup and no API keys required.
Both the attack and defense paths are free.
Next Tuesday, ๐ข๐ฐ๐๐ผ๐ฏ๐ฒ๐ฟ ๐ฒ: ๐๐๐ ๐ฌ๐ด โ ๐๐ถ๐ฑ๐ฑ๐ฒ๐ป ๐๐ผ๐ป๐๐ฒ๐
๐ ๐๐
๐ฝ๐ผ๐๐๐ฟ๐ฒ, covering system prompts, hidden instructions, guardrails, and other context that shouldnโt be exposed.
#AISecurity #AppSec #LLMSecurity
Replying to @Mohamed_Jeddah
Great, I will ping you and share the details once things are finalized.
1
31
Ten years ago I published a free Android application security series at manifestsecurity(dot) com/android-application-security. At the time, Android security had almost no free material to learn from โ web security had plenty. That gap is why I wrote it. It still gets traffic today, and people still message me saying it's how they learned Android pentesting.
AI security has that same gap right now. So I'm rebuilding it โ this time for AI.
genaisecuritylab.com/learn is live.
It teaches LLM security the way you'd actually test it. Short concept material, then live labs in your browser where you run the full loop: Attack โ Defend โ Verify. You exploit a real target, implement the fix, and verify the fix holds. Every topic has an offensive and a defensive track. It's a live system you break with your own hands โ not a module you click through.
Two are live today: AI Security Foundation, and Prompt Injection start to finish โ direct, jailbreaks, encoding, smuggling, indirect, tool/agent-mediated, and the defense for each.
Over the next 12 weeks I'll publish the rest free, one part a week, for pentesters and AppSec engineers who already know web and need to add AI to their scope:
โข System prompt and secret extraction
โข RAG data leakage and cross-tenant bleed
โข Agent and MCP tool abuse, excessive agency
โข Improper output handling โ second-order injection
The point: applications are shipping with LLM components now, and clients are starting to ask for AI testing. This gets you ready for that.
Every part maps to the OWASP LLM Top 10 (2026), the Agentic Top 10, and the MCP Top 10 โ and every part ends in a live hosted lab you attack right in the browser. No setup, no API keys, no Docker. Attack it, fix it, prove the fix holds.
I'm picking 100 people as the founding cohort โ early access to each part before it goes public, a private channel with me, and your feedback directly shaping the curriculum. If you want in, say so in the comments. I'll reply to every one and send you the link.
#AISecurity #AppSec #LLMSecurity
AI agents now cover most of the deterministic vulnerabilities we spent the last 15 years mastering ๐
It's made me think about how our role as pentesters and security engineers will evolve โ not because those skills are gone, but because AI/LLM, Agentic and MCP security are quietly moving from "specialty" to essential skill.
Iโve shared a few thoughts here: genaisecuritylab.com/blog/noโฆ
Would love to hear your thoughts on this! ๐






