Ryan Cobb (@cobbr_io) header
Ryan Cobb (@cobbr_io) profile picture

Developing training labs and CTFs @SpecterOps Developer: Covenant, SharpSploit, PSAmsi

Dallas, TX
Joined December 2010
Photo by Ryan Cobb (@cobbr_io)Photo by Ryan Cobb (@cobbr_io)Photo by Ryan Cobb (@cobbr_io)Photo by Ryan Cobb (@cobbr_io)Photo by Ryan Cobb (@cobbr_io)Photo by Ryan Cobb (@cobbr_io)Photo by Ryan Cobb (@cobbr_io)Photo by Ryan Cobb (@cobbr_io)Photo by Ryan Cobb (@cobbr_io)Photo by Ryan Cobb (@cobbr_io)
I’m very excited to share a new project that I’ve been working on for a long time: Covenant, a collaborative .NET command and control framework! [blog/tool] Entering a Covenant: .NET Command and Control - posts.specterops.io/entering…
22
286
10
581
Software uptime has been so good over the last decade that a few services go down temporarily and everyone assumes it’s WW3. With millions of services on the internet, I’m honestly amazed we don’t see more outages than we currently do. Uptime is hard.
1
9
728
Ryan Cobb retweeted
AI agents can now break into realistic enterprise networks on their own. A capability once limited to trusted-access programs is available to anyone. Today we’re releasing Vals CyberOffense. @willknight from @WIRED wrote an article about our work, link in the thread.
16
15
12
145
18,691
Ryan Cobb retweeted
We found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on our multi-step offensive cybersecurity task. Sol was tested via the Daybreak trusted access program. We have been building a cybersecurity benchmark based on simulated corporate networks. This is a valuable indicator for the cyber-risk a model poses if in the hands of a bad actor. We plan to make the full evaluation public in the coming weeks but wanted to share our first immediate finding here.
104
227
194
2,489
458,802
Ryan Cobb retweeted
New from the Tradecraft Academy 👇 Built in partnership w/ @OpenAI through OpenAI Daybreak Defense Network, Adversary Intelligence: LLM Tradecraft provides hands-on training to build, evaluate, attack, & defend LLM & agentic systems. More from @harmj0y: ghst.ly/4AAOLn7
10
27
2,973
Ryan Cobb retweeted
New content for AP:A will be debuting at SpecterBash 🎉 This will be the first time the new Agents and Identities section will be taught as part of the course AND there is a brand new CTF for participants.
See Azure and Entra ID the way attackers do. 👀 During our Azure course at #SpecterBash, learn to identify the misconfigs and attack paths that matter, including those involving agents and identities. Get the attacker’s perspective: ghst.ly/45COqlF
2
5
1,588
Ryan Cobb retweeted
Our Red Team tested GPT-6 on simulated cyber eval scenarios with cyber classifiers disabled and found that it performed a range of malicious actions. It tries to conduct supply chain attacks against simulated open source providers and goes beyond its scope to attack targets on the simulated internet. deploymentsafety.openai.com/…
Replying to @_robertkirk
1: Astra performed a range of malicious actions in this simulated eval, including conducting supply chain attacks against open-source providers. This happened even when we updated its scope to more explicitly exclude the public internet.
10
33
9
186
33,770
We need a rule, if you are publicly complaining about getting safeguarded by Fable cause “cyber” with a benign prompt: You also need to show us the skills and tools loaded into your context. And if contains 100 different red teaming and hacking skills, you are banned from complaining about this again.
1
24
5,041
My apparently very unpopular opinion is that the explanation here demonstrates OpenAI was running evals in a way that should have been expected to be reasonably safe (no internet access, monitoring, etc.), but the models have significant alignment problems that require an undue burden of safety for evaluation setups (must withstand multiple 0-day discoveries). If you want to complain about the other examples we have seen where the agent had unrestricted internet access, fine I get it. But if you want to complain about the setup here, you just want to complain. I’m sure there’s improvements that can be made, but the primary story here is both model misalignment and capability, not unsafe eval infra.
Yesterday, my OpenAI collaborator and I gave a detailed talk on the Huggingface incident, our models creating "the message board", model misalignment, and more. youtube.com/watch?v=87DyyMV0… I hope it can answer a lot of the questions folks have, and we will release a full detailed postmortem at a later time!
10
6
1
66
8,354
Worrying that agents are hacking targets that any human would know are clearly not authorized for the test. Emphasizes the need for human-in-the-loop for real-world offense-tasked agents. Testing these alignment issues is also valuable. We need to create safer ways to test it.
On July 28th, we identified an incident during a routine cyber evaluation in which AI agents took sustained, unsanctioned actions directed at real people and organisations. The behaviour came mostly from one model (Anthropic's Mythos 5), with a small number of events from another (OpenAI's GPT-5.6-Sol). In the most serious case, an agent used social engineering to try and get malicious code into an open-source project. As was standard in our cyber testing, we had intentionally permitted internet access, and model-provider cyber classifiers were deliberately disabled - conditions that do not reflect how frontier models are made available to the public. Even under test conditions, this incident is significant: it is the first time we have seen risks around autonomy and deception manifest this clearly in the real world. We are taking this incident seriously and working with labs, involved parties, and others to improve evaluation standards and best practice for disclosure - and sharing this openly so others can learn. You can read the incident report and full technical document here: aisi.gov.uk/blog/incident-re…
1
4
1,022
Ryan Cobb retweeted
Introducing Tradecraft Academy 📚 Our new on-demand training platform brings the practitioner-led education we are known for to learners anywhere, anytime. Get started with BloodHound Basics today. Read more: ghst.ly/4ywKLmM Start learning: ghst.ly/4wBbZqE
20
3
32
3,919
On our cyber range "The Last Ones", GLM-5.2 matches Opus 4.5, released ~7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5, from ~7 months before it.
4
8
9
92
112,432
Ryan Cobb retweeted
LLM security and jailbreak testing gets messy fast. I built Jailbreaker to make it easier to run and track jailbreak, prompt injection, and agent behavior tests. Blog - specterops.io/blog/2026/06/2… Github - github.com/SpecterOps/Jailbr… Would love to hear what you think. @SpecterOps
1
13
24
2,621
Ryan Cobb retweeted
Testing an LLM once is easy. Testing it consistently is harder. Neeraj Gupta's latest GhostWorks research introduces Jailbreaker, an open-source platform built to make jailbreak, prompt injection, & agent behavior testing repeatable and easier to manage. ghst.ly/4gdCHk1
2
8
30
2,856
Ryan Cobb retweeted
Registration for training at #BHUSA 2026 is officially open, and we are bringing a full slate of adversary-focused courses to Las Vegas. 🎰 Early reg. pricing ends May 22, so now is the time to lock in your seat. 🧵 Links to each course ⤵️
1
5
1
15
4,747
OpenAI’s GPT-5.5 is the second model to complete one of our multi-step cyber-attack simulations end-to-end 🧵
91
393
232
2,338
1,792,126
We conducted cyber evaluations of Claude Mythos Preview and found that it is the first model to complete an AISI cyber range end-to-end. 🧵
110
541
183
2,983
1,275,197
Ryan Cobb retweeted
HEY EVERYONE. THE BLOG POST IS OUT. I put an LLM in an AMSI provider and some cool stuff came out. Really excited to finally have this released.
"Offense and defense aren't peers. Defense is offense's child." - @JohnLaTwC We built an LLM-powered AMSI provider and paired it against a red team agent. Then, @0xdab0 wrote a blog about it: dreadnode.io/blog/llm-powere… A few observations from the experiment: >>> To advance, we must generate unique, ground-truth datasets. >>> Defenses will need to live at the edge. >>> The real potential lies in the interaction between red and blue. >>> This is a blueprint for generative adversarial reinforcement learning.
4
16
73
12,205
SCOM monitors critical systems, but insecure defaults make it a powerful attack vector. At #BHEU, @unsigned_sh0rt & @breakfix show how to abuse SCOM for credential theft, lateral movement, and domain escalation, plus how to defend it. ghst.ly/4aoggph
31
1
73
25,687