About ScopeJudge: dreadnode.io/research/scope-… ScopeJudge on GitHub: github.com/dreadnode/scopeju…

Jev adds a fast check for contextual scope decisions, a promising step toward efficient runtime judges. Of course, not everything is a nail with this new hammer. Hard limits belong in code: permissions, network restrictions, and sandbox controls. If code can decide, enforce it there. [4/4]

Then, we tested judging escalation: start with Jev and bring in smarter judges when needed. Low-confidence decisions go to GLM; disagreements go to Opus. Most decisions stayed with Jev. The chain scored comparably to the paper’s best at half the cost. [3/4]

We tested Jev on 4,897 recorded agent actions, compared its decisions with human reviewers, and measured it against the paper’s published results. When provided the user’s request and an action to check, Jev had the highest score in that setting. [2/4]

Does Jev live up to the hype? Based on the results of running it against our ScopeJudge benchmark, it does. @typesafeai's Jev was competitive with leading LLM judges, catching agent scope violations at pennies per thousand checks, with 130 millisecond responses on average. [1/4]

👀👀👀👀👀👀👀👀👀👀

We're out here cybermaxxing models and mogging Mythos. Thanks to all who attended @Dr_Machinavelli's @LabsSentinel LabsCon talk this afternoon 😎

💪💪💪💪💪 Tomorrow (9/17) @Dr_Machinavelli takes the stage at the final @SentinelOne @labscon_io to discuss how to leverage offensive cyber capabilities to improve defenses in the age of AI. More info: labscon.io/speakers/martin-w…

Qwen 3.8 Flash eval results are live on DreadIndex, landing at #19 on our leaderboard. It does well for its cost, but remains light on offensive security capability (not surprising given it is a flash model). See how it compares to other models: dreadnode.io/research/dreadi…

GLM-5.3-Flash results now on DreadIndex: dreadnode.io/research/dreadi…

Here's where to catch the Dreadnode crew at year two of @OffensiveAIcon: > Join us for the welcome reception at The Shelter Club on Sunday evening! > @mkultraWasHere is closing out Day One of talks, presenting on model cheating behavior. > Dynamic duo @shanejcaldwell and @0xdab0 take the stage on Tuesday for a session on implementing a judge model as a runtime monitor, and how to keep agents in scope. See you in Oceanside! 🏄

Worried about your production agents going out of scope? Us too. AgentJudge is our agent hall monitor that stops out of scope tool calls before they execute. Before a tool runs, the judge reads the agent’s intent, the proposed call, and a rubric you define, then returns allow, deny, or ask (escalate to you). The agent stays autonomous; the judge is the guardrail. Available in the TUI today, UI updates coming to the Dreadnode Platform soon! 👀 Get Started: docs.dreadnode.io/getting-st… AgentJudge Docs: docs.dreadnode.io/tui/guard-… Related Research: dreadnode.io/research/scope-…

Dreadnode side quest: ALFRED (Agentic Latex for Research, Editing, and Drafting) Principal AI Research Engineer @mkultraWasHere built a helpful LaTeX agent to support research writing — and today we’re open-sourcing it. You describe the paper, it sets up the template, pulls citations, and builds the PDF framework. Conference templates, lit and peer reviews, bring your own model, everything runs locally. Watch this tutorial for a tour of the agent, from install through first compiled draft: youtube.com/watch?v=ZP0Nnyvo… Repo: github.com/dreadnode/alfred

Read more on the US open-weight model performance gap via our Head of Policy @velvethamm3r: dreadnode.io/research/from-c…

Two new additions to #DreadIndex: nemotron-3-ultra-550b-a55b and gemma-4-31b-it. Landing at the bottom of the leaderboard, these evaluations offer two more proof points to increase investment in US open models. 🔗: dreadnode.io/research/dreadi… NEW: we recently added a toggle to view only open weight models.

& check out Shane's research on cost-aware pre-execution gating for offensive security agents #yearofthejudge dreadnode.io/research/scope-…

New article from @CNET discusses why AI agents keep hacking their way out of test environments — and what to do about it. Dreadnode Principal Research Engineer @shanejcaldwell's answer: an AI hall monitor. Agent judges for runtime monitoring, escalating to a human when scope is breached. Read the full article: cnet.com/tech/services-and-s…

Mine The Gap Heading to Vegas for @CrowdStrike Fal.Con next week? Don't miss @Dr_Machinavelli's Day Zero keynote where he pits red and blue team agents against each other to generate training data that can be used to increase AI performance, impose realistic constraints, and operate at scale. 🗣️ CrowdStrike Day Zero Threat Research Summit 📍 Virgin Hotel Las Vegas 🗓️ Monday, August 31 ⌚ 9:15 - 9:45 AM PT 🔗 crowdstrike.com/en-us/events…

New models are regularly being added to DreadIndex: dreadnode.io/research/dreadi… Observations from the latest evals—Deepseek v4 Pro 0813, GLM 5.3, and Qwen 3.8 Max—in the thread 🧵⬇️ Have you tried these models out yet? Curious if our eval results align to first-hand operator usage.