Pinned Tweet
Considering the releases we had this week, GPT-6 Sol is the best option for lower refusal rate in cybersecurity related workflows
We just open sourced AgentStoryline! A time machine for your agents. Replay your logs and visualise it as a storyline.
Watch your subagents talking to each other as if they were having a private chat.
Inspect every message, model response and tool
From day 1 we support Artex logs, @OrcaRouter .orca files (experimental) and more...
github.com/humanwill-ai/agen…
🤖 Made with AI
Meta Muse Wonder is a very good model for cybersecurity tasks. Refusal rate is reasonably low, this is a model you can actually use 😍
Mistral Large 4 is out. We are surprised by the promising scores in our cyber false refusal eval:
- False refusals: 18.40%
- Mean usefulness: 3.12/4
Well done Mistral!
“Don’t share our code with unapproved destinations.” How do you apply that rule across AI tools? We’re exploring it with HumanWill Jev Policies. Our first public preview is live:
github.com/humanwill-ai/huma…
...and while you just get refusals from your models, someone is getting useful answers to very same questions by using an uncensored model 🤩
Introducing Pirate Face
The first verifiable torrent index for AI - every Hugging Face open model, provably identical to the original and kept alive forever by the swarm.
pirateface.co
The permanence layer for AI. How it works 👇
Anthropic paper mentions how GLM-5.3 has 'strong capabilities for autonomously building end-to-end cyber exploits. But GLM-5.3 is unlike other frontier models in that it has been released without meaningful safeguards to limit misuse'
That means that while hackers can make use of really powerful models to identify and exploit vulnerabilities, defenders will face more and more restrictions by using closed models.
We believe that false refusal are a measurable security cost, they represent a risk and increase remediation time and analyst workload. And all that should be measured.
Sonnet 5.5 introduced for the first time cyber safeguards very close to what the already introduced in Claude Opus 5.5. 1000x more false refusals
In our false refusal cyber eval Opus 5.5 is reaching a never seen before refusal rate of almost 94%, the model cyber-classifiers are even more aggressive than Fable 5.1 😶
1/3 Grok 4.7 vs 4.6: fewer false refusals overall, but not across the board.
On HumanWill’s 424-question cybersecurity challenge set, the rate fell from 64.39% to 62.74%.
The biggest improvement? Binary investigation: 65.00% → 41.25%.
2/3 False refusals by topic (4.6 → 4.7):
Binary: 65.00% → 41.25%
Network: 75.28% → 74.16%
Vulnerability validation: 56.79% → 62.96%
Source-code security: 60.71% → 63.10%
Filesystem/artifacts: 63.33% → 70.00%
Two topics improved; three worsened.
3/3 Binary investigation had 19 fewer refused questions; the other topics combined had 12 more.
Same questions and judge policy. These are challenge-set false refusals, not everyday rates or proof of technical correctness.
github.com/humanwill-ai/huma…
