Fully automated vulnerability research is changing the cybersecurity landscape Claude 3 Opus is capable of reading source code and identifying complex security vulnerabilities used by APTs. But scaling is still a challenge. Demo: claude.ai/share/ddc7ff37-f97… This is beginner-level prompt engineering: I just simply asked the model to role-play a cyberdefense assistant and to look for a class of vulnerability. And yet, even with this trivial prompting, Claude was able to identify the vulnerability which was unveiled in googleprojectzero.blogspot.c… a month after our training data cutoff: Code defect scanning is not new, but this technique points the way to a more nuanced, complete analysis—especially with very large, 1M token context windows. This is part of a larger story: there are now two different ways that vulnerability discovery is being automated by defenders. 1) Defenders can wire-up LLMs to cluster fuzzers Starting with last generation’s models, Google pioneered this work here: security.googleblog.com/2023… . This has now been implemented by a number of players on the defenders’ side since last year. The technique is to have LLMs write the test harnesses and triage the results. This has, reportedly, increased the fuzzer signal-to-noise ratio by 20x, depending on the technique and software. Google has new research in this area posted recently. 2) Defenders can ask models to analyze code using large context windows The next level-prompt is to take large amounts of related code, and iterate through each class of coding vulnerability requesting that the output be produced pointing to a line of code in JSON format. Currently, with frontier models like Claude 3, doing this for every file in a codebase as large as the Linux kernel could get pricey. But, for those with the time, there’s vulnerabilities to find via this method. And the price will, of course, go down over time as compute gets more accessible. We measure these capabilities as a part of our Responsible Scaling Policy evals. We don't see very advanced capabilities today, but capabilities are jumpy. We could see that coming very soon and what we don't know is how offense/defense dominant this will be. But I think, right now, these tools are going to be super useful/important for mostly defending current systems. We are building advanced cyber reasoning evals and looking at this closely—come collaborate with us! I am proud that we’re also partnering with @DARPA to support #AIxCC, a first-of-its-kind competition challenging the masses to identify new ways to shift the #AI cybersecurity offense-defense balance in the defender’s favor. Fully automated vulnerability discovery is here and it will only become more available. I believe that—in the future—the availability of patches to fix vulnerabilities from these methods will be coming out daily. Defenders should prepare to patch their environments on a rolling basis.