Keynote: The End of Rare: Defending When Offense Is Cheap

Keynote: The End of Rare: Defending When Offense Is Cheap

Speakers: David Weston, Agentic Security Leader, Microsoft

Conference: Black Hat USA

YouTube: https://www.youtube.com/watch?v=uNdlqEqxUVw

Overview

In his compelling Black Hat USA keynote, "The End of Rare: Defending When Offense Is Cheap," David Weston, Agentic Security Leader at Microsoft, delivered a stark warning and an optimistic roadmap for the future of cybersecurity. Weston argues that the foundational assumption of scarcity in cyberattacks—that undermining security boundaries is inherently difficult and costly—is rapidly being eroded by the advent of Agentic AI. This shift, he contends, is not merely an incremental improvement for attackers but a fundamental change in the economics of offense, making vulnerability discovery, exploit generation, and sophisticated evasion techniques cheaper, faster, and more accessible than ever before.

Weston, drawing on his two decades in the trenches of security and his recent immersion in the AI world, highlights a dramatic surge in vulnerability volume and the alarming ease with which AI can now generate functional exploits and evade detection. He challenges the traditional defensive strategies that rely on rapid patching or reactive detection, asserting that these approaches will inevitably fail in a world where offense is commoditized. Instead, he proposes a proactive, AI-driven defense strategy focused on durable security, shifting left to secure by design, leveraging formal methods for provable safety, and employing AI agents for comprehensive infrastructure prevention.

This talk is critical for the cybersecurity community because it not only articulates an existential threat to current defensive paradigms but also offers a tangible, optimistic path forward. Weston's vision is one where defenders harness the same productivity advantages of AI to fundamentally change the "physics" of security, making attacks more expensive and less effective. By investing in memory-safe languages, formal verification, and infrastructure as code, the industry can build an inherently more resilient digital ecosystem, transforming the landscape from one of constant, reactive combat to one of strategic, AI-augmented prevention.

Background

▶ Watch: Black Hat challenge: discover and engage (2:00)

For decades, the cybersecurity industry has operated on a deeply ingrained, often unstated, assumption: that security boundaries are inherently difficult to undermine, thus making successful attacks rare and costly. These boundaries—encompassing network segmentation, process isolation, identity management, encryption, and data protection policies—form the bedrock of modern digital defense. The perceived difficulty of breaching these fundamental layers has shaped economic models, such as bug bounties, where the payout for a hypervisor bypass can be twenty times that of a process boundary crossing, reflecting the expertise and effort required.

This scarcity principle extends beyond mere vulnerability discovery to actualized risk, i.e., successful exploitation. An implemented attack is traditionally even more costly, requiring specialized expertise for mitigation bypasses and sophisticated techniques. Consequently, despite tens of thousands of CVEs being reported annually, only around 90 in-the-wild exploits are tracked by Google each year, underscoring the exceptional nature of successful boundary undermining. This economic reality has meant that most breaches today occur "far above the boundaries," typically involving credential theft, phishing, or social engineering, which are significantly cheaper attack vectors. When vulnerabilities are exploited, they are often known issues that organizations have had time to patch.

This historical context has led to two dominant, albeit increasingly fragile, security strategies. The first is a diminished focus on the Software Development Life Cycle (SDLC), with less emphasis on static analysis, safer languages, or the principle of least privilege. The underlying logic here is that if vulnerabilities are rarely exploited, organizations can afford to "patch fast" once a vulnerability becomes known. The second strategy is a reliance on detection and response, where security operations centers (SOCs) and vendors aim to quickly identify and neutralize threats. This approach often assumes that attackers, being "software developers" themselves, cannot afford to constantly change their implants, C2 infrastructure, or lateral movement tools, allowing defenders to build durable detections based on invariant Tactics, Techniques, and Procedures (TTPs) and artifacts. However, Weston argues that these strategies are critically vulnerable to a future where AI commoditizes offense, rendering exploits and evasions cheap and ubiquitous, effectively removing the scarcity that has historically protected defenders.

Key Findings

▶ Watch: Celebrating Black Hat scholarship attendees (5:50)

David Weston's keynote unveils several alarming findings that underscore the rapid erosion of attack scarcity, driven by Agentic AI:

1. Explosive Growth in Vulnerability Volume: Microsoft's internal data, specifically from the Microsoft Security Response Center (MSRC), reveals an unprecedented surge in vulnerability reports. MSRC is now doubling the number of vulnerabilities it processes and patches every six weeks. Since March of the current year, the volume of vulnerabilities has increased ninefold. This trend is not isolated to Microsoft but is observed across other major software vendors like Google and Apple, indicating a systemic shift. Weston strongly correlates this surge with AI, noting that Microsoft's internal vulnerability harness, deployed on Windows on April 1st, discovered 66% of all critical and important issues found since that date, compared to the entirety of the previous year. These aren't trivial bugs; the data set included seven remote TCP/IP vulnerabilities that cross both kernel and remote boundaries, the kind that previously took a year of bespoke crafting.

1. Automated Exploit Generation as a Commodity: The shift extends from vulnerability discovery to exploit generation. Weston shares data from Microsoft's internal MdASH harness, which found approximately 200 Linux kernel vulnerabilities in Azure Linux distributions. Crucially, a new module added to MdASH could automatically generate 182 crash-level Proof of Concepts (PoCs), many of which were fully working root exploits, from these static analysis results. The average cost for this automated exploit generation was a mere $3.61 in tokens and took only 21 minutes. This internal capability is mirrored externally; platforms like Exploit Gym demonstrate frontier models generating 157 exploits out of 898 real-world vulnerabilities, including complex browser bugs. While non-deterministic mitigations like ASLR still pose challenges, Weston is confident that automatic exploit generation will be commonplace and commoditized by year-end.

1. Cheap Evasion and Dynamic Attack Frameworks: The assumption that attacker TTPs and tooling remain relatively invariant for detection is also being challenged. AI enables attackers to generate bespoke tools and frameworks per target, circumventing traditional signature-based or behavior-based detection methods. Weston cites Anthropic's November report on a cyber operator using cloud code with sub-agents to achieve 80-90% efficacy in top-tier operations. More concretely, a Drago report from May detailed an AI-assisted group targeting a water utility that built its command-and-control (C2) and implant framework during the operation, generating 17,000 lines of C2/implant code for that single engagement. This demonstrates that evasion is no longer a scarce property requiring extensive human effort.

1. Trajectory Towards Autonomous Operations: The UK's National Cyber Security Centre (NCSC) and Security Industry (SI) have been testing frontier models on their ability to conduct multi-step autonomous breach operations. These models are already achieving 9.8 steps out of a 32-step operation using 10 million tokens, representing a 59% increase in capability in just the past year. This trajectory suggests that fully autonomous breach operations will become a standard part of the threat landscape.

1. Restriction is Insufficient: Weston dismisses the idea that restricting access to frontier AI models will maintain scarcity. He points to platforms like CyberJ, where the top entrants for vulnerability discovery benchmarks are not just frontier models but sophisticated harnesses. These harnesses leverage frontier models but also inject context and cyber expertise through tooling and markdown files, demonstrating that cyber knowledge can be encoded outside the core model itself. This means that even if powerful models are restricted, the underlying methodology can be replicated and deployed by a wider range of actors.

These findings collectively paint a picture of a cybersecurity future where the traditional advantages of defenders—relying on the cost and complexity of offense—are rapidly diminishing, necessitating a radical re-evaluation of defensive strategies.

Technical Deep Dive

▶ Watch: Embracing infosec as political (6:55)

The core of Weston's proposed defense strategy lies in leveraging AI's productivity to build durable security and shift left, fundamentally changing the "physics" of cyber defense. This involves a multi-pronged technical approach focusing on memory safety, formal methods, and infrastructure as code.

Memory Safety and AI-Assisted Language Conversion

Weston emphasizes that approximately 70% of vulnerabilities patched by major vendors today are memory safety issues. These classes of bugs, such as buffer overflows, use-after-frees, and double-frees, can be eliminated by adopting safer system-level languages like Rust and Golang. He cites compelling evidence from Google, where Android, an operating system used by billions, saw its memory safety vulnerabilities drop from 76% in 2019 to less than 20% in 2025 due to the adoption of Rust. Google has written five million lines of Rust code with a thousand times fewer defects and has shipped zero memory safety issues in that codebase. Similarly, Azure's hypervisor, a critical containment boundary, was rewritten in open-source Rust and now scales past 1.5 million virtual machines without a single incident.

The traditional barrier to widespread adoption of these languages is the significant cost and expertise required for conversion and refactoring large, legacy C/C++ codebases. However, AI is changing this dynamic:

• Rust Assistant (Microsoft Research): This project demonstrated the ability to automatically fix 74% of compilation failures in Rust code, significantly lowering the barrier to entry for developers.
• Safe C/Check C: For existing C codebases, AI can infer memory safety contracts, generating 86% of the necessary code contracts to check for spatial safety vulnerabilities like buffer overflows.
• Automatic Code Conversion: Advanced projects are tackling full code conversion:
• Sila (Google, Microsoft): This research project successfully converted a SHA3 method (a few thousand lines of C code) into safe Rust, passing all tests with less than a 1% performance difference.
• Rustler: A more sophisticated project that uses dependency graphs and context to generate Rust code from C, iteratively refining it until it passes tests.
• DARPA Tractor: DARPA is actively sponsoring projects and providing datasets for automatic code conversion, recognizing its strategic importance.

By making memory-safe language adoption easier and faster, AI can dramatically reduce the attack surface for common vulnerability classes.

Formal Methods for Logical and Cryptographic Assurance

While memory safety eliminates a large class of bugs, it does not guarantee overall security. Weston points out that memory safety does not mean security, as logical flaws, authentication issues, and cryptographic vulnerabilities remain. These are often the most scarce and complex areas of security expertise.

Here, AI is proving capable of assisting with formal methods, a rigorous approach to proving the correctness of software. Traditionally, formal methods (like model checking and symbolic checking) have been limited by the difficulty of writing specifications, creating proofs, state space explosion in complex programs, and high maintenance costs. However, they offer deterministic safety and can provide a reproduction if a property is violated.

Weston highlights several breakthroughs:

• Anthropic's Claude: This frontier model demonstrated the ability to find a cryptographic attack in the Hawk post-quantum encryption algorithm and showed an 800x increase in attack performance against AES128. It also found a significant forgery bug in WolfSSL. These are traditionally high-scarcity, complex findings.
• AI Scaling Formal Methods:
• CBMC/K-Frama-C: Tools used by Amazon and AWS for checking critical libraries like libC and crypto implementations, demonstrating the real-world application of formal verification.
• Apple and Microsoft's Crypto Verification: Both companies have applied formal verification to their core crypto libraries, finding critical issues that passed all other forms of testing (fuzzers, human review). Microsoft, for instance, found a single incorrect bit in ProtoChem, a key encapsulation algorithm for Post-Quantum Cryptography (PQC), only through formal verification. AWS uses Cedar for formally verifying access control policies (ABAC).
• Anias and Lean: Microsoft demonstrated that, largely driven by AI agents, they could generate 30,000 lines of Lean code in a single week to formally verify AES GCM within the simcrypt library. Lean is a functional language for describing mathematical proofs. This achievement means the verified crypto is provably free of logical, cryptographic, and memory safety issues.

Weston argues that formal methods, augmented by AI, can become the "oracle for correctness" that is "tailor-made" for AI-generated code, providing mathematical grounding for the soundness and reliability of critical security boundaries. He cites the OpenAI escape incident, where despite strong isolation, nine logical vulnerabilities were found, emphasizing that formal verification of such package systems could have prevented the incident.

Infrastructure as Code (IaC) and Agent Armies for Prevention

Finally, Weston addresses the vast amount of existing infrastructure, much of which is compromised not by zero-day exploits but by configuration issues. He laments that "very little of that infrastructure can be reasoned about by agents" because it lacks a standardized, machine-readable format like Infrastructure as Code (IaC).

By converting infrastructure into IaC or other persisted policies, AI agents can holistically reason about and manage security posture:

• Graph Computation: Agents can create graphs or ontologies of all assets and network flows within an organization. This allows them to identify centrality of risk, focus remediation efforts, and detect anomalous connections (e.g., an unexpected connection from an accounting system to a domain controller).
• Automated Posture Management: Agents can validate security posture, identify overprivileged accounts, and pinpoint devices with the largest attack surface.
• Proactive Hunting and Remediation: Once a graph is computed, agents can scale out proactive hunting activities far beyond human capabilities and even resolve issues. An analysis of Checkov, a static analysis tool for IaC, showed that agents could resolve 78% of its findings.

This approach enables defenders to reduce attack surface, improve configuration, and shift left on the infrastructure side, preventing issues from reaching production and making them less accessible to attackers.

Demo / Proof of Concept

▶ Watch: Geopolitical cyber conflict examples (7:10)

While David Weston's keynote did not feature a live, interactive demonstration in the traditional sense, he presented compelling evidence and insights derived from internal Microsoft research and development, alongside references to external benchmarks, which serve as crucial proofs of concept for his arguments.

Weston extensively detailed the capabilities of Microsoft's internal tools and initiatives:

• MdASH (Microsoft's internal vulnerability harness): This system, deployed on Windows, serves as a proof that AI is driving the surge in vulnerability discovery. Its ability to find 66% of critical and important issues since April 1st, compared to the entire previous year, is a direct result of its AI-driven capabilities. Furthermore, MdASH's module that automatically generates 182 crash-level PoCs (many full root exploits) for Linux kernel vulnerabilities, at a cost of $3.61 and 21 minutes, is a stark demonstration of automated exploit generation.
• Rust Assistant (Microsoft Research): This project demonstrated AI's ability to automatically fix 74% of Rust compilation failures, proving AI's utility in accelerating the adoption of memory-safe languages.
• Sila (Google and Microsoft Research): The successful automatic conversion of a SHA3 method from C to Rust, with all tests passing and minimal performance impact, is a concrete example of AI-driven code transformation.
• Anias and Lean (Microsoft's Formal Verification Work): The achievement of generating 30,000 lines of Lean code in a single week to formally verify AES GCM, largely driven by AI agents, stands as a significant proof of concept for AI's role in scaling formal methods and achieving mathematical guarantees of software correctness.
• Checkov Integration: The analysis showing that AI agents could resolve 78% of findings from Checkov, an IaC static analysis tool, provides a practical example of AI's potential in automated infrastructure posture management and remediation.

Beyond internal projects, Weston referenced external benchmarks and real-world incidents as further proofs of AI's advancing offensive capabilities:

• Exploit Gym: This platform demonstrates the ability of frontier models to generate 157 exploits out of 898 real-world vulnerabilities, highlighting the commoditization of exploit generation.
• CyberJ: The observation that top entrants in vulnerability discovery benchmarks are AI-powered harnesses, rather than just raw frontier models, illustrates that sophisticated tooling can amplify AI's capabilities even with restricted model access.
• Anthropic and Drago Reports: These real-world reports of AI-assisted operators achieving high efficacy (80-90%) and dynamically generating 17,000 lines of C2 code during an operation serve as proofs of concept for AI's impact on evasion and autonomous operations.
• UK's SI Testing: The data showing frontier models achieving 9.8 out of 32 steps in autonomous breach operations, with a 59% increase in capability this year, provides a trajectory-based proof of concept for the future of AI in cyber conflict.

These examples, drawn from both cutting-edge research and observed threat actor behavior, collectively underscore Weston's central thesis: AI is fundamentally altering the economics of offense, and these capabilities are no longer theoretical but are being actively demonstrated and deployed.

Defensive Implications

▶ Watch: Community as cornerstone in rapid change (9:40)

David Weston's keynote provides a critical call to action for defenders, urging a fundamental shift in strategy to counter the commoditization of offense by Agentic AI. The defensive implications are profound, demanding a pivot from reactive measures to proactive, AI-augmented durable security.

1. Prioritize Secure by Design for High-Risk Surfaces: Defenders must identify their most critical and high-risk components (e.g., operating system kernels, hypervisors, core network services, web applications) and commit to making them secure by design. This means moving away from a "patch fast" mentality and investing upfront in inherent security. The immediate focus should be on adopting memory-safe languages like Rust or Golang for new development and actively converting existing vulnerable codebases. The proven efficacy in Android and Azure hypervisor demonstrates that a significant portion of common vulnerabilities can be eliminated at the source.

1. Embrace AI-Assisted Formal Verification for Critical Boundaries: For the most critical security boundaries—those governing authentication, cryptography, and core logic—defenders must move beyond traditional testing and adopt formal methods. AI can dramatically scale this process by assisting in specification generation, proof construction, and even converting code to formal proof languages like Lean. This allows for mathematical guarantees of correctness, ensuring that critical components are free from logical flaws, cryptographic vulnerabilities, and memory safety issues. Organizations should start by applying this to their most sensitive cryptographic primitives, access control systems, and core security functions.

1. Transform Infrastructure Management with Infrastructure as Code (IaC) and AI Agents: The vast majority of infrastructure compromises stem from misconfigurations, not zero-day exploits. Defenders must convert their infrastructure into Infrastructure as Code (e.g., using Terraform, Ansible, CloudFormation). This machine-readable format enables AI agents to reason holistically about the environment. These "agent armies" can then:

• Automate Posture Management: Continuously analyze configurations for vulnerabilities, overprivileged accounts, and attack surface.
• Proactive Threat Hunting: Compute attack graphs and ontologies to identify anomalous connections or potential lateral movement paths that humans might miss.
• Automated Remediation: Leverage AI to not just identify but also automatically resolve configuration issues, as demonstrated by the Checkov analysis.

This approach shifts prevention left in the infrastructure lifecycle, reducing the attack surface before it ever reaches production.

1. Re-evaluate Detection Strategies: While detection and response remain necessary, defenders must acknowledge that AI will make traditional TTP-based detection less durable. Attackers will be able to generate bespoke implants and dynamically change their methods, rendering static indicators and even behavioral patterns less effective over time. The emphasis should shift towards prevention at the source and deterministic safety through secure by design and formal methods, rather than solely relying on catching attackers in the act.

1. Invest in AI for Defensive Productivity: Weston's core message is that AI is a productivity engine for both offense and defense. Defenders must leverage AI's ability to automate, analyze, and reason at scale to match or exceed the accelerating pace of threats. This means investing in AI-driven tools for code analysis, vulnerability management, security policy enforcement, and operational automation to achieve "durable change in attacker economics."

By implementing these strategic shifts, defenders can move beyond a reactive, hand-to-hand combat scenario and fundamentally alter the "physics" of cyber warfare, making offense more expensive and less effective, thereby regaining the advantage in a world where attacks are no longer rare.

Key Takeaways

• AI is Ending Attack Scarcity: The traditional assumption that cyberattacks are rare and costly due to the difficulty of undermining security boundaries is rapidly being invalidated by Agentic AI, which is commoditizing vulnerability discovery and exploit generation.
• Traditional Defenses Are Insufficient: Reactive strategies like "patch fast" and detection based on invariant attacker TTPs will fail as AI enables attackers to generate exploits cheaply, quickly, and create bespoke, evasive tooling for every target.
• Embrace Secure by Design and Memory Safety: Defenders must proactively invest in building inherently secure software by adopting memory-safe languages like Rust, which AI can accelerate through automated code conversion and error fixing, drastically reducing the most common vulnerability classes.
• Leverage AI for Formal Verification: For critical security boundaries (e.g., cryptography, core logic), AI can scale formal methods, enabling mathematical proofs of correctness that guarantee software is free from logical, cryptographic, and memory safety issues, providing deterministic safety where it matters most.
• Build an Agent Army for Infrastructure Prevention: Convert infrastructure to Infrastructure as Code (IaC) to allow AI agents to holistically reason about and manage security posture. These agents can proactively identify and remediate configuration issues, compute attack graphs, and perform automated threat hunting at scale.
• Change the Physics, Not Just the Economics: Defenders must use AI's productivity advantage to shift left and invest in durable security mechanisms, making offense fundamentally more expensive and less effective, thereby regaining the strategic advantage in the evolving cyber landscape.

About the Speaker(s)

David Weston is the Agentic Security Leader at Microsoft, where he spearheads the development of AI models, agents, and evaluation systems specifically designed for defense at scale. With over 20 years of experience, Weston has been deeply entrenched in the security trenches, witnessing and responding to some of the most significant cyber events, including WannaCry and Stuxnet. His extensive background includes building operating system security for Windows, Linux, and Azure, leading efforts in EDR (Endpoint Detection and Response) systems, and managing vulnerability discovery and red team operations. In the nine months prior to this keynote, he transitioned into the AI world, focusing on creating vulnerability discovery harnesses, training frontier models for cyber capabilities, and architecting advanced AI-driven defenses. This unique blend of deep defensive experience and cutting-edge AI expertise positions him to offer a critical perspective on the future of cybersecurity.