Is Defense Winning?

Is Defense Winning?

Speakers: Unknown

Conference: Black Hat USA

YouTube: https://www.youtube.com/watch?v=ZYv_292wMa4

Overview

In a thought-provoking and introspective session at Black Hat USA, a seasoned cybersecurity expert from Columbia University School of International and Public Affairs posed a fundamental question to the community: "Is Defense Winning?" This talk wasn't about a new exploit or a revolutionary defense technology, but rather a critical examination of the collective progress (or lack thereof) in cybersecurity over decades. The speaker challenged the audience to look beyond individual enterprise successes or failures and assess whether the global defense community is, on the whole, gaining ground against attackers.

The presentation highlighted a pervasive sense of stagnation, despite immense investments in resources, talent, and innovation. The speaker argued that while headlines constantly barrage us with news of breaches, vulnerabilities, and rising cybercrime, a clear, overarching metric for defensive success remains elusive. This talk, the fourth in a series dating back to 2014, aimed to spark a community-wide discussion on how to define, measure, and ultimately achieve victory in the ongoing cyber conflict, reframing it not merely as a scientific problem but as a critical public policy challenge.

Background

▶ Watch: Introduction: Is defense winning? (0:00)

The speaker initiated the discussion by drawing parallels to historical military analyses, referencing their work on the first military history book of cyberspace. A central theme was the enduring advantage held by attackers, encapsulated in the speaker's assertion: O is greater than D (Offense is greater than Defense). This isn't a new phenomenon; the speaker presented quotes from as far back as 1970, illustrating that the fundamental challenges faced by defenders today – such as the difficulty of securing an open system or the consistent success of red teams – were recognized by their "grandparents" in the field five decades ago.

Despite hundreds of billions of dollars invested, countless hours of training, thousands of innovations and patents, and the personal sacrifices of cybersecurity professionals (worked weekends, missed family events), the community has seemingly failed to shift this inherent advantage towards the defensive side. The speaker lamented this persistent imbalance, noting that while the industry celebrates individual breakthroughs, the collective strategic position of defense against offense has remained largely unchanged. This historical context underscores the urgency of the talk's central question: if decades of effort haven't moved the needle, what fundamental shifts are required to achieve a measurable advantage? The speaker posited that the current approach lacks the discipline and framework necessary to assess collective progress, treating a complex public policy issue as a purely scientific one.

Key Findings

▶ Watch: Historical context: Attacker advantages are 50 years old (2:55)

The primary "finding" of this conceptual talk is not a technical discovery, but rather the stark realization that the cybersecurity community lacks a cohesive, disciplined framework to determine if defense is collectively "winning." The speaker emphasizes that while individual organizations might improve their posture, there is no community-wide metric or understanding of whether the aggregate defense is outperforming the aggregate offense. This absence of a clear measure prevents strategic shifts and perpetuates the cycle of reactive defense.

A crucial insight presented is the need to view this challenge as a public policy problem rather than solely a science problem. While achieving scientific rigor in measurement might take a decade, the speaker argued that the urgency of the situation demands immediate, disciplined action in framing the problem and identifying indicators. This shift in perspective is critical for developing a framework that can measure success. The speaker proposed a conceptual spectrum of cybersecurity supremacy, ranging from attacker supremacy (where even basic threat actors achieve substantial objectives against elite defenders) to defense supremacy (where adversaries struggle to achieve simple goals even against poorly resourced organizations). The current state, implicitly, lies somewhere in the middle, but without clear indicators, its exact position and trajectory remain unknown. The speaker's core assertion that "O is greater than D" serves as the baseline understanding for the current state, suggesting that attackers generally possess systemic advantages that allow them to achieve their objectives more readily. The talk concludes that without a shared understanding of what "winning" looks like and how to measure it, the community is effectively fighting blind, making it impossible to ascertain if the massive collective investment is yielding the desired strategic outcome.

Technical Deep Dive

▶ Watch: O > D: Offense greater than Defense despite investment (3:27)

This talk, while delivered at a technical conference, did not delve into specific technical vulnerabilities, exploits, or defensive architectures. Instead, its "technical deep dive" was a meta-analysis of the problem of measurement itself within the cybersecurity domain. The speaker's focus was on the conceptual framework required to assess the strategic balance between offense and defense, rather than the intricate details of any particular technology or protocol.

The speaker's argument for "logical propositions" as indicators for winning or losing represents the closest point of a "technical" framework, albeit at a high, abstract level. These propositions, though not detailed in the provided transcript, are intended to serve as the foundational criteria upon which defensive success could be objectively measured. For example, such propositions might involve metrics like the average time to detect and respond to an intrusion, the cost-effectiveness of defensive measures versus offensive campaigns, or the percentage of successful attacks against a defined baseline of protected assets. The challenge, as highlighted by the speaker, is that the community has yet to agree upon or rigorously apply such propositions across the board.

The underlying "technical" problem, from the speaker's perspective, is the systemic advantage that offense holds, which they term "O is greater than D." This isn't a specific technical weakness but a broad observation about the inherent asymmetry in cybersecurity. Attackers often need only one successful vulnerability to exploit, while defenders must secure an entire attack surface. This fundamental imbalance, compounded by the complexity of modern IT environments and the rapid evolution of threats, means that even with advanced security tools and protocols, the attacker often finds a path. The talk implicitly suggests that a true "technical deep dive" into the effectiveness of defense would require a systematic analysis of these propositions, potentially involving aggregated data on attack success rates, vulnerability remediation times, and the impact of various defensive controls across a wide range of organizations and threat actors. However, the talk's primary contribution is to identify the absence of this overarching measurement framework as the critical issue, rather than to provide the technical specifics of such a framework itself.

Demo / Proof of Concept

▶ Watch: Defining spectrums of attacker and defense supremacy (4:09)

The nature of this presentation was conceptual and analytical, focusing on the overarching strategic question of cybersecurity defense rather than a specific technical solution or vulnerability. Therefore, no live demonstration or proof of concept was presented during the talk. The speaker's objective was to initiate a critical discussion and propose a framework for evaluating collective progress, not to showcase a new tool, exploit, or defensive technique.

Defensive Implications

▶ Watch: Approaching cybersecurity as a public policy problem (4:50)

The defensive implications stemming from this talk are profound and call for a significant paradigm shift in how the cybersecurity community approaches its collective mission. If, as the speaker argues, "O is greater than D" and this imbalance has persisted for decades despite massive investment, then current defensive strategies are fundamentally insufficient at a systemic level.

First and foremost, defenders must recognize the urgent need for standardized, community-wide metrics to assess collective progress. Without knowing if we are winning or losing, and by how much, strategic resource allocation and policy decisions remain guesswork. This implies a move towards greater data sharing, anonymized incident reporting, and the development of common indicators of compromise (IOCs) and defensive success metrics that transcend individual organizational boundaries.

Secondly, the call to treat cybersecurity as a public policy problem rather than solely a science problem has significant implications. This means engaging with policymakers, economists, and social scientists to understand the broader societal impacts of cyber insecurity and to craft regulations, incentives, and international agreements that can shift the balance. For instance, policies that promote secure by design principles, liability for insecure products, or collaborative threat intelligence sharing could have a far greater systemic impact than isolated technical advancements. Defenders should advocate for and participate in these policy discussions, bridging the gap between technical expertise and strategic governance.

Thirdly, the recognition of the attacker's "system-wide advantages" (O > D) should prompt defenders to reconsider traditional reactive security models. Instead of solely focusing on patching individual vulnerabilities or deploying point solutions, there needs to be a greater emphasis on resilience, deception, and active defense strategies that aim to fundamentally alter the cost-benefit analysis for attackers. This might involve adopting zero-trust architectures more broadly, implementing advanced threat hunting capabilities, or leveraging AI/ML for automated defense to reduce the attacker's window of opportunity and increase their operational costs. The goal is not just to block attacks, but to make attacking prohibitively expensive and difficult, even for well-resourced adversaries.

Finally, the talk implicitly advocates for a more disciplined approach to security investment and innovation. Rather than simply spending more money or developing more tools, defenders need to strategically evaluate whether these investments are contributing to a measurable shift in the O > D equation. This requires a feedback loop where defensive efforts are continually assessed against agreed-upon "logical propositions" of success, allowing for adaptation and refinement of strategies based on empirical evidence of their impact on the overall balance of power in cyberspace.

Key Takeaways

• The fundamental question of "Is Defense Winning?" remains unanswered: Despite decades of effort and investment, the cybersecurity community lacks a cohesive, disciplined framework to determine if defense is collectively gaining ground against attackers.
• Attackers hold persistent "system-wide advantages": The speaker asserts that "O is greater than D," meaning offense inherently holds advantages that have been recognized since at least the 1970s, making it easier for attackers to achieve objectives.
• Cybersecurity is a public policy problem, not just a science problem: Achieving scientific rigor in measuring success might take a decade, but the urgency demands a disciplined, public policy approach to define and track progress using logical propositions.
• A spectrum of supremacy exists, but our position is unclear: The talk outlines a conceptual spectrum from "attacker supremacy" to "defense supremacy," but without clear metrics, the community cannot ascertain its current standing or trajectory.
• Massive investment hasn't shifted the balance: Hundreds of billions of dollars, extensive training, and thousands of innovations have not fundamentally altered the attacker's advantage, calling into question the effectiveness of current collective strategies.
• The community needs to define and measure "winning": To make meaningful progress, defenders must agree on what constitutes "winning" and establish disciplined, measurable indicators to track collective success, moving beyond anecdotal evidence and individual incident responses.

About the Speaker(s)

The speaker, whose name was not explicitly stated in the provided transcript, is a highly experienced and distinguished figure in the cybersecurity domain. They currently hold a position at Columbia University School of International and Public Affairs. The speaker has a unique background, having served in both the White House and on the review boards for both Defcon and Black Hat, indicating a deep understanding of both policy and technical aspects of cybersecurity. This particular talk marks their fourth appearance at Black Hat on the same overarching topic, with previous talks dating back to 2014, where they discussed "saving cyberspace" and improving defense against offense. Their work includes authoring what they describe as the "first military history book of cyberspace," further highlighting their expertise in strategic and historical analysis of cyber conflict.