Exposing Attacks Hiding in the Sheer Noise of False Positives

Exposing Attacks Hiding in the Sheer Noise of False Positives

Speakers: E Tahoun; L Hamida

Conference: DEF CON 32

YouTube: https://www.youtube.com/watch?v=h8E6w3vWzWo

Overview

In his DEF CON 32 talk, "Exposing Attacks Hiding in the Sheer Noise of False Positives," Ez Tahoun delivered a compelling argument for a fundamental shift in how the cybersecurity industry approaches detection and response. Rather than focusing on the latest tools or buzzwords like AI, Tahoun emphasized the critical importance of context and a use case-driven framework for identifying genuine threats amidst the overwhelming volume of security alerts. The core problem addressed is the pervasive issue of alert fatigue and the resultant difficulty in distinguishing sophisticated attacks from the "sheer noise" of false positives generated by conventional security systems.

Tahoun's presentation was less a technical deep dive into a new exploit or defensive mechanism, and more a philosophical call to action for blue teams, security operations analysts, and data scientists. He argued that despite technological advancements, security today is paradoxically less effective than decades ago due to increasing complexity and a reliance on fragmented tools that lack holistic understanding. By bringing in industry practitioners like Harry Sarwari, CIO at SAP NS2, and George Alra, CISO for Ruby Life (Ashley Madison), Tahoun grounded his arguments in real-world operational challenges, highlighting how human investigators often succeed where automated systems fail, precisely because of their ability to contextualize disparate pieces of information.

This talk is particularly relevant for any organization struggling with an unmanageable volume of security alerts, a high false positive rate, or difficulty in correlating events into coherent incident narratives. Tahoun challenged the audience to prioritize understanding the "use case" – the specific problem to be solved and the desired outcome – before investing in any particular tool or technology. This approach aims to empower defenders to cut through the noise, uncover genuinely malicious activity, and build more resilient security operations centered around meaningful intelligence rather than mere data aggregation.

Background

▶ Watch: Talk introduction: focusing on use cases, not just AI (0:00)

The landscape of cybersecurity has transformed dramatically over the past few decades, evolving from relatively simple, isolated systems to complex, interconnected global infrastructures. Ez Tahoun vividly illustrates this shift, noting that "it used to be so fucking simple. Now it's so fucking complicated." Fifty years ago, the attack surface was limited, and threats were often more straightforward. Today, the proliferation of cloud environments, remote workforces, IoT devices, and sophisticated supply chains has created an unprecedented level of complexity, vastly expanding the potential points of compromise.

This increased complexity directly correlates with an explosion in the volume of security telemetry: logs, events, and alerts generated by a multitude of security tools. Traditional Security Information and Event Management (SIEM) systems, like ArcSight which George Alra recalled dealing with as a security operations analyst, were designed to aggregate and correlate this data. However, the sheer scale of modern environments often overwhelms these systems, leading to a deluge of alerts – many of which are false positives or low-priority informational events. This phenomenon, known as alert fatigue, is a critical problem for blue teams, rendering security operations "bloated and ineffective." Analysts spend an inordinate amount of time sifting through irrelevant data, leading to burnout and, more dangerously, the very real possibility of legitimate threats being missed amidst the noise.

Tahoun emphasized that the root causes of this ineffectiveness are often "not technology," but rather "people and process." Organizations frequently acquire new security tools without a clear understanding of the specific problems they need to solve or how these tools integrate into existing workflows. This ad-hoc approach results in a fragmented security posture where tools operate in silos, generating isolated alerts that lack the necessary context to form a complete picture of an attack. The speaker's analogy of the Stuxnet worm highlights this: while Stuxnet required a physical breach (a USB drive) and years to circulate, modern threats can exploit vulnerabilities across vast, interconnected smart grids with far greater speed and ease, making effective detection more critical than ever. The fundamental challenge, as Tahoun articulates, is determining "what alerts, events and logs are relevant to each other" to uncover a true incident.

Key Findings

▶ Watch: Speaker's core philosophy: 'context is all you need' (1:30)

Tahoun's talk doesn't present new technical vulnerabilities or exploits, but rather critical insights into the operational failings of contemporary cybersecurity and the inherent strengths of human analysis. The central "finding" is the stark contrast between the outstanding contextualization capabilities of human investigators and the limitations of current automated security systems in providing meaningful context.

He observed that when practitioners like his guest CISOs recount incidents, they describe a multi-step investigative process: "Step one, we found this thing. Step two, there was this other thing. Step three, there's like this other thing that we found out and we uncovered." This narrative highlights that successful incident response is rarely about a single, definitive alert. Instead, it's about connecting disparate pieces of information, understanding their relationships, and building a cohesive timeline of events – a process that requires deep contextual understanding. Human investigators excel at this because they can draw upon tacit knowledge, experience, and an understanding of the business environment, threat actor motivations, and system behaviors that are difficult for automated systems to replicate.

Conversely, the "bloated and ineffective" nature of current security tools stems from their inability to automatically provide this level of context. They generate a high volume of isolated alerts, each signaling a potential anomaly, but often failing to link them into a broader attack chain. This leads to the "sheer noise of false positives" where genuine attacks can hide. Tahoun implicitly finds that the industry's focus on "tools" and "AI" without first defining the "use case" – the specific attack scenarios or behaviors that need to be detected – is a misdirection. The true finding is that effective detection hinges on a clear understanding of the problem and the necessary context to solve it, a capability currently best demonstrated by skilled human analysts. The challenge, then, is to bridge this gap, either by augmenting human capabilities or by designing systems that can better emulate contextual reasoning.

Technical Deep Dive

▶ Watch: Guest intro: Harry Sori, SAP/DoD, discusses large projects (2:15)

While not detailing a new technical exploit, Ez Tahoun's talk provides a critical technical deep dive into the limitations and ineffectiveness of prevailing security architectures and methodologies. The core technical problem lies in the inability of current security tools and processes to effectively correlate and contextualize the massive volume of security telemetry they generate.

Modern security environments are characterized by an overwhelming influx of logs, events, and alerts from diverse sources: endpoint detection and response (EDR) agents, network intrusion detection/prevention systems (NIDS/NIPS), firewalls, cloud platform logs, identity and access management (IAM) systems, and more. SIEM solutions, like the aforementioned ArcSight, are designed to ingest, normalize, and store this data, applying rules and signatures to identify known malicious patterns or anomalies. However, as Tahoun points out, this rule-based approach often falls short.

The speaker highlights the reliance on detection rules and the challenge of mapping them to higher-level adversary behaviors. He introduces MITRE ATT&CK as a crucial framework for this purpose. ATT&CK provides a globally accessible knowledge base of adversary tactics and techniques based on real-world observations. It offers a common language and reference point for understanding "what happened when we talk about the what, the who, the when" of an incident. The technical benefit of ATT&CK is its ability to elevate discussions from low-level alerts (e.g., "process X made a network connection to IP Y") to higher-level adversary techniques (e.g., "Command and Control: Ingress Tool Transfer").

However, Tahoun implicitly suggests that even with frameworks like ATT&CK, a significant gap remains. Simply having rules mapped to ATT&CK techniques is insufficient if the underlying data lacks the necessary context or if analysts don't understand how to interpret these mappings. He notes that "a lot of people in this industry are so scared to ask or say that they don't know," leading to a superficial understanding and ineffective application of powerful frameworks. The technical challenge is not just generating alerts, but enriching them with contextual information from across the environment (e.g., asset criticality, user behavior baselines, threat intelligence, vulnerability data) to make them actionable.

The "bloated and ineffective" nature of security tools arises from several technical deficiencies:

• Lack of Semantic Correlation: Tools often generate alerts in isolation. A firewall alert, an EDR alert, and an identity log might all relate to the same attack phase but are not automatically linked by the SIEM without complex, brittle correlation rules.
• Poor Signal-to-Noise Ratio: Overly broad rules, misconfigured sensors, or legitimate but unusual activity can generate a high volume of false positives. This technical noise obscures actual threats.
• Static Rule Limitations: Signature-based and static rule-based detections struggle against novel attacks or variations of known techniques. They lack the adaptability to recognize subtle deviations from normal behavior without generating excessive false positives.
• Data Silos: Despite SIEMs, data often remains logically siloed. Full context requires integrating information from various security layers, business context (e.g., who owns this system, what is its function?), and external threat intelligence feeds.

Tahoun's emphasis on "context is all you need" is a technical plea for systems that can intelligently aggregate, enrich, and correlate disparate data points to form a coherent narrative of an attack. This implies a need for more sophisticated event correlation engines, potentially leveraging graph databases or advanced analytics (including AI, if applied correctly to a defined use case) to identify relationships and patterns that current systems miss. The goal is to move beyond simply alerting on individual anomalous events to detecting chains of events that collectively signify malicious activity, thus revealing "attacks hiding in the sheer noise."

Demo / Proof of Concept

▶ Watch: Guest intro: George Alra, Ashley Madison CISO, on Arcsite (2:40)

Ez Tahoun's talk did not feature a live demonstration or a technical proof of concept of a specific tool or methodology. Instead, the presentation focused on a conceptual framework and a philosophical shift in approach to cybersecurity operations. The speaker relied on real-world anecdotes and the validation of his industry practitioner guests (Harry Sarwari and George Alra) to illustrate the problem of alert fatigue and the critical need for context in security investigations. His emphasis was on the "use case" and "people and process" rather than showcasing a specific technical solution.

Defensive Implications

▶ Watch: Why practitioners are vital: human contextualization in incidents (4:00)

The core defensive implication of Ez Tahoun's talk is a mandate for security teams to fundamentally rethink their approach to threat detection and incident response, moving away from a tool-centric mindset to one that prioritizes context and use cases. For blue teams struggling with alert fatigue and the challenge of identifying real threats, several actionable strategies emerge:

1. Prioritize Use Cases Over Tools: Before acquiring any new security product or implementing a new detection rule, organizations must clearly define the specific attack scenarios or adversary behaviors they aim to detect. This "use case-first" approach ensures that technology investments are aligned with actual defensive needs, rather than being driven by vendor features or marketing. For example, instead of asking "What can this EDR do?", ask "How do we detect an attacker moving laterally after initial access?"
2. Enhance Contextualization of Alerts: Security operations centers (SOCs) need to enrich raw alerts with as much contextual information as possible. This includes:

• Asset Criticality: Understanding the business impact of a compromised system.
• User Baselines: Knowing typical user behavior to identify anomalies.
• Vulnerability Data: Correlating alerts with known vulnerabilities on affected systems.
• Threat Intelligence: Linking indicators of compromise (IOCs) to known threat actors and campaigns.
• Business Process Awareness: Understanding normal application and network flows.

This enrichment allows analysts to quickly discern the true severity and relevance of an alert, reducing false positives.

1. Invest in People and Process: Tahoun explicitly states that the problem is often "people and process," not technology. This means:

• Training and Education: Equipping analysts with the skills to understand complex attack chains and leverage frameworks like MITRE ATT&CK effectively. Addressing the fear of asking questions to foster a culture of continuous learning.
• Streamlined Workflows: Developing clear, repeatable playbooks for incident response that guide analysts through contextualization and investigation steps.
• Collaboration: Fostering strong communication channels between security, IT operations, and business units to gather necessary context.

1. Leverage MITRE ATT&CK Effectively: Go beyond simply mapping rules to ATT&CK techniques. Use the framework to:

• Identify Coverage Gaps: Understand which adversary tactics and techniques your current defenses can (or cannot) detect.
• Develop Detections: Design new detection rules specifically targeting critical ATT&CK techniques relevant to your threat model.
• Communicate Effectively: Use ATT&CK as a common language to describe incidents and defensive strategies across the organization and with external partners.

1. Focus on Chained Events, Not Isolated Alerts: Shift detection strategies from identifying individual anomalous events to recognizing sequences of events that collectively indicate an attack. This requires more sophisticated correlation logic, potentially leveraging graph databases or behavioral analytics to link seemingly disparate activities into a cohesive narrative. The goal is to emulate the human investigators' ability to connect "Step one," "Step two," and "Step three" into a complete incident picture.
2. Continuous Improvement and Tuning: Regularly review detection rules, analyze false positive rates, and tune security tools based on real-world incident data. This iterative process is crucial for refining the signal-to-noise ratio and ensuring that security operations remain effective against evolving threats.

By adopting these defensive implications, organizations can move closer to Tahoun's vision of having "context is all you need," enabling them to expose attacks that would otherwise remain hidden in the overwhelming noise of modern security environments.

Key Takeaways

• Context is Paramount: Effective cybersecurity hinges on understanding the full context of security events, not just isolated alerts. Without context, even advanced tools struggle to differentiate real threats from false positives.
• Use Cases Drive Effectiveness: Security investments and detection strategies should start with clearly defined "use cases" – specific attack scenarios or adversary behaviors to detect – rather than being tool-driven.
• Human Intelligence is Key: Skilled human investigators excel at contextualizing disparate information to uncover attack chains, a capability automated systems often lack. Security operations should aim to augment and leverage this human strength.
• Address People and Process: The primary challenges in modern security are often rooted in organizational processes and human factors, such as alert fatigue, lack of training, and poor communication, rather than solely technological limitations.
• Leverage MITRE ATT&CK Strategically: The MITRE ATT&CK framework provides a crucial common language for understanding adversary tactics and techniques, but its effective application requires deep understanding and integration into detection engineering and incident response processes.
• Complexity Breeds Insecurity: The ever-increasing complexity of IT environments has inadvertently made security less effective by generating an unmanageable volume of data and alerts, making it easier for sophisticated attacks to hide.

About the Speaker(s)

Ez Tahoun is the primary speaker for this DEF CON 32 talk, passionately advocating for a use case-driven approach in cybersecurity. While his specific title and company are not detailed in the transcript, he is introduced by the Packet Hacking Village, suggesting a background deeply rooted in practical network and security challenges. Tahoun's presentation style emphasizes real-world problems faced by practitioners, often challenging conventional wisdom in the security industry.

Ez Tahoun brought two industry practitioners to provide validation for the problems he discussed:

Harry Sarwari is the CIO at SAP NS2, where he supports the Department of Defense. He oversees a substantial $4.5 billion project, indicating significant experience in managing large-scale, complex, and high-stakes IT and security environments, particularly within the defense sector.

George Alra serves as the CISO for Ruby Life, a company that operates online dating platforms, including its flagship brand, Ashley Madison. George humorously describes his role as helping people "get securely laid." His career began as a security operations analyst, where he gained firsthand experience dealing with traditional SIEM systems like ArcSight, making his insights into alert fatigue and the need for context particularly relevant.