Red + Blue = Better Security: Adobe's Purple Team Success Story

Red + Blue = Better Security: Adobe's Purple Team Success Story

Speakers: Ivan Koshkin, Detection Engineering Lead, Adobe

Conference: SAINTCON

YouTube: https://www.youtube.com/watch?v=F67RuPbIG-Y

Overview

In this insightful talk from SAINTCON, Ivan Koshkin, Detection Engineering Lead at Adobe, delves into the transformative power of purple teaming, illustrating how a collaborative approach between red and blue security teams can significantly bolster an organization's overall security posture. Koshkin shares Adobe's successful journey in establishing and maturing its purple team function, demonstrating how this synergy moves beyond traditional adversarial dynamics to foster a mutually beneficial relationship. The core premise is simple yet profound: by aligning offensive (red team) and defensive (blue team) capabilities, organizations can achieve more effective threat detection and response, ultimately disrupting adversaries rather than business operations.

The presentation emphasizes the critical role of detection engineering within the blue team, positioning it as the primary interface for purple team interactions. Koshkin meticulously outlines the challenges faced during the initial implementation phase, the refined methodologies that led to optimization, and the tangible outcomes Adobe experienced. This talk is essential for security professionals seeking to bridge the gap between their offensive and defensive security operations, providing a practical blueprint for enhancing an organization's resilience against inevitable cyber threats.

Adobe's experience serves as a compelling case study, highlighting that investing in a robust purple team strategy not only leads to more relevant and effective security controls but also fosters continuous growth and adaptation on both sides of the security spectrum. The speaker’s personal journey from help desk to leading detection engineering underscores the evolving landscape of cybersecurity and the increasing recognition of specialized functions like detection engineering in modern defense strategies.

Background

▶ Watch: Introduction, speaker's background, and talk agenda (0:00)

To appreciate the value of purple teaming, it's crucial to understand the distinct, yet complementary, roles of detection engineering within the blue team and the red team itself, particularly in contrast to traditional penetration testing.

Detection Engineering: The Blue Team's Foundation

Detection engineering is a specialized function within an organization's cyber defense center (often referred to as the blue team). While relatively new as a recognized discipline, its importance has surged as organizations realize the inevitability of compromise. The primary goal of detection engineering is to enable rapid detection and automated response to cyber threats. Koshkin elaborates on its key functions:

1. Research: Identifying detection opportunities, prioritizing them based on risk, and continuously improving instrumentation and detection workflows.
2. Detection Development and Maintenance: Building and maintaining detection rules, focusing on achieving a low false positive rate, ensuring proper instrumentation (e.g., log collection), and supporting security operations with effective dashboards and alerts. This function typically consumes 80-90% of a detection engineer's time.
3. Automation: Developing automated response and remediation capabilities, such as automated power resets, host isolation, artifact gathering, and adding context to alerts. This aims to minimize human intervention and accelerate response.

Without a strong detection engineering function, organizations risk undetected data breaches, discovering compromises only through public news, and a reactive, news-driven security response that erodes customer trust. The guiding principle for detection engineering is to "disrupt threats, not business," balancing rapid response with operational continuity, especially when dealing with sensitive systems.

Red Team vs. Penetration Testing: Clarifying the Offensive Role

Koshkin draws a clear distinction between penetration testing and red teaming, which are often conflated:

• Penetration Testing: Typically short-duration (days to weeks), with a specific, limited scope and predefined targets. It often follows a checklist or consistent methodology, primarily driven by compliance requirements. The goal is to find as many vulnerabilities as possible within a defined scope to facilitate patching.
• Red Teaming: A longer-duration engagement (weeks to months), driven by a specific objective (e.g., exfiltrating a "crown jewel" asset) rather than a checklist. Red teams have an almost unlimited scope, moving across environments (email, endpoints, servers, CI/CD pipelines) as needed to achieve their objective. Their methodology is agnostic, using whatever tools necessary—even developing new ones on the fly—to emulate real-world adversaries. The core purpose is to test an organization's detection and response capabilities against realistic threat scenarios and Tactics, Techniques, and Procedures (TTPs), rather than just identifying vulnerabilities.

Koshkin uses the analogy of a "practice squad" for the blue team: a red team provides invaluable training against a realistic "opponent," preparing the blue team for actual incidents. If red and blue teams operate in silos, the organization misses out on this crucial training benefit, effectively leaving a powerful practice squad on the bench. The lack of interaction between these teams means defensive capabilities are often built on theoretical assumptions rather than validated against real-world attack simulations.

Key Findings

▶ Watch: Detection Engineering's role within the Blue Team (2:40)

Adobe's journey into purple teaming yielded several critical findings and successes, demonstrating the profound benefits of integrated offensive and defensive security operations:

1. Enhanced Detection Relevance and Effectiveness: Prior to purple teaming, Adobe's detection engineering focused on building rules based on general threat intelligence and theoretical scenarios. Through purple team collaboration, detections became specifically tailored to the actual threats and attack patterns relevant to Adobe's unique environment. The red team's emulation provided concrete log data and behavioral insights, allowing the blue team to build highly effective and resilient detections that worked against real-world attack simulations.

1. Improved Blue Team Understanding of Threat Patterns and Attacker Mindset: Detection engineers, while skilled in defensive strategies, are not inherently offensive security experts. Collaborating directly with the red team provided invaluable exposure to the adversary mindset. This direct interaction helped blue team members understand how attackers think, pivot, and exploit weaknesses, leading to the development of more sophisticated and robust threat detection logic. This cross-pollination of knowledge is crucial in an industry where no single individual can be an expert across all disciplines.

1. Parallel Growth and Continuous Improvement: The purple team framework fostered a dynamic of continuous improvement for both teams. As the blue team developed better detections, the red team was compelled to become more stealthy, creative, and adaptive in their operations to avoid detection. This constant push-and-pull mechanism resulted in both teams maturing their capabilities simultaneously. The blue team's detections became more robust, and the red team's emulation became more realistic and challenging, creating a virtuous cycle of security enhancement.

1. Overcoming Foundational Challenges: Adobe successfully navigated common hurdles in establishing purple teaming, including:

• Cultural and Mindset Clash: Resolved by establishing clear communication, shared Objectives and Key Results (OKRs), and joint metrics, transforming an adversarial dynamic into a collaborative one.
• Resource Constraints: Addressed by developing a clear prioritization framework that effectively integrated red team recommendations into the detection engineering backlog alongside other critical tasks.
• Communication Issues: Solved by standardizing information sharing through a dedicated, internal GitHub repository, ensuring all necessary metadata from red team operations was centrally documented and easily accessible.
• Leadership Buy-in: Secured by demonstrating early Return on Investment (ROI) through quick wins, such as identifying true positives and improving detection coverage, which showcased the tangible benefits of collaboration.

These findings collectively underscore that a well-implemented purple team strategy moves beyond theoretical security improvements, delivering concrete, measurable enhancements in an organization's ability to defend against sophisticated cyber threats.

Technical Deep Dive

▶ Watch: Key day-to-day functions of Detection Engineering (3:15)

Adobe's purple team success story is built upon a structured methodology that addresses common challenges and leverages the strengths of both red and blue teams. The journey was categorized into three phases: crawl, walk, and run, each tackling specific aspects of integration and optimization.

Crawl Phase: Addressing Initial Challenges

The initial phase focused on identifying and resolving fundamental challenges that often hinder red and blue team collaboration:

1. Cultural and Mindset Clash:

• Problem: Red teams traditionally aim to "break stuff" and generate noise to highlight vulnerabilities, while blue teams strive for stability, low noise, and preventative measures. This often led to red team operations creating more work for blue teams without perceived mutual benefit.
• Solution: Adobe fostered clear communication and established shared goals and OKRs between both teams. Metrics such as Mean Time To Detect (MTTD) and true positive rates became joint targets, incentivizing collaboration. When both teams contribute to improving these metrics, both "win," aligning their objectives.

1. Resource Constraints:

• Problem: The detection engineering team already had a substantial backlog of tasks from incidents, research, and requests from other product teams. Red team recommendations added to this, leading to conflicting priorities and potential burnout.
• Solution: A clear prioritization framework was developed. This framework allowed red team content to be objectively assessed and placed within the detection engineering backlog based on its relevance, impact, and urgency, ensuring that high-value red team recommendations were appropriately prioritized for sprint planning.

1. Communication Issues:

• Problem: Information sharing was fragmented, occurring across various platforms like Slack DMs, random channels, emails, and disparate wikis. This made it difficult for detection engineers to find the necessary details from red team operations to build effective detections.
• Solution: Adobe standardized information sharing by creating an internal GitHub repository. This centralized platform served as the single source of truth for documenting red team operations. The red team was mandated to log specific metadata throughout their multi-month operations, including details about IP addresses used, process executions, specific tools deployed, and the TTPs employed. This structured documentation allowed the blue team to easily access granular data for detection development.

1. Leadership Buy-in:

• Problem: Leadership often requires a demonstrable Return on Investment (ROI) before allocating significant resources or time to new initiatives like purple teaming, especially if it diverts from existing OKRs.
• Solution: The teams focused on securing initial "quick wins." Red team would identify low-hanging fruit or conduct operations that led to immediate, clear improvements in detection coverage or the identification of true positives. These successes were then widely shared with leadership and across the organization, demonstrating the tangible benefits and positive feedback generated by the collaboration, thereby encouraging further investment.

Walk Phase: Optimizing Methodologies – The Purple Team Cycle

Once initial challenges were overcome, Adobe established a refined, iterative purple team collaboration cycle:

1. Red Team Operation: The cycle begins with the red team conducting an objective-driven operation, which can span weeks or months. During this period, they meticulously document their actions and observations in the shared GitHub repository.

1. Purple Team Collaboration Session: Upon completion of the red team operation, a dedicated collaboration session is scheduled. This deep-dive meeting involves both teams reviewing the operation, discussing the specific threat actor being emulated (e.g., e-crime actor, script kiddie, Advanced Persistent Threat (APT)), and breaking down the patterns of their behavior. This session is crucial for the blue team to gain a high-level understanding of the attack chain and the adversary's intent.

1. High-Quality Detection Recommendations: Following the session, the red team formally submits detailed detection recommendations. These recommendations adhere to the pre-established criteria for submission, ensuring they contain all necessary metadata (e.g., specific TTPs, relevant IoCs, observed log patterns from systems like Splunk forwarders, SIS logs, or Windows ED logs) for the detection engineering team to act upon without further back-and-forth.

1. Internal DE Sprint Planning: The detection engineering team integrates these red team recommendations into their sprint planning. Given the high relevance and real-world validation provided by red team emulation, these items are often prioritized highly within the backlog.

1. Production Deployment and Validation: After developing and testing the new or improved detections, often with the red team's assistance for re-emulation and real-time validation, they are deployed across the enterprise. The red team can "re-emulate that threat and then see that detection work or not work in real time," providing an immediate feedback loop for tuning.

1. Continuous Measurement: Post-deployment, the effectiveness of the new detections is continuously measured against metrics like MTTD and true positive rates, ensuring positive outcomes and identifying areas for further refinement.

This iterative cycle ensures that detection capabilities are constantly tested, validated, and improved against realistic threats, moving beyond theoretical assumptions to evidence-based security enhancements.

Demo / Proof of Concept

▶ Watch: Goals and benefits of optimized Detection Engineering (4:05)

While the talk does not detail a live, explicit demo within the presentation itself, the entire red team operation at Adobe functions as a continuous, large-scale Proof of Concept (PoC) for the detection engineering team. The speaker emphasizes that a key benefit of purple teaming is the ability to "validate detections are working against real-world attack scenarios and they're not just theories."

During the "Walk Phase" of Adobe's purple team cycle, the red team's objective-driven operations inherently serve as the demonstration. They execute specific TTPs and utilize various tools and techniques to achieve their goals, generating real-world log data and system behaviors. The blue team then uses this "live" attack data to test their existing detections or develop new ones. The immediate feedback loop, where the red team can "re-emulate that threat and then see that detection work or not work in real time," is the ultimate demonstration of a detection's efficacy. This process allows the blue team to identify blind spots (e.g., servers lacking proper logging like Splunk forwarders, SIS log, or Windows ED logs), tune out false positives by understanding benign patterns, and confirm that their theoretical detection logic holds up against actual adversary behavior. Therefore, the red team's ongoing engagements are not just simulations, but practical, continuous demonstrations of an organization's defensive capabilities under pressure.

Defensive Implications

▶ Watch: Consequences of lacking a Detection Engineering function (5:15)

The insights from Adobe's purple team journey offer critical defensive implications for any organization serious about enhancing its cybersecurity posture:

1. Establish a Formal Purple Team Function: Organizations should actively work to integrate their red and blue teams into a formal purple team structure. This means moving beyond siloed operations to foster intentional, structured collaboration with shared goals and metrics. If a red team exists but isn't actively collaborating with the blue team, a significant defensive advantage is being missed.

1. Prioritize Detection Engineering: Recognize detection engineering as a core, strategic function within the blue team. Investing resources into this area is crucial for developing the capabilities needed for rapid detection and automated response, which are essential for minimizing business impact during a compromise.

1. Standardize Communication and Documentation: Implement clear, standardized channels for information sharing between red and blue teams. Adopting a centralized platform, such as an internal GitHub repository or similar knowledge base, for documenting red team operations, including TTPs, tools used, IP addresses, and process executions, is paramount for efficient detection development.

1. Develop a Robust Prioritization Framework: Create a transparent prioritization framework for the detection engineering backlog. This allows for objective assessment and integration of red team recommendations alongside other critical tasks, ensuring that high-value, real-world validated detection opportunities are addressed promptly.

1. Secure Leadership Buy-in through ROI: Actively seek and maintain leadership buy-in by consistently demonstrating the Return on Investment (ROI) of purple team efforts. Highlight tangible successes, such as improved Mean Time To Detect (MTTD), reduced false positive rates, identification of critical blind spots, and enhanced coverage against specific MITRE ATT&CK techniques.

1. Focus on MITRE ATT&CK Coverage Gaps: Leverage frameworks like MITRE ATT&CK to strategically guide red team operations. By identifying specific coverage gaps in existing detections, red teams can focus their emulations on those areas, providing targeted data for blue team improvement. This ensures that purple team efforts are tactical and address the most pressing defensive needs.

1. Continuous Validation and Measurement: Implement a culture of continuous validation where detections are regularly tested against emulated threats. Regularly measure key metrics like MTTD and true positive rates to track progress and identify areas for further optimization. This iterative process ensures that defensive capabilities evolve in response to a dynamic threat landscape.

1. Comprehensive Logging and Instrumentation: Actively work to eliminate blind spots in logging. Ensure that all critical systems, including those in less-monitored corners of the environment, have adequate instrumentation (e.g., Splunk forwarders, SIS log, Windows ED logs) to capture the necessary telemetry for detection. Red team operations are excellent for revealing these gaps.

By adopting these defensive implications, organizations can transform their security operations from a reactive posture to a proactive, continuously improving defense system, better equipped to face sophisticated adversaries.

Key Takeaways

• Mutual Benefit is Key: The collaboration between red and blue teams through purple teaming is a profoundly mutually beneficial relationship that significantly enhances an organization's overall security posture.
• Detection Engineering is Central: A robust detection engineering function, focused on research, development, maintenance, and automation, is critical for rapid threat detection and effective response.
• Red Team as a Practice Squad: Red team operations serve as an invaluable "practice squad" for the blue team, validating detection capabilities against realistic, objective-driven attack scenarios and real-world TTPs.
• Structured Collaboration Overcomes Hurdles: Overcoming initial challenges like cultural clashes, resource constraints, communication issues, and leadership buy-in is achievable through clear communication, shared goals, standardized processes (e.g., a shared GitHub repository for metadata), and demonstrating ROI.
• Continuous Improvement Cycle: Establishing an iterative purple team cycle, from red team operations and post-op collaboration to detection development, deployment, and continuous measurement, drives parallel growth and maturity for both offensive and defensive teams.
• Strategic Use of Frameworks: Leveraging frameworks like MITRE ATT&CK can guide purple team efforts, helping to identify and address specific detection coverage gaps strategically.

About the Speaker(s)

Ivan Koshkin is the Detection Engineering Lead at Adobe, bringing a wealth of experience from various facets of cybersecurity. He began his professional journey in 2016 in help desk support, transitioning through several years of system administration before moving into security engineering and compliance at UVU. Koshkin then joined Adobe's Security Operations Center (SOC) and identified an opportunity to establish and specialize in the detection engineering function, where he has been instrumental ever since. He is a recognized speaker, having presented at SAINTCON (previously on the detection engineering lifecycle) and B-sides, and was featured on the Cyberwire podcast earlier this year. On a personal note, Ivan enjoys traveling, with upcoming trips to Japan and Paris, and is an avid video game player, often logging thousands of hours in games like League of Legends. He also enjoys outdoor activities. A fun fact shared by Koshkin is that English is his second language, as he is a first-generation American.