Modernizing Incident Response Using Techniques that Scale

Modernizing Incident Response Using Techniques that Scale

Speakers: Eric Capuano, Co-founder, Recon Infosec; Whitney Champion, Co-founder, Recon Infosec

Conference: Security Fest

YouTube: https://www.youtube.com/watch?v=Znl7TBFAz9E

Overview

In the rapidly evolving landscape of cyber threats, traditional incident response (IR) methodologies often fall short, leaving organizations vulnerable to fast-moving adversaries. This talk, "Modernizing Incident Response Using Techniques that Scale," presented by Eric Capuano and Whitney Champion, addresses these critical shortcomings by introducing a powerful, open-source-driven framework designed to dramatically accelerate and scale IR operations. The speakers, drawing from their extensive experience in security operations, digital forensics, and building a global managed detection and response (MDR) provider, Recon Infosec, highlight the inefficiencies prevalent in many IR teams today and propose a concrete, actionable solution.

The core premise of their presentation is that world-class incident response doesn't have to be prohibitively expensive or overly complex. Instead, by strategically leveraging a suite of free, open-source tools—Velociraptor, Hayabusa, Plaso, Time Sketch, and Open Relic—teams can achieve unprecedented speed, repeatability, and reliability in their investigations. Capuano and Champion not only detail the capabilities of each tool but also demonstrate how they can be seamlessly integrated through a custom "Open Relic Pipeline" to create an automated, end-to-end forensic acquisition and analysis workflow, transforming what once took days into a matter of minutes. This approach empowers defenders to keep pace with threat actors and ensures comprehensive remediation, even in the most challenging breach scenarios.

Background

▶ Watch: Introduction to modernizing incident response and talk goals (0:00)

The speakers begin by dissecting the significant challenges plaguing contemporary incident response teams, many of which stem from outdated practices and dependencies. A primary issue is the prevalence of bespoke, custom approaches to IR. Many teams rely on a collection of "random scripts that some guy wrote six years ago," leading to inconsistent methodologies, inefficiencies, and a lack of repeatability. This "duct tape and bubble gum" approach hinders scalability and reliability, making it difficult to onboard new analysts or ensure consistent quality across investigations. Eric Capuano notes that if you ask five IR professionals how they do their job, you'll get seven different answers, highlighting the lack of standardized, best-practice-driven processes. Legacy tools, such as using Excel for timeline analysis, further exacerbate these inefficiencies.

Another critical problem identified is the heavy dependence on commercial third-party tooling. Many IR teams feel they can only perform effective incident response if they have top-tier EDR or SIEM products like CrowdStrike or SentinelOne. While these tools offer value, an over-reliance on them turns IR analysts into "tool operators" rather than holistic practitioners. This creates several vulnerabilities: if the tool is unavailable, malfunctioning, or inaccessible (as one team experienced, wasting "many, many, many hours"), the IR process grinds to a halt. Furthermore, commercial tools are often expensive, leading organizations to deploy them on only a fraction of their environment (e.g., 40%), leaving significant blind spots. The need to "call a sales rep" for more licenses during an active incident is an untenable situation.

Finally, dependencies on other teams introduce significant bottlenecks. Even a highly skilled IR team can be hampered if they lack direct control over critical infrastructure like firewalls, relying instead on IT or network engineering teams with different priorities, hours, and Service Level Agreements (SLAs). In a fast-paced battle with a threat actor, this loss of time is unacceptable, as adversaries are often "16 hours ahead" and operate with extreme urgency. The speakers emphasize that removing these dependencies is crucial for achieving the speed required to counter modern threats.

Beyond these operational issues, a fundamental flaw in many IR processes is a lack of efficiency measurement. While most IR professionals are familiar with academic frameworks like the six-step "picker process" (Preparation, Identification, Containment, Eradication, Recovery, Post-Incident Activity), the actual time spent in the critical "Identification," "Containment," and "Eradication" phases is often protracted. Capuano stresses the importance of measuring the performance of the IR process, as unmeasured processes cannot be improved. With threat actors moving faster than ever—Mandiant's dwell time report showing an average of 11 days, down from over 400—IR teams simply cannot afford to measure their response time in "days, if not weeks." The ability to quickly scope a breach and move to eradication is paramount; shortcutting this process only leads to an endless game of "whack-a-mole" where persistence mechanisms are missed, and the threat actor inevitably returns.

One of the most significant technical hurdles identified is the fallacy of requiring full disk images for every forensic investigation. While relevant for law enforcement or criminal investigations, enterprise IR rarely necessitates capturing multi-terabyte disk images. This practice, often a relic of "ancient ways of doing forensics," adds days to an investigation, creating an "artificial requirement" that drastically slows down response times. Capuano argues that for the vast majority of cases, only a small fraction (e.g., 4-5 GB) of data from a 2 TB disk is forensically critical (registry hives, event logs, prefetch files, etc.). Modernizing to triage acquisition—collecting only the essential forensic artifacts—is presented as a foundational step to achieving the necessary speed.

Key Findings

▶ Watch: Whitney Champion's background and focus on automation (3:30)

The central finding of this talk is that a robust, scalable, and highly efficient incident response capability can be built using a carefully selected suite of open-source tools, integrated into an automated pipeline. The speakers demonstrate that by moving away from bespoke, manual processes and expensive commercial dependencies, organizations can achieve significantly faster response times and more thorough investigations.

The key findings and contributions are:

1. Shift from Full Disk Imaging to Triage Acquisition: The fundamental change in forensic data collection, emphasizing the acquisition of only critical artifacts rather than entire disk images, drastically reduces collection time from days to minutes. This is crucial for keeping pace with modern adversaries.
2. The Power of Open-Source Tools: The talk highlights five specific open-source tools—Velociraptor, Hayabusa, Plaso, Time Sketch, and Open Relic—as the backbone of a modern IR toolkit. These tools are free, powerful, and actively maintained, offering capabilities often exceeding or complementing commercial alternatives.
3. Seamless Integration and Automation: The most significant contribution is the "Open Relic Pipeline," a custom integration developed by Whitney Champion. This pipeline acts as middleware, connecting Velociraptor's acquisition capabilities with Open Relic's orchestration, which then feeds processed data into Plaso, Hayabusa, and Time Sketch for rapid analysis and collaborative visualization.
4. Dramatic Speed Improvement: The integrated pipeline demonstrates a profound acceleration of the IR process. Benchmark testing showed that collecting event logs via Velociraptor, processing them with Hayabusa and Plaso, and ingesting them into Time Sketch can take as little as "a minute" for a single system, with parallel processing enabling similar speeds across hundreds of systems.
5. Community-Driven Improvement: The reliance on open-source projects like Cape Files encourages community contributions, allowing practitioners to share knowledge (e.g., new log locations like ManageEngine) and collectively raise the bar for forensic data collection.
6. Scalability and Repeatability: The Docker-based architecture of Open Relic and the single-binary deployment of Velociraptor ensure that the entire solution is highly scalable and repeatable, deployable on-prem or in the cloud, capable of handling large-scale breaches by simply allocating more CPU resources.

Technical Deep Dive

▶ Watch: Beginning of the core problem statement for IR (4:30)

The proposed solution hinges on a modular yet integrated suite of open-source tools, each addressing a specific facet of the incident response workflow. The overarching principle is to move from manual, time-consuming processes to automated, rapid triage and analysis.

Triage Acquisition: The Foundation of Speed

The critical first step is the adoption of triage acquisition. Instead of capturing entire disk images, which can take hours or days for a 2 TB drive, triage focuses on collecting only the "critically important data" for forensic analysis. This typically amounts to 4-5 GB of data, including registry hives, event logs (EVTX), prefetch files, browser history, and other volatile data. This approach is not a compromise but a modernization, acknowledging that for most enterprise IR, legal requirements for full disk images are unnecessary.

Velociraptor: The Endpoint Swiss Army Knife

Velociraptor is presented as the primary tool for triage acquisition and endpoint interaction. Created by Mike Cohen (formerly of Google Rapid Response - GRR), Velociraptor is a free, open-source endpoint visibility and response tool that offers:

• Client-Server Architecture: Enables scalable collection from remote systems, even across global networks.
• Rapid Triage: Extremely fast at collecting specific forensic artifacts.
• On-Box Parsing: Can perform forensic data parsing directly on the endpoint, returning only the processed output, further reducing data transfer needs.
• Cross-Platform Support: Runs on Windows, macOS, and Linux.
• Intuitive Web Interface: Simplifies deployment and management of agents and hunts.
• Single Binary Deployment: Both the server and endpoint agents are single binaries, allowing for deployment in "about 5 minutes" across environments of any size.
• VQL (Velociraptor Query Language): A powerful query language for defining custom hunts and data collections.
• Integration with Cape Files: Utilizes open-source target definitions from Eric Zimmerman's Cape Files GitHub repository, which specify the locations of common forensic artifacts. This repository is community-contributed, ensuring its comprehensive and up-to-date nature.
• Automation via Labels: Velociraptor's UI allows analysts to apply labels (e.g., "compromised") to endpoints, which can trigger automated actions (e.g., initiating a triage acquisition) on the backend.

Capuano emphasizes Velociraptor's power, stating it can be used "from A to Z of an entire IR from the triage acquisitions to knife fighting with an adversary," and that it is "more powerful than your really expensive EDR tool" for certain tasks.

Hayabusa: Retroactive Threat Detection

Once event logs are collected (e.g., via Velociraptor), Hayabusa steps in. Developed by Zach Matthysse of Yamato Security, Hayabusa is an incredibly fast, Rust-based event log parsing tool.

• Retroactive Detections: Its primary function is to analyze collected EVTX files and generate threat detections based on a corpus of "about 3,500 threat signatures."
• Sigma Rules: Integrates Sigma rules from Florian Roth's team, along with its own built-in detection logic.
• Speed and Scalability: Written in Rust and multi-threaded, it processes large volumes of event logs rapidly, effectively acting as a "retroactive SIEM" for environments where no SIEM was present or fully logging.
• Output Formats: Generates output in CSV, JSON, or directly to Elastic Search or Splunk, facilitating further analysis.

Hayabusa fills a critical gap, allowing IR teams to quickly identify past malicious activity even without prior logging infrastructure.

Plaso: The Timeline Engine

Plaso (log2timeline) is an open-source forensic tool developed by Kristinn Gudjonsson (formerly of Google) that serves as the "kitchen sink of forensic artifact parsing."

• Consolidated Parsing: It consolidates the functionality of numerous individual forensic tools into one, parsing "anything forensically valuable" from raw event logs, registry hives, prefetch files, and more.
• Super Timelines: Generates comprehensive "super timelines" of every event on a system, providing granular detail about threat actor activities, initial access, and lateral movement.
• Temporal Analysis: Converts disparate forensic artifacts into a unified temporal view, crucial for reconstructing attack sequences.

Plaso is indispensable for building a complete chronological understanding of an incident.

Time Sketch: Collaborative Timeline Analysis

Time Sketch is an open-source web-based tool, created by Johan Bergren (based in Stockholm, Sweden), designed for collaborative timeline analysis and forensics.

• Forensics-Focused UI: Similar to Elastic Kibana but purpose-built for forensic investigations.
• Collaborative Environment: Allows multiple analysts to view, annotate, and investigate timelines together, fostering teamwork during an incident.
• Integration with Plaso and Hayabusa: Ingests the "super timelines" from Plaso and detections from Hayabusa, providing a single pane of glass for all relevant incident data.
• Visualizations: Offers graphing and visualization capabilities to quickly understand activity across systems.

Time Sketch transforms raw data into an interactive, collaborative investigative environment.

Open Relic: The DFIR Orchestrator

Open Relic, also created by Johan Bergren, is the orchestration and automation layer that ties everything together. It's described as "SOAR for DFIR."

• Container-Based Platform: Leverages Docker to containerize forensic tools and define workflows.
• Visual Workflow Builder: Allows users to visually construct and customize playbooks for automated processing. For example, it can be configured to automatically send EVTX files to Hayabusa and Plaso, then export results to Time Sketch.
• Marketplace: Features a growing marketplace of additional tools and capabilities (e.g., Floss, Kapa), with the ability for users to contribute new Docker templates.
• AI Integration: Now includes LLM (Large Language Model) capabilities for "analyst enablement" and "decision enablement," allowing interaction with forensic data using local or cloud-based AI (Claude, ChatGPT).

Open Relic provides the automation necessary to execute complex forensic workflows with minimal manual intervention, dramatically increasing speed and reducing human error.

The Open Relic Pipeline (Whitney's Contribution)

Whitney Champion's "Open Relic Pipeline" project is the crucial middleware that integrates these disparate tools into a single, cohesive, and easily deployable solution.

• Bootstrap Installation: A single install.sh script deploys and configures Velociraptor, Open Relic, and Time Sketch, ensuring they are not only installed but also pre-integrated and ready to communicate.
• Velociraptor Artifacts: Custom Velociraptor artifacts were developed to monitor for completed triage acquisitions, zip the collected files, and then push them to the Open Relic API for processing.
• Automated Workflows: The pipeline leverages Open Relic to define automated workflows that intelligently process different types of collected data (e.g., event logs are automatically sent to Hayabusa and Plaso).
• Scalability: Designed for both on-premise and cloud deployment, its performance scales directly with allocated CPU resources, allowing for parallel processing of hundreds or thousands of systems during a large breach.

This pipeline effectively bridges the gap between individual powerful tools and a fully automated, end-to-end incident response system, making advanced DFIR accessible and actionable for teams without extensive DevOps expertise.

Demo / Proof of Concept

▶ Watch: Problem: Inefficient and bespoke incident response approaches (5:00)

The talk included a live demonstration of the integrated pipeline, showcasing the seamless flow of data from endpoint acquisition to collaborative timeline analysis. The demo environment consisted of a Velociraptor client (a Windows host) connected to a Velociraptor server, which was integrated with Open Relic and Time Sketch.

The demonstration steps were:

1. Velociraptor Client Labeling: A host in Velociraptor was labeled "IR1" to categorize it for the incident.
2. Triage Acquisition Initiation: A specific Cape triage collection was initiated on the labeled host, focusing solely on event logs (EVTX files) for speed, rather than a full disk image.
3. Automated Processing Trigger: In the background, Velociraptor server event monitoring artifacts (part of the Open Relic Pipeline) were configured to watch for the completion of Cape triage collections. Once the event logs were collected and zipped, these artifacts automatically pushed the data to the Open Relic API.
4. Open Relic Workflow Execution: Upon receiving the data, Open Relic automatically kicked off a predefined workflow. For event logs, this workflow included:

• Sending the EVTX files to Plaso for super timeline generation.
• Sending the EVTX files to Hayabusa for retroactive threat detection.
• Exporting the processed outputs (CSV timelines, Hayabusa detections) to Time Sketch.

1. Rapid Results in Time Sketch: Within "less than five minutes," the Hayabusa timeline was visible and ready for analysis in Time Sketch. While Plaso's processing for a full super timeline on a system with only two CPU cores was noted to take longer (up to half an hour), the speakers highlighted that throwing "32 cores at this thing" in a real-world scenario would complete the Plaso processing much faster, allowing parallel execution for many systems.

The demo vividly illustrated the speed and automation benefits. What would traditionally take hours of manual effort—collecting logs, running multiple parsing tools, and then importing results into a visualization platform—was condensed into a few minutes of automated processing. This "0 to 60" capability, powered by the Open Relic Pipeline, makes advanced DFIR accessible and efficient.

Defensive Implications

▶ Watch: Problem: Heavy reliance on commercial IR tools (6:30)

The framework presented by Eric Capuano and Whitney Champion offers profound defensive implications, enabling organizations to dramatically improve their incident response posture:

1. Accelerated Response Times: The most significant benefit is the drastic reduction in the time from identification to eradication. By shifting to triage acquisition and automating the analysis pipeline, IR teams can gain crucial minutes and hours, potentially preventing major data exfiltration or ransomware encryption. This allows defenders to keep pace with, or even outmaneuver, fast-moving threat actors.
2. Cost-Effective World-Class IR: By leveraging powerful, free, and open-source tools, organizations can build a world-class incident response capability without the exorbitant costs associated with commercial EDRs and SIEMs. This democratizes advanced DFIR, making it accessible to smaller organizations or those with limited budgets.
3. Reduced Dependencies and Bottlenecks: The integrated pipeline minimizes reliance on expensive commercial tools and external IT teams. IR teams gain direct control over data collection, processing, and analysis, removing critical bottlenecks that often delay response efforts. This fosters self-sufficiency and agility.
4. Enhanced Scalability and Repeatability: The containerized and script-driven deployment ensures that the IR infrastructure can be rapidly scaled up or down as needed. Whether responding to a single host or a large-scale enterprise-wide breach, the system can parallel-process data, maintaining efficiency. The standardized workflows ensure consistent, high-quality investigations regardless of the analyst.
5. Comprehensive Retrospective Analysis: Tools like Hayabusa enable organizations to perform retroactive threat hunting and detection, even in environments with previously inadequate logging or SIEM coverage. This allows teams to uncover long-dormant threats or understand the full scope of an attack that began before improved monitoring was in place.
6. Improved Collaboration and Knowledge Sharing: Time Sketch facilitates collaborative analysis, allowing multiple analysts to work on the same timelines simultaneously, sharing insights and accelerating the investigation. Furthermore, the open-source nature of tools like Cape Files encourages community contributions, collectively raising the bar for forensic knowledge and artifact collection.
7. Proactive Preparation: The ability to deploy and test this entire pipeline quickly during the "Preparation" phase of IR means teams can be battle-ready before an incident occurs. Practicing with these tools and workflows can significantly improve muscle memory and reduce panic during a real event.
8. Analyst Empowerment and Enablement: Automation frees analysts from repetitive, manual tasks, allowing them to focus on higher-level analytical thinking and decision-making. The integration of LLM capabilities in Open Relic further enhances analyst enablement by providing AI-driven insights into forensic data.

Key Takeaways

• Modern IR requires speed and scalability: Traditional, bespoke methods and reliance on full disk imaging are too slow for today's fast-moving threat actors.
• Open-source tools offer powerful, cost-effective solutions: Velociraptor for rapid triage, Hayabusa for retroactive threat detection, Plaso for comprehensive timeline generation, and Time Sketch for collaborative analysis are core components.
• Automation is critical for efficiency: Open Relic serves as a DFIR orchestrator, automating complex forensic workflows and reducing manual effort.
• Integrated pipelines dramatically accelerate investigations: The "Open Relic Pipeline" seamlessly connects these tools, enabling event log collection, processing, and visualization in minutes, not days.
• Community contribution strengthens defenses: Open-source projects like Cape Files benefit from collective knowledge, ensuring up-to-date artifact collection capabilities for all.
• Decoupling from commercial tools and team dependencies enhances agility: Building an in-house, open-source-driven capability reduces reliance on external factors, empowering IR teams to respond independently and swiftly.

About the Speaker(s)

Eric Capuano is a seasoned information security professional with approximately 15 years of dense, action-packed experience. He began his career in the United States Air Force as a cyber warfare operator. Transitioning to the private sector, Eric supported local government and later co-founded Recon Infosec, a managed detection and response (MDR) provider based out of Austin, Texas, with Whitney Champion. He is passionate about security education, serving as a SANS instructor for seven years and running trainings at conferences like Black Hat, Wild West Hackfest, and various B-Sides events globally. His expertise lies in scaling security operations, incident response, and digital forensics.

Whitney Champion brings over two decades of experience in the security space, having stumbled into it after her college web server was hacked, prompting her to "learn how to do this better." She started her career as a security analyst for the government, working on various projects before transitioning to the startup world. Whitney is the co-founder of Recon Infosec alongside Eric Capuano. Her primary focus in recent years has been on security infrastructure, automation, and orchestration, building tools and solutions that enhance effectiveness and simplify complex security operations. She is instrumental in designing and implementing scalable and repeatable security processes.