Detecting Website Intrusion and Account Compromise with Machine Learning

Detecting Website Intrusion and Account Compromise with Machine Learning

Speakers: Robin, Lead Security Machine Learning Practitioner, Meta

Conference: RSAC 2024

YouTube: https://www.youtube.com/watch?v=5AOqYA32sGY

Overview

In an era where cyber threats are increasingly sophisticated and evasive, traditional signature-based detection methods often fall short. This talk, "Detecting Website Intrusion and Account Compromise with Machine Learning," presented by Robin from Meta at RSAC 2024, delves into a robust, machine learning-driven approach to identify and respond to malicious activities targeting web applications. The core premise is that machine learning excels at understanding and identifying deviations from normal behavioral patterns, offering a powerful alternative to static rule sets.

The presentation provides a comprehensive guide, starting from the fundamental concepts of machine learning, progressing through the intricate processes of data collection, feature engineering, model development, and rigorous testing. Robin highlights the critical importance of leveraging diverse log sources and carefully crafting model inputs to achieve high-fidelity detections. The architecture and methodologies described are not theoretical but are grounded in practical application, having been refined through the analysis of billions of events and responses to hundreds of real-world enterprise incidents at Meta.

This approach offers significant advantages for security teams, providing broad threat coverage and an effective prioritization mechanism for incident response. By moving beyond simple indicators of compromise, machine learning allows defenders to detect subtle, behavioral anomalies indicative of advanced intrusion attempts and account compromises, ultimately strengthening an organization's security posture against evolving threats.

Background

▶ Watch: Talk introduction and agenda overview (0:00)

The landscape of cyber security often sees the terms "Artificial Intelligence" (AI) and "Machine Learning" (ML) used interchangeably, but Robin clarifies their distinct roles. Artificial Intelligence is the overarching concept of enabling machines to sense, reason, act, or detect like humans. Machine Learning, on the other hand, is a specific application of AI that empowers systems to automatically extract knowledge from data and learn from it. This distinction is crucial for understanding when and why ML is the appropriate tool for security detection.

Traditional security detection methods, heavily reliant on payload or signature-based analysis (e.g., specific hashes, regular expressions), are highly effective for known threats. However, they struggle against novel attacks or when an adversary employs polymorphic techniques to evade detection. This is where machine learning shines. Robin emphasizes two primary scenarios where ML-based detection is superior:

1. Non-payload or signature-based detection: When the malicious activity doesn't have a distinct, static signature.
2. Behavioral-based detection: Especially when aggregating a broad array of features and activities. ML is particularly adept at learning "normal" behavior patterns from good input data, making it highly effective for identifying anomalies.

The talk categorizes machine learning algorithms into three types, focusing on traditional ML rather than Large Language Models (LLMs):

• Supervised Learning: This is the most common paradigm, where the model learns from a well-labeled dataset. For security, this would mean having many examples (thousands of data points) of both normal and malicious user behaviors, each clearly tagged as a true positive or negative. While ideal for high-quality models, starting with such a dataset is rarely feasible for new threat types due to the scarcity of labeled incident data.
• Unsupervised Learning: This approach deals with unlabeled data, focusing on identifying patterns, clusters, or anomalies within the input without prior knowledge of what constitutes "normal" or "malicious." It's typically the starting point for security ML, as it doesn't require pre-existing incident labels. While initially having the "worst performance" among the three, it provides a baseline.
• Semi-Supervised Learning: A hybrid approach that leverages a small amount of labeled data alongside a larger pool of unlabeled data. This is often the intermediate step, where initial anomalies identified by unsupervised methods are labeled, and then these labels are used to refine the model's understanding of similar unlabeled data points (e.g., through techniques like label spreading).

The recommended progression for implementing ML-based detection is to start with unsupervised learning, gradually gather labeled data (often through adversarial emulation), and then transition to semi-supervised learning, with the ultimate goal of developing a robust supervised learning model once sufficient high-quality labeled data is accumulated. This iterative approach allows organizations to build effective detection capabilities even with limited initial incident data.

Key Findings

▶ Watch: Three types of machine learning algorithms explained (3:00)

The presentation highlights several critical findings and principles for successfully implementing machine learning in security operations:

1. ML for Behavioral Detection is Paramount: Machine learning is not a silver bullet, but it is indispensable for detecting threats that are not payload or signature-based, particularly when focusing on broad behavioral anomalies. It excels at learning normal patterns and identifying deviations, offering coverage traditional methods cannot.
2. Structured ML Development Lifecycle: A well-defined lifecycle, starting from threat modeling and TTP enumeration, through data engineering, model development, and rigorous testing, is crucial. Early engagement with incident response (IR) teams is vital to understand requirements, such as the need for feature importance for explainability, which directly influences model architecture choices.
3. Data Quality and Feature Engineering are King: The quality and relevance of input data ("features") directly dictate model performance. This involves combining 3-6 different upstream and downstream data sources (e.g., application logs, authorization service logs), filtering out noise (e.g., automated banner loads, service accounts), and transforming raw data into meaningful numerical representations. Robin emphasizes that "you can't just throw the kitchen sink at the model and expect it to have good output."
4. The Power of Aggregated Identifiers: Relying on a single identifier (e.g., user ID) for behavioral profiling is insufficient. Aggregating activity across multiple identifiers (e.g., user, IP address, session cookie) provides a much richer and more complete behavioral timeline, enabling the detection of broader attack patterns rather than isolated suspicious events.
5. Iterative Model Improvement: The journey from unsupervised to semi-supervised to fully supervised learning is a practical necessity given the typical scarcity of labeled security incident data. Adversarial emulation and synthetic data generation are critical testing methodologies to validate model performance and generate labeled data before deployment.
6. Feature Selection and Intuition: Not all features are created equal, nor should they all be used. Feature selection is essential to reduce noise, improve model efficiency, and prevent issues from highly correlated features. Additionally, domain expertise and "intuition" regarding which logs and behaviors are genuinely indicative of attacks should guide the feature engineering process.
7. Threat Intelligence Generation: A significant benefit of a well-implemented ML detection system is the automatic generation of internal threat intelligence. By consistently logging model inputs, their mapping to TTPs, and their status (e.g., benign, malicious), organizations can build a valuable repository for their security teams.

Technical Deep Dive

▶ Watch: Recommended progression for ML model development (4:00)

The technical deep dive provided by Robin walks through the entire machine learning development lifecycle for security applications, focusing heavily on data and features, which constitute the majority of the effort.

ML Development Lifecycle

1. Thought Modeling & TTP Enumeration: The process begins by clearly defining the Threat Tactics, Techniques, and Procedures (TTPs) that the model aims to detect. It's crucial to scope models appropriately; for instance, combining authentication-related TTPs (direct login, SSO) into one model is effective, but adding unrelated TTPs like malicious file uploads would dilute the model's focus and introduce too many irrelevant features.
2. Engage Incident Detection and Response (IDR) Team Early: A critical, often overlooked step is involving the IDR team from the outset. Their requirement for explainability (knowing why something was flagged) directly impacts model architecture. If feature importance is needed, only certain model types are viable.
3. Development Phase (Data Domain): This is the most time-consuming phase.

• Log Understanding & Annotation: Deeply understand raw application and infrastructure logs. This often involves annotating or grouping similar error logs (e.g., consolidating 20 error types down to 5-10 meaningful categories).
• Feature Engineering: The process of transforming raw log data into meaningful numerical inputs (features) for the model. This involves creating aggregates, counts, and ratios over various time windows.
• Feature Selection: After generating all possible features, this step involves selecting a subset that are unique, additive, and not highly correlated. Too many features (e.g., 200-300) can overwhelm a model. Prioritize features based on TTP relevance and avoid "throwing the kitchen sink" at the model.

1. Model Development: Setting up AI infrastructure and training the model. This involves experimenting with different model architectures and techniques.
2. Testing:

• Adversarial Emulation: Running targeted red team operations against the application, guided by specific TTPs the model is designed to detect. This helps validate the model's effectiveness against real-world attack simulations.
• Synthetic Data: Generating artificial data that mimics real user behavior and attack patterns to augment training datasets, especially when real incident data is scarce.

1. Staging & User Testing (A/B Testing): Deploying the model in a staging environment or to a small user group to validate outputs, confirm expected behaviors, and ensure the response plan works. Thresholds for escalation to the IR team are also validated here.
2. Continuous Improvement & Threat Intelligence: Once deployed, the system should log model inputs, their mapped TTPs, and detection status. This continuous feedback loop creates a valuable source of threat intelligence for security teams.

Feature Types

Since traditional ML models cannot directly process raw text, features must be transformed into numerical representations:

• Text Features: Raw text data from sources like URL parameters or JSON inputs are converted into embeddings. An embedding is an array of numbers that numerically represents words or characters, allowing the model to understand the semantic relationships between them.
• Categorical Variables: Represent distinct categories, often encoded as numerical values.
• Nominal Variables: Categories without any inherent order. Example: different event types (authentication event, page load event, form submission event) might be encoded as 5, 13, and 17, respectively. The numbers themselves have no relational meaning.
• Ordinal Variables: Categories with a meaningful order or relationship. Example: threat intelligence risk levels (low, medium, high) could be encoded as 1, 2, and 3, reflecting increasing risk.
• Boolean Variables: Binary flags representing true/false or yes/no states. Example: "Has this IP/user been involved in a security investigation before?" (1 for true, 0 for false).
• Numeric Variables: Standard floating-point numbers or integers representing quantitative data (e.g., counts, durations, request sizes).

Detailed Example: Authentication System Model

Robin provides a detailed example of building a real-time model for an authentication system to detect user enumeration and password spraying attacks.

1. Initial Data Exploration:

• Starting with raw logs often reveals issues like null values (missing data), non-unique values for critical fields (e.g., all client interfaces logged as the same string), and a lack of crucial identifiers (e.g., IP address, session cookie).
• Using tools like Pandas in Python is common for data introspection.
• A key observation: a single identifier (e.g., user ID) showing "15 invalid password submissions over the past hour" might be suspicious but not enough for escalation.

1. Importance of Multiple Identifiers: To build a complete behavioral profile, multiple identifiers are crucial. If the same activity is linked to:

• An IP address that has 50 "user does not exist" errors in an hour.
• The same IP address has 20% more request volume than the next highest.
• The same IP address has 500 password submissions over the previous day.
• This combined context tells a much "very different story," indicating a broader attack from a specific device/actor, not just a single user's mistake.

1. Data Augmentation: Combining data from various sources is essential. In the example, error logs were found in a separate table. Unioning these tables based on common identifiers (like email addresses) increases the count of non-null values and provides richer context (e.g., including IP addresses). Derived columns (e.g., count of specific error types per event time) are created to extract granular insights.
2. Final Feature Set & Aggregations:

• Filtering: Service accounts or automated user accounts are filtered out because their behavioral profiles differ significantly from human users.
• Time Periods: Features are aggregated over various time windows (e.g., 7 days in the example). The choice of time window should align with user behavior patterns (e.g., a finance application might need longer lookbacks for month-end activities). Robin suggests the "I over 2DK" rule (e.g., 30, 15, 7, 3 days) for defining multiple aggregation periods.
• Granularity: The final time period chosen depends on the detection's intended real-time (e.g., 1-hour granularity) or asynchronous (e.g., 1-day granularity) execution.
• Feature Selection (Revisited): Even with a rich dataset, not all features should be used. Highly correlated features (those that move together) add little unique information and can be removed. Prioritizing features based on their relevance to high-priority TTPs is also crucial. Finally, human intuition and domain expertise remain invaluable in selecting features that are genuinely indicative of attacks.

Demo / Proof of Concept

▶ Watch: Best practices for combining TTPs in models (5:00)

While the talk provided a comprehensive architectural and methodological framework for building machine learning-based intrusion detection, it did not feature a live demonstration or specific proof-of-concept tool. The focus was on the underlying processes, data considerations, and development lifecycle rather than showcasing a particular implemented system or attack scenario. The examples given were illustrative of data challenges and feature engineering techniques.

Defensive Implications

▶ Watch: Importance of available data and log sources (6:00)

The insights shared by Robin offer several crucial defensive implications for organizations looking to enhance their security posture:

1. Embrace ML for Behavioral Detection: Organizations must move beyond purely signature-based detection for website intrusion and account compromise. Invest in ML capabilities to detect subtle behavioral anomalies that indicate sophisticated attacks, especially those involving user enumeration, password spraying, or other non-payload-centric TTPs.
2. Prioritize Comprehensive Logging: The foundation of effective ML detection is high-quality, comprehensive log data. Ensure that application, infrastructure, and authorization services logs are collected and correlated. Critically, these logs must include multiple identifiers like user IDs, IP addresses, and session cookies to build rich behavioral profiles.
3. Invest Heavily in Data Engineering: Recognize that feature engineering and data preparation are the most time-consuming yet impactful parts of the ML lifecycle. Dedicate resources to understanding raw logs, annotating them, and transforming them into meaningful features. Do not simply feed raw, unfiltered data to models.
4. Engage Incident Response Early and Often: Integrate the IDR team into the ML development process from the very beginning. Understand their requirements for explainability and actionable intelligence, as this will guide model architecture and output design. This ensures that ML detections are not just accurate but also useful for human responders.
5. Implement Robust Testing with Adversarial Emulation: Before deploying models to production, rigorously test them using adversarial emulation (red team operations) and synthetic data. This validates the model's effectiveness against realistic attack scenarios and helps generate valuable labeled data for continuous improvement.
6. Adopt an Iterative ML Strategy: Start with unsupervised learning to identify anomalies, then use a semi-supervised approach as labeled data becomes available, eventually aiming for supervised models. This practical progression allows for continuous improvement even with limited initial incident data.
7. Leverage Model Outputs for Threat Intelligence: Design ML systems to continuously log model inputs, their mapped TTPs, and detection statuses. This automatically generated internal threat intelligence can be invaluable for understanding attack trends, informing defensive strategies, and improving future detection capabilities.
8. Be Mindful of Feature Selection: Avoid over-engineering by including too many correlated or irrelevant features. Employ careful feature selection, guided by TTP prioritization and security intuition, to build efficient and accurate models.

Key Takeaways

• Machine learning is essential for detecting non-signature-based and broad behavioral anomalies in website intrusion and account compromise scenarios, where traditional methods fall short.
• A structured ML development lifecycle is crucial, progressing from initial thought modeling and TTP enumeration, through intensive data engineering and feature selection, to rigorous testing with adversarial emulation.
• Data quality, comprehensive logging, and multi-identifier aggregation are paramount for building effective behavioral profiles and ensuring high-fidelity model inputs.
• Early engagement with incident response (IR) teams is critical to align ML model design with operational requirements, especially regarding detection explainability.
• The path from unsupervised to semi-supervised to supervised learning is a practical strategy for building robust models, especially when initial labeled security incident data is scarce.
• Model outputs, when properly logged and mapped to TTPs, can serve as a valuable source of internal threat intelligence for ongoing security posture improvement.

About the Speaker(s)

The talk was presented by Robin, a Lead Security Machine Learning Practitioner at Meta. While their specific title wasn't explicitly stated in the provided transcript or metadata, Robin's comprehensive understanding and leadership in developing and deploying machine learning-based detection systems for website intrusion and account compromise are evident. Robin has been instrumental in sponsoring and scaling these efforts at Meta for over two years, working closely with various teams including the Enterprise Foundation, Enterprise Product Supplier Research, Cross Meta Security, and the Purple Team. Their expertise spans from fundamental ML concepts and data engineering to model deployment, testing, and integration with incident response workflows within a large enterprise environment.