Attributions for ML-based ICS Anomaly Detection: From Theory to Practice

Attributions for ML-based ICS Anomaly Detection: From Theory to Practice

Speakers: Clement Fung

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=vLmO3Ii4sDA

Overview

Industrial Control Systems (ICS) form the bedrock of critical infrastructure, orchestrating vital processes from power generation to water treatment. The integrity of these systems is paramount, as evidenced by historical incidents like Stuxnet and attacks on the Ukrainian power grid. Machine learning (ML) based anomaly detection offers a promising real-time defense mechanism, learning normal ICS behavior to flag deviations indicative of an attack. However, a significant limitation of current ML-based solutions is their inability to provide actionable context: they merely signal that an anomaly has occurred, not where or why. This lack of interpretability severely hampers operators' ability to diagnose root causes and mount effective responses.

This talk, "Attributions for ML-based ICS Anomaly Detection: From Theory to Practice," presented by Clement Fung at the NDSS Symposium, addresses this critical gap. The research systematically investigates the efficacy of attribution methods in pinpointing manipulated sensors or actuators during an ICS attack. While prior work hinted that simple reconstruction error rankings might suffice, this study provides a comprehensive evaluation across diverse attack scenarios, detection models, and attribution techniques, challenging previous assumptions and offering practical guidance.

The core of this research revolves around two key questions: Can attribution methods accurately identify the manipulated feature in an ICS attack, and which method performs best? Furthermore, how do intrinsic properties of ICS attacks, such as detection timing, manipulation magnitude, or the type of component under attack, influence attribution accuracy? The findings provide a foundational understanding for developing more interpretable and actionable ML-based security solutions for critical infrastructure, ultimately enhancing incident response capabilities.

Background

▶ Watch: Introduction: Problem of ICS anomaly detection attribution (0:00)

Industrial Control Systems (ICS) are complex cyber-physical environments where sensors gather data from physical processes, and actuators execute commands to control them. Attacks on ICS often involve adversaries infiltrating the system to perform false-data-injection attacks, manipulating sensor or actuator values to induce harmful physical outcomes. This research considers an attacker model where legitimate readings are replaced with curated, malicious values over a period.

To evaluate anomaly detection and attribution methods, the study utilized a blend of real-world and synthetic datasets. SWaT (Secure Water Treatment) and WADI (Water Distribution) are public datasets containing actual manipulation-based ICS attacks, complete with ground-truth labels for manipulated components and attack objectives. To further diversify the anomaly set and enable controlled experimentation, the researchers generated 89 synthetic anomalies using a public MATLAB simulator of the Tennessee Eastman Process (TEP), a well-known chemical process model. These synthetic manipulations were systematically introduced at varying magnitudes (+2, -2, +3, and +5 standard deviations from the benign mean) to ensure physical realizability and facilitate systematic analysis of manipulation properties.

The research assumes a partially secured ICS environment where operators have full white-box access to the ML-based anomaly detection models. Two primary categories of anomaly detection methods were evaluated:

1. Statistical Methods:

• PASAD (Process-Aware Stealthy Attack Detector): This method compresses time-series signals into a lower-dimensional subspace. Anomalies are detected when new inputs project too far from the subspace's centroid, scoring anomalies based on a "departure score."
• Auto-Regressive (AR) models: These linear models predict future process values from previous ones, with each feature modeled independently. Anomaly scores are derived from the cumulative sum of prediction errors.

1. Deep-Learning-Based Methods: These unsupervised models are trained to predict the next ICS state from a window of prior states. Anomaly detection occurs when the total difference between the predicted and observed state (i.e., reconstruction error) exceeds a predefined threshold. This unsupervised approach is preferred in ICS due to the rarity and difficulty of generalizing explicit attack labels. Three common architectures were evaluated:

• Convolutional Neural Networks (CNNs): Utilized one-dimensional convolutional kernels to learn temporal patterns.
• Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTMs): Both are recurrent neural networks, adept at learning longer-term dependencies and maintaining states over time.

Attribution methods are techniques designed to quantify the impact of each input feature on a model's prediction. In the context of ICS anomaly detection, this translates to identifying which sensor or actuator input is most responsible for an anomaly detection. These methods are broadly categorized:

1. Black-box Attribution Methods: These methods estimate model behavior by repeatedly querying the model, perturbing inputs, and observing outputs.

• LIME (Local Interpretable Model-agnostic Explanations): Fits a local linear model and uses feature coefficients as attribution scores.
• SHAP (SHapley Additive exPlanations): Employs Shapley values from game theory to assign attribution scores, providing a fair distribution of impact.
• LEMNA (Local Explanation for Machine learning-based Network Anomaly detection): A security-focused method that fits a combined fused Lasso regression and Gaussian mixture model.

1. White-box Attribution Methods: These methods leverage the internal gradients of the model to determine how predictions change with respect to inputs.

• Saliency Maps (SM): Compute the internal gradient of the quantity of interest (in this case, Mean Squared Error (MSE)) with respect to the input features, quantifying each feature's influence on increasing the MSE.
• Variants such as Smoothed Gradients (SG), Integrated Gradients (IG), and Expected Gradients (EG) enhance saliency maps by modifying gradient computation or input selection, often through noise addition or path integration.

Prior work on ICS anomaly detection attribution is limited. Kravchik and Shabtai (2022) found that raw-error ranking could localize attacks in 8 out of 10 CNN-detected cases on SWaT, but only for a few top-ranked features and a subset of attacks. Hwang and Lee (2021) used SHAP with an LSTM, identifying manipulated features in only two attacks using the top three error features. This research significantly expands upon these prior efforts by evaluating attributions across 156 distinct manipulations from three diverse datasets and considering a wide array of timing and attack scenarios. While alternatives like physics-based or rule-based detection and fault isolation exist, they often demand extensive system knowledge, explicit fault definitions, or struggle with generalization, making data-driven attribution a more practical approach for operators lacking such detailed domain expertise.

Key Findings

▶ Watch: Overview of paper's key findings and contributions (0:50)

The research yielded several critical findings that challenge existing assumptions and provide actionable insights for improving ICS anomaly attribution:

1. Raw Reconstruction Error Ranking is Inadequate: Contrary to prior suggestions, simply ranking features by their raw reconstruction error (MSE) performs poorly for accurate attribution. For deep-learning models, less than half of attacked features were correctly identified by the highest-error feature, with AvgRank scores ranging from 0.14 to 0.19. Statistical models performed even worse, indicating that raw error alone is insufficient, especially as it often flags secondary effects as anomalous.
2. Initial ML-based Attribution Performance is Subpar: When computed at the immediate detection time of an anomaly, ML-based attribution methods (like SM, SHAP, and LEMNA) performed similarly to, or even worse than, raw-error ranking. This highlights a significant disparity between theoretical potential and practical performance in real-world, dynamic ICS attack scenarios.
3. Detection Timing is Crucial for Attribution Accuracy: The timing of attribution computation relative to the anomaly's start significantly impacts accuracy. Attributions computed using an input window that starts at the same time as the anomaly ("best-guess timing") significantly outperform those computed at the actual detection time ("practical timing"). This is because later detection points often incorporate noise from secondary system effects, obscuring the initial manipulation.
4. ML-based Attributions Outperform Raw Errors with Optimal Timing: When attributions are computed with inputs aligned with the anomaly's start, ML-based methods, particularly LEMNA, consistently outperform raw-MSE rankings. For example, LEMNA's AvgRank for CNNs dropped from 0.246 to 0.112 with best-guess timing, demonstrating the importance of appropriate input window selection.
5. Attack Properties Significantly Affect Attribution Difficulty:

• Manipulation Magnitude: Lower-magnitude manipulations are statistically more difficult to attribute (Pearson correlation coefficients from -0.214 to -0.605, p < 0.001).
• Feature Type: While ML-based methods performed well on continuous-valued actuators and sensors, they struggled significantly with categorical-valued actuators (AvgRank > 0.248), suggesting a need for type-aware models. Raw-error ranking was generally worse for all actuators than for sensors.
• Multi-point Attacks: Attributing attacks involving simultaneous manipulation of multiple features is generally harder, with LEMNA's AvgRank for multi-point attacks being over three times higher than for single-point attacks.
• Stealthier Manipulations: Summing and linear manipulations, designed to be stealthier, cause later detection (3-5 times later) and significantly worsen attribution accuracy.

1. Ensemble Attribution Methods Offer Superior Performance: An ensemble approach, combining a weighted average of MSE, Saliency Map (SM), and LEMNA attributions, consistently outperforms any individual attribution method. Optimal performance was achieved with a beta-weighted average (β = 2.5) for actuators, computed between 25 and 50 seconds after anomaly detection.
2. Operators Value Even Imperfect Attributions: A survey of ICS operators revealed that even if a single manipulated feature cannot be identified with perfect accuracy, current best-performing attribution methods are perceived as "moderately useful" (average score 3.14). Operators prefer seeing attribution scores for the top 10-20 features to balance accuracy and information, indicating that valuable insights can be derived from imperfect attributions to guide incident response.

Technical Deep Dive

▶ Watch: Background: ICS attack models and anomaly detection methods (2:00)

The methodology for evaluating attribution methods for ML-based ICS anomaly detection was rigorously structured, encompassing data preparation, anomaly detection model implementation, attribution method adaptation, and a novel evaluation metric.

Data Preparation:

The study leveraged three distinct datasets:

• SWaT (Secure Water Treatment): Comprising 32 attacks and 43 distinct manipulations.
• WADI (Water Distribution): Including 15 attacks and 24 manipulations.

These real-world datasets provided labeled attack start/end times and identified manipulated features. To broaden the scope and enable controlled experimentation, the researchers generated 89 synthetic anomalies using a public MATLAB simulator of the Tennessee Eastman Process (TEP). Manipulations were systematically introduced at four magnitudes: +2, -2, +3, and +5 standard deviations from the benign mean of the target feature. This allowed for precise control over manipulation magnitude, defined as the absolute difference between the maximum manipulated value and the benign mean, divided by the benign standard deviation, ranging from 0.06 to over 35 standard deviations. Features were categorized by feature type: sensors, continuous-valued actuators, or categorical-valued actuators.

Anomaly Detection Model Implementation:

The study implemented both statistical and deep learning anomaly detection models, carefully following prior work to ensure fair comparison:

• Statistical Models:
• PASAD: The open-source implementation was used with parameters tuned for each dataset: N=30000, L=5000, r=10 for SWaT/WADI; N=10000, L=5000, r=16 for TEP.
• Auto-Regressive (AR): A linear model was implemented in Python, utilizing p=10 prior states and the Adam optimizer.
• Deep Learning Models:
• CNNs, GRUs, and LSTMs were implemented as 2-layer, 64-unit, 50-history models. All were trained with the Adam optimizer and early stopping, ensuring validation error remained below 0.25.

Attribution Method Adaptation:

Both baseline and ML-based attribution methods were adapted for the ICS anomaly detection context:

• Baseline Attribution (Raw-error ranking): This method, previously suggested in literature, ranks features based on their per-feature prediction error. For AR and deep-learning models, this was calculated as the squared difference between the observed and predicted value: (xj - xj_hat)^2. For PASAD, it was the squared departure distance.
• Adapted ML-based Attribution Methods:
• Counterfactuals: Additive and subtractive counterfactuals were implemented, using Mean Squared Error (MSE) as the quantity of interest. These methods perturb feature values and measure the resulting change in MSE.
• LIME, SHAP, and LEMNA: These black-box methods, originally for classification/regression, were adapted by treating the anomaly detection task as a single-output regression problem. For a given input, the feature with the highest prediction error was identified, and its LIME, SHAP, or LEMNA attributions were computed.
• Saliency Map (SM): For white-box methods, SM computes the internal gradient of the MSE with respect to the input features, quantifying each feature's influence on increasing the MSE. Other white-box variants like Smoothed Gradients, Integrated Gradients, and Expected Gradients were also evaluated but did not outperform SM for this specific application.

Evaluation Metric: AvgRank:

To quantitatively compare attribution methods across diverse datasets, the researchers proposed AvgRank. For each anomaly, features were sorted by their attribution scores in descending order. The proportional placement of the manipulated feature within this sorted list was recorded. AvgRank is the average of these proportional rankings across all anomalies. A lower AvgRank indicates better performance; for instance, an AvgRank of 0.2 signifies that the manipulated feature is, on average, ranked within the top 20% of features. This metric effectively captures both descriptive sparsity (identifying a small set of features) and completeness (consistent performance across diverse inputs).

Initial Attribution Performance (RQ1):

Initial evaluations of raw-error ranking (Table III) revealed that for all models, less than half of attacked features were correctly identified by the highest-error feature. Deep-learning models yielded AvgRanks between 0.14 and 0.19, while statistical models performed significantly worse (AvgRank > 0.3), likely due to their independent feature modeling being prone to flagging secondary effects. Surprisingly, ML-based attribution methods, when computed at the initial detection time (Figure 2), performed worse than raw-error ranking for deep-learning models, with CNN's MSE AvgRank at 0.187 compared to over 0.2 for SM, SHAP, and LEMNA. This highlighted a critical timing issue.

Effect of Detection Timing (RQ2):

ICS anomaly detection operates on time-series data, and the timing of attribution is paramount. As shown in Figure 3, a GRU model detecting SWaT attack #10 often triggered alerts over 100 seconds after the initial manipulation, due to the system's response and accumulating errors. Computing attributions at this late stage led to many features appearing anomalous due to secondary effects, hindering accurate pinpointing. Detections varied significantly (Table IV): early (0-49 timesteps), slightly late (50-99), or very late (100-end). Figure 4 demonstrated that attributions computed within 50 seconds of anomaly start often performed the worst, while those 50-100 seconds performed best.

To address this, two timing strategies were compared: "practical timing" (input window immediately preceding detection) and "best-guess timing" (input window starting at anomaly start). Figure 5 illustrates that best-guess timing significantly outperformed practical timing in 10 out of 12 cases. For example, LEMNA's AvgRank for CNNs improved from 0.246 to 0.112 with best-guess timing, making it the best-performing individual method under optimal timing.

Effect of Attack Properties (RQ2):

Statistical tests (Table V) revealed strong correlations between attack properties and attribution accuracy:

• Manipulation Magnitude: A statistically significant relationship (Pearson correlation coefficients from -0.214 to -0.605, p < 0.001) showed that lower-magnitude manipulations were consistently more challenging to attribute.
• Feature Type: Raw-error ranking performed better for sensor-based attacks (AvgRank 0.152-0.162) than actuator-based attacks (AvgRank > 0.306). However, ML-based methods excelled on continuous-valued actuators (AvgRank < 0.060) and sensors (AvgRank < 0.198) but struggled most with categorical-valued actuators (AvgRank > 0.248), suggesting that treating categorical features as continuous poses a challenge.
• Multi-point Attacks: These were generally more difficult, with LEMNA's AvgRank for multi-point attacks being over three times higher than for single-point attacks.
• Stealthier Manipulations: Summing and linear manipulations (Table X) resulted in attacks being detected 3-5 times later and significantly worsened attribution accuracy across all methods.

Ensemble of Attribution Methods:

Given the varying strengths of individual methods, an ensemble attribution method was developed (Sec. IV-D). This ensemble uses a weighted average of normalized MSE, Saliency Map (SM), and LEMNA attributions. For sensors, a raw average was used. For actuators, a beta-weighted average (β) was applied, giving higher weight to ML-based methods. Figure 6 demonstrates that for actuators, the optimal β value (found to be 2.5, computed 25-50 seconds after detection) significantly outperformed a raw average (β=1), consistently achieving the lowest AvgRank across most timesteps. This ensemble effectively combines the initial accuracy of raw errors with the improving accuracy of ML-based methods over time.

Demo / Proof of Concept

▶ Watch: Explanation of black-box and white-box attribution methods (4:00)

While the talk did not feature a live, interactive demonstration of an attack and real-time attribution, the research itself constitutes a robust proof of concept through its systematic experimental validation framework and the creation of supporting open-source artifacts. The core of this practical demonstration lies in the meticulous design and execution of the methodology.

A key aspect of this "proof of concept" was the modification and utilization of a public MATLAB simulator for the Tennessee Eastman Process (TEP). This simulator was instrumental in generating a controlled environment for well-defined manipulations. By systematically introducing 89 synthetic anomalies at precise magnitudes (+2, -2, +3, and +5 standard deviations), the researchers were able to create a diverse yet controlled set of attack scenarios. This controlled generation allowed for a detailed analysis of how attack properties, such as manipulation magnitude and feature type, directly impact attribution accuracy, which would be challenging to achieve with only real-world, often opaque, attack data.

Furthermore, the study's comprehensive evaluation across 286 synthetic ICS anomalies, in addition to real-world attacks from SWaT and WADI, served as the practical validation of the proposed attribution techniques. The detailed analysis of AvgRank scores, detection timings, and the performance of individual and ensemble attribution methods across these diverse anomalies provided empirical evidence for the research findings. The development of a novel AvgRank metric also served as a practical tool for quantifying and comparing attribution performance in a meaningful way, demonstrating its utility in evaluating the effectiveness of various techniques.

Finally, to support future research and practical application, the team created an open-source library for attribution methods. This library, along with the modified ICS simulator and the dataset of 286 synthetic anomalies, serves as tangible artifacts that allow other researchers and practitioners to reproduce the findings, test new methods, and build upon this work. These resources collectively act as the practical demonstration and validation of the theoretical underpinnings and empirical results presented in the talk.

Defensive Implications

▶ Watch: Methodology begins: datasets and experimental setup (6:00)

The findings of this comprehensive research offer several critical implications and actionable recommendations for security practitioners and researchers working to defend Industrial Control Systems.

For Practitioners:

1. Re-evaluate Real-time Attribution Workflows: The intuitive approach of computing attributions immediately upon anomaly detection is suboptimal. The research clearly demonstrates that attribution accuracy is significantly compromised when attributions are computed too early or too late relative to the anomaly's true start. Practitioners should consider attributions in workflows beyond immediate real-time detection, incorporating post-hoc analysis. Leveraging data historians and applying an input window that closely follows the anomaly's start (e.g., 25-50 seconds after detection, as suggested by the optimal ensemble timing) can provide significantly more accurate insights. For attacks missed by real-time systems, domain expertise can guide the exploration of optimal timings for post-hoc attribution.
2. Adopt an Ensemble Attribution Strategy: No single attribution method is a "silver bullet." The research strongly advocates for using an ensemble of attribution methods. A weighted ensemble combining raw reconstruction error (MSE), Saliency Maps (SM), and LEMNA consistently outperforms individual methods. Specifically, for actuators, a beta-weighted average with β = 2.5 performed optimally. Implementing such an ensemble provides more robust and accurate insights, reducing reliance on any single potentially flawed method.
3. Understand and Plan for Attribution Challenges: Defenders must be aware of the inherent difficulties in attributing certain types of attacks. Low-magnitude manipulations, categorical-actuator-based attacks, multi-point attacks, and stealthier manipulation patterns (like summing or linear changes) are significantly harder to attribute accurately. Security teams should develop specific playbooks or enhanced diagnostic procedures for these challenging scenarios, potentially integrating additional context from process engineers or physical monitoring.
4. Provide Contextual, Actionable Information to Operators: Even imperfect attributions are valuable. The survey of ICS operators confirmed that attributing anomalies to a smaller set of features (e.g., top 10-20 features) is "moderately useful" and provides a crucial starting point for investigation. Operators need contextual information to reduce diagnostic time and effectively respond. Anomaly detection systems should present attribution scores in an interpretable format, highlighting the most suspicious features rather than just binary alerts.
5. Consider Feature Types in Model Design: The significant difference in attribution accuracy between sensors, continuous actuators, and especially categorical actuators, suggests that future anomaly detection and attribution models should differentiate and handle various feature types appropriately. Treating categorical variables as continuous can lead to poor attribution.

For Researchers:

1. Focus on ICS-Specific Attribution Methods: Generic ML attribution methods don't perfectly translate to the unique characteristics of ICS. Future research should design attribution methods specifically for ICS anomaly detection, addressing challenges such as time-dependency, complex feature dependencies, and the distinct nature of sensor versus actuator data.
2. Evaluate on Diverse and Complex Attacks: Current public datasets often lack the diversity needed for robust evaluation. Future studies should prioritize evaluations that include the most challenging attack types identified: low-magnitude, categorical-actuator-based, and stealthy manipulations.
3. Integrate Operator Feedback Early: Attribution method design and evaluation should be guided by operators' preferences and workflows, not solely by top-feature accuracy metrics. This ensures that research outputs are practically useful and adoptable in real-world ICS environments.

By integrating these defensive implications, practitioners can move beyond simply detecting anomalies to actively pinpointing their source, thereby enhancing the resilience and responsiveness of critical infrastructure against sophisticated cyber threats.

Key Takeaways

• Timing is Paramount: The accuracy of ML-based ICS anomaly attributions critically depends on the input window's timing relative to the anomaly's start, with inputs aligned to the anomaly's inception significantly outperforming attributions made at the detection time.
• ML-based Attributions Excel with Correct Timing: When attributions are computed with optimal timing (aligned with anomaly start), ML-based methods like LEMNA consistently outperform simple raw reconstruction error rankings, providing more precise insights into manipulated features.
• Attack Characteristics Impact Attribution Difficulty: Low-magnitude manipulations, attacks on categorical-valued actuators, multi-point attacks, and stealthier manipulation patterns (e.g., summing or linear) are notably more challenging to attribute accurately, highlighting areas for future research and defensive focus.
• Ensemble Methods Provide Superior Robustness: A weighted ensemble approach, combining raw reconstruction error, Saliency Map (SM), and LEMNA, consistently outperforms any single attribution method, offering a more reliable and accurate solution for identifying manipulated features.
• Operator Value in Imperfect Attributions: Even if perfect pinpointing is not achieved, ICS operators find moderately accurate attributions for the top 10-20 features highly valuable as a starting point for incident investigation, significantly reducing manual effort and improving response efficiency.
• Need for ICS-Specific Attribution Design: Generic ML attribution methods are not perfectly suited for ICS. Future research and practical deployments should focus on developing attribution techniques specifically tailored to ICS's unique time-series data, feature dependencies, and operational context.

About the Speaker(s)

The speaker for this presentation was Clement Fung. Additional details regarding their title or affiliation were not provided in the talk transcript or associated metadata.