Enhance Stealthiness and Transferability of Adversarial Attacks with Class Activation Mapping Ensemble Attack

Enhance Stealthiness and Transferability of Adversarial Attacks with Class Activation Mapping Ensemble Attack

Speakers: Hui Xia

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=1IkzRZYpK10

Overview

This talk introduces the Class Activation Mapping Ensemble Attack (CAM-EA), a novel adversarial attack method designed to significantly enhance the stealthiness and transferability of adversarial examples against Deep Neural Networks (DNNs). Presented by Hui Xia at the NDSS Symposium, the research addresses two critical shortcomings of existing adversarial attacks: their tendency to produce easily perceptible perturbations and their limited effectiveness, particularly in scenarios with a low number of attack iterations or when targeting unseen models (black-box attacks).

The core innovation of CAM-EA lies in its intelligent use of Class Activation Mapping (CAM). By leveraging CAM, the method identifies the specific image regions most critical to a DNN's decision-making process. Perturbations are then strategically applied to these critical regions, rather than indiscriminately across the entire image. This targeted approach dramatically improves the visual imperceptibility of adversarial examples, making them more stealthy, while simultaneously boosting attack efficacy and convergence speed. Furthermore, to overcome the inherent challenge of transferability in black-box attacks, CAM-EA ensembles the CAMs from multiple substitute models, ensuring the generated adversarial examples remain potent across a diverse range of target models without requiring direct knowledge of their internal architecture.

The significance of CAM-EA extends beyond merely creating more effective attacks; it serves as a crucial tool for evaluating and improving the robustness of DNNs, especially in safety-critical applications like autonomous driving and medical imaging. By demonstrating the vulnerabilities of models to highly stealthy and transferable attacks, this research highlights the urgent need for more sophisticated defense mechanisms. CAM-EA's ability to quickly and effectively deceive models, even against various defense strategies, underscores the ongoing arms race in adversarial machine learning and provides a new benchmark for assessing model security.

Background

▶ Watch: Introduction: Challenges in adversarial attacks and CAM-EA proposal (0:00)

The pervasive adoption of Deep Neural Networks (DNNs) across diverse domains, from image classification to natural language processing, has brought to the forefront a critical security concern: adversarial attacks. These attacks involve introducing subtle, often imperceptible, perturbations to legitimate inputs, causing DNNs to make incorrect predictions. The consequences can be severe, ranging from misclassifying stop signs in autonomous vehicles to bypassing facial recognition systems. In response, researchers have developed various defense mechanisms, including adversarial training, defensive regularization, and adversarial example detection. However, adversarial attacks continue to evolve, posing persistent challenges.

Adversarial attacks are broadly categorized based on their methodology and the attacker's knowledge of the target model. Gradient-based attacks, such as the Fast Gradient Sign Method (FGSM), are computationally efficient but often lack transferability. Iterative methods like the Basic Iterative Method (BIM) and Projected Gradient Descent (PGD) generate more accurate adversarial examples by repeatedly applying small perturbations, but they tend to overfit the target model in black-box settings, limiting their utility. Enhancements like Momentum Iterative FGSM (MIFGSM) and Nesterov Accelerated Gradient FGSM (NIFGSM) aim to stabilize updates and improve transferability, but their overall impact often remains limited. Variance-based Moment Iterative FGSM (VMIFGSM) further explores gradient information, yet still faces transferability challenges.

Another class of attacks, input transformation-based attacks, including Diverse Input Method (DIFGSM), Translation-Invariant Attack Method (TIFGSM), and Scale-Invariant Method (SINIFGSM), leverage image invariances through techniques like rotation, translation, and scaling. While these methods offer some improvements in transferability, the overall impact is often not substantial. Admix attack also computes gradients on mixed images to generate more transferable examples, but with limited success.

Model ensemble attacks represent a more direct approach to improving transferability by attacking multiple models simultaneously. Early ensemble methods averaged predictions or integrated logit outputs and losses. However, these methods often suffer from uniform integration, which can lead to local optima, and their efficiency in generating adversarial examples can be slow, as observed with the Stochastic Variance-Reduced Ensemble attack (SVRE).

A critical limitation observed across these existing methods is their tendency to add indiscriminate perturbations to all pixel locations. This leads to several issues:

1. Poor Perceptual Quality: The widespread, untargeted noise makes adversarial examples easily perceptible to humans, compromising their stealthiness.
2. Low Attack Capability: Indiscriminate perturbations are less efficient in influencing the model's decision boundary, resulting in lower attack success rates, especially in scenarios with limited attack iterations (low-round attacks).
3. Limited Transferability: Perturbations optimized for an entire image of a specific model may not generalize well to other models with different architectures or decision boundaries.

The inspiration for CAM-EA stems from the growing field of machine learning interpretability. Tools like Shapley Additive explanations have been shown to be useful in generating adversarial malware. Similarly, Class Activation Mapping (CAM) can effectively reveal the connection between DNN decisions and specific image regions. By understanding why a DNN makes a particular decision, it becomes possible to apply perturbations more strategically, focusing on the most influential pixels. However, the inherent targeted nature of CAM scores could potentially limit transferability if not properly handled. CAM-EA distinguishes itself by not just ensembling model outputs, but specifically ensembling the CAMs of multiple models, thereby allowing for targeted, stealthy, and broadly transferable perturbations.

Key Findings

▶ Watch: CAM-EA's unique contributions: transferability and weighted perturbations (1:08)

The research presents CAM-EA as a pioneering black-box adversarial attack method that simultaneously addresses the critical challenges of attack transferability and perturbation weighting. Its key findings and contributions are multifaceted:

1. First Black-Box Attack Combining Transferability and Perturbation Weighting: CAM-EA is the first method to integrate Class Activation Mapping (CAM) for intelligent, pixel-wise perturbation weighting with an ensemble strategy for enhancing attack transferability in a black-box setting. This dual focus overcomes the trade-offs often seen in prior work, where improvements in one aspect typically came at the expense of another.

1. Significant Enhancement of Stealthiness: By leveraging CAM scores to guide perturbations, CAM-EA focuses changes on the most critical regions of an image, avoiding excessive noise in unimportant areas. This results in adversarial examples that are far less perceptible to human observers. Quantitatively, in non-targeted attacks with comparable attack capability to VMIFGSM, CAM-EA significantly reduces the L2 norm by 24.08, Low_fre by 13.91, and improves SSIM by 0.04 and PSNR by 0.69. This positions CAM-EA's perceptibility notably close to the best-performing TPGD method, which is known for its high visual quality. In targeted attacks, CAM-EA similarly achieves the best perceptibility among eight baseline methods with comparable attack capabilities, improving PSNR by 2.12 against VNIFGSM and by 5.13 against SINIFGSM.

1. Superior Attack Efficacy and Faster Convergence: The targeted perturbation strategy of CAM-EA leads to a dramatic improvement in attack capability, particularly in low-round attack scenarios where computational resources or interaction rounds are limited.

• For non-targeted attacks, CAM-EA consistently demonstrates the most vigorous attack ability and the fastest convergence rate, reaching the attack capability of the most powerful baseline around the 20th epoch, approximately ten epochs faster than other methods. At the 10th epoch, it improves the average attack ability against 13 target models by 11.69% compared to VMIFGSM, and by 37.15% compared to TPGD. It also achieves 17% and 15% higher attack success rates against EfficientNet and RegNet, respectively, compared to VMIFGSM.
• For targeted attacks, CAM-EA exhibits significant advantages in attack capability, convergence speed, and low-iteration attack capability, achieving the fastest convergence and most potent attack efficacy, significantly outperforming eight baseline methods at low attack epochs.

1. Robust Transferability through CAM Ensembling: The method's innovative approach of ensembling the CAMs from multiple substitute models significantly boosts the transferability of the generated adversarial examples. This ensures that the perturbations are effective across a diverse range of unseen target models, which is a cornerstone of practical black-box attacks.

1. Notable Evasiveness Against Defense Mechanisms: CAM-EA demonstrates a strong ability to bypass various state-of-the-art defense methods, including Feature Squeezing, Label Smoothing, and Shan's adversarial training method. While attack capability decreases against defended models compared to undefended scenarios, CAM-EA consistently outperforms baseline methods in these challenging conditions, making it a valuable tool for accurately assessing the true robustness of DNN models.

1. Rigorous Ablation Studies Validate Design Choices: Comprehensive ablation studies confirmed the rationality and effectiveness of using CAM scores as perturbation weights and the modular design. Comparing CAM-EA against uniform, Gaussian, CALM, and Fix perturbation methods, CAM-EA exhibited the strongest attack capability and transferability. Furthermore, the studies clearly demonstrated that the addition of the CAM-based weighting module (Module 3) significantly improves attack performance, which is further enhanced by combining it with the loss function's second term (Module 2).

Technical Deep Dive

▶ Watch: Demonstrated superior performance of CAM-EA across metrics (1:35)

The Class Activation Mapping Ensemble Attack (CAM-EA) is meticulously designed for the black-box attack setting, where an attacker has no access to the internal architecture, parameters, or gradients of the target model, only its input and output predictions. The methodology is built upon the strategic construction of substitute models and the innovative application of Class Activation Mapping.

Overview of Methodology

CAM-EA's core idea is to generate adversarial examples that are both visually stealthy and highly transferable. This is achieved by:

1. Substitute Model Construction: Creating a gradient substitute model to determine the general direction of perturbations and several CAMs substitute models to calculate pixel-wise perturbation weights.
2. CAM-based Perturbation Weighting: Utilizing CAM to compute a class activation score for each pixel. This score serves as an intelligent weight, ensuring that perturbations are concentrated on image regions most critical to the DNN's decision. This targeted approach significantly enhances stealthiness and accelerates attack convergence by rapidly shifting the decision region of benign images.
3. Ensembling CAMs for Transferability: To ensure broad transferability across different black-box target models, CAM-EA integrates the CAMs from multiple CAM-based substitute models. Unlike traditional ensemble methods that average logit outputs or losses, this approach leverages the interpretability of CAMs to create more robust and transferable adversarial examples.

Substitution Model Selection

The efficacy of CAM-EA heavily relies on the appropriate selection of substitute models. These models are chosen based on factors such as complexity, similarity to potential target models, and their ability to generate informative CAMs. All selected models are pre-trained on the ImageNet dataset and sourced from the PyTorch library.

• Gradient Substitute Model: ResNet50 is chosen for its proven effectiveness in determining reliable gradient signs, which dictate the direction of perturbations.
• CAMs Substitute Models: WideResNet101, Inception_v2, and ResNet34 are selected. These models are known for producing larger, more informative heatmap regions, which are essential for effective and generalized perturbation weighting.

Problem Definition

Adversarial attacks are framed as an optimization problem. Given a target model $F$ with parameters $\theta$ and an input image $x$, the objective is to find a small perturbation $\delta$ such that the perturbed image $x' = x + \delta$ yields a significantly different output from $x$, while $\delta$ remains imperceptible to humans. This is mathematically expressed as a constrained minimization problem:

$\min_{\delta} d(x', x) \quad \text{subject to} \quad F(x; \theta) \neq F(x'; \theta)$

Here, $d(x', x)$ represents a distance metric (e.g., L2 norm), and $\epsilon$ is the perturbation constraint, typically set as $||x' - x||_{\infty} \leq \epsilon$.

Non-targeted Attack

For non-targeted attacks, the goal is to maximize the prediction error, causing the DNN to misclassify the input into any incorrect class, without specifying a particular target. The optimization problem is:

$x' = \arg \max L(x', y; \theta) \quad \text{subject to} \quad ||x - x'||_{\infty} \leq \epsilon$

In CAM-EA, the perturbation weight $M_c$ for each pixel $(a, b)$ is determined by ensembling the class activation scores from $n$ CAM-based substitute models. Specifically, it takes the maximum class activation score across these models:

$M_c(a, b) = \max(m^1(a, b), m^2(a, b), \dots, m^n(a,b))$

Each $m^i(a, b)$ is calculated as a weighted sum of feature map responses $F^k_l(a, b)$ for the class of interest $c$, where $\alpha_k$ is a normalization factor.

The iterative update rule for generating the adversarial example $X_{t+1}$ incorporates this CAM-derived perturbation weight:

$X_{t+1} = X_t + (\lambda \alpha) (w M_c) \text{sign}(g_{t+1})$

Where:

• $\lambda$ is the perturbation step size factor.
• $\alpha$ is the learning rate.
• $w$ is the perturbation magnitude factor.
• $g_{t+1}$ is the momentum gradient, which incorporates a variance term $U_{t+1}$ to better explore the gradient landscape. This ensures that perturbations are applied discriminately, improving both stealthiness and attack efficacy.

Targeted Attack

For targeted attacks, the objective is more specific: to make the model classify the perturbed image $x'$ as a specific target label $y'$. The objective function is formulated as:

$x' = \arg \max -L(x', y'; \theta) - ||x_{tar} - x'||_2^2 \quad \text{subject to} \quad ||x - x'||_{\infty} \leq \epsilon$

Similar to the non-targeted approach, an integrated class activation map for the target class, $X_{tar} = \max(x_{tar\_cam}^1, x_{tar\_cam}^2, \dots, x_{tar\_cam}^n)$, is used to guide perturbations. The update rule is analogous, using $M_{tc}$ (activation map score for the specified target class) as the perturbation weight:

$X_{t+1} = X_t + (\lambda \alpha) (w M_{tc}) \text{sign}(g_{t+1})$

This ensures that the attack is highly effective in guiding the model towards the desired target class while maintaining stealthiness.

Implementation Details

The validation of CAM-EA was conducted using a rigorous experimental setup:

• Dataset: The ILSVRC 2012 validation set was used, along with data subsets from specific references, ensuring all test images were correctly classified by all models in their benign state.
• Models:
• CAMs Substitute Models: WideResNet101, Inception_v2, ResNet34.
• Gradient Substitute Model: ResNet50.
• Target Models: A diverse suite including AlexNet, VGG16, EfficientNet_b0, ResNet18, ResNet34, ResNet50, WideResNet50, WideResNet101, Inception_v2, MobileNet_v2, ConvNeXt, ViT, and RegNet. All models were pre-trained on ImageNet and sourced from PyTorch.
• Metrics:
• Perceptual Metrics: Peak Signal-to-Noise Ratio (PSNR), Mean Squared Error (MSE), Structural Similarity Index Measure (SSIM), Low-frequency error (Low_fre), CIEDE2000, and L2 norm.
• Attack Capability Metrics: Attack Success Rate (ASR) against a single target model, and Average Attack Success Rate (AASR) across all 13 target classifiers.
• Baselines: Ten state-of-the-art adversarial attack methods were used for comparison: PGD (52), TPGD (60), DIFGSM (56), TIFGSM (53), MIFGSM (53), NIFGSM (54), SINIFGSM (54), VMIFGSM (55), VNIFGSM (55), and SVRE (59). For SVRE, the ensemble model comprised Inception_v3, Inception_v4, InceptionResnet_v2, and ResNet101.
• Defense Methods: Seven defense mechanisms were tested for robustness analysis: Feature Squeezing (Fea. Squ.) (62), Label Smoothing (Lab. Smo.) (63), Engstrom (64), Salman (65), Singh (66), Liu (67), and Shan (68).
• Parameters: Consistent parameters were used across all attack methods: perturbation constraint $\epsilon = 16/255$, learning rate $\alpha = 1/500$ or $1/255$, momentum parameter $\mu = 1$, number of sampled examples $N=5$ for variance calculation. The perturbation magnitude factor $w$ was set to 2 for non-targeted attacks and 4 for targeted attacks, and the step size factor $\lambda$ was 0.75.

Demo / Proof of Concept

▶ Watch: Critique: Indiscriminate perturbations limit existing attack methods (3:09)

While the talk does not describe a live, interactive demonstration, the research effectively provides a proof of concept through visual depictions and extensive quantitative analysis of generated adversarial examples. Figures 1 and 2 in the accompanying paper visually illustrate adversarial examples created by CAM-EA and various baseline methods under both non-targeted and targeted attack scenarios. These visual comparisons serve as a direct demonstration of CAM-EA's superior perceptibility.

For instance, the visual quality of CAM-EA's adversarial images is notably higher, with perturbations being far less discernible to the human eye compared to many baseline methods. This qualitative observation is rigorously backed by quantitative perceptual metrics. In non-targeted attacks, when CAM-EA's attack capability is comparable to powerful methods like VMIFGSM and VNIFGSM, its visual stealthiness is significantly better. Specifically, the L2 norm is reduced by 24.08, Low_fre by 13.91, and SSIM improves by 0.04 while PSNR increases by 0.69 compared to VMIFGSM. Similarly, for targeted attacks, CAM-EA achieves the best perceptibility among eight baseline methods, with a PSNR improvement of 2.12 against VNIFGSM and 5.13 against SINIFGSM. These metrics, coupled with the visual evidence, unequivocally demonstrate that CAM-EA generates adversarial examples that are both highly effective and remarkably stealthy, fulfilling its core design objectives. The evaluation of attack success rates across 12 target models, depicted in Figures 4 and 5, further serves as a proof of concept for CAM-EA's superior attack efficacy and faster convergence, solidifying its practical utility.

Defensive Implications

▶ Watch: CAM-EA's unique strategy: Ensembling CAMs for transferability (3:49)

The findings presented by CAM-EA carry significant implications for the design and deployment of defense mechanisms against adversarial attacks. The robustness analysis conducted against seven state-of-the-art defense methods provides crucial insights into the current landscape of model security.

CAM-EA consistently demonstrates the most significant impact on reducing model robustness compared to baseline methods, under both non-targeted and targeted attack scenarios. Notably, defenses such as Feature Squeezing (Fea. Squ.) and Label Smoothing (Lab. Smo.) are most severely affected, indicating their particular vulnerability to CAM-EA's targeted and transferable perturbations. Shan's adversarial training method also showed a substantial reduction in effectiveness, while Engstrom's defense exhibited a relatively weaker impact, suggesting some level of resilience, though still compromised.

Even when facing these defenses, CAM-EA maintains its superior performance, consistently outperforming the baseline methods. This suggests that while defense mechanisms can mitigate the attack's full potential, they are often insufficient to completely neutralize highly sophisticated attacks like CAM-EA. This inherent evasion capability of CAM-EA highlights a critical gap in current defense strategies, many of which may not adequately account for attacks that intelligently target critical decision regions and exhibit strong transferability.

Furthermore, the research explores the relationship between image perceptibility and model robustness when defenses are applied. As the transformation parameter $\lambda$ (which controls perturbation magnitude) increases, leading to poorer image quality and higher perceptibility, the robustness of models equipped with defense methods gradually declines. This observation, illustrated in Figure 6, indicates that even robust models become more vulnerable as the adversarial perturbations become more pronounced. This reinforces the importance of stealthiness in adversarial attacks, as it allows attackers to achieve their goals without easily being detected or triggering robust defense mechanisms designed for more obvious perturbations.

The existence of CAM-EA, a method capable of generating highly stealthy, effective, and transferable adversarial examples, serves as a powerful tool for security researchers and practitioners. It underscores the necessity of developing more advanced and comprehensive defense mechanisms that can specifically counter attacks leveraging interpretability techniques and ensemble strategies. Defenders should:

1. Re-evaluate Current Defenses: Existing defenses need to be rigorously tested against attacks like CAM-EA that combine targeted perturbation and high transferability.
2. Focus on Interpretability-Aware Defenses: New defense strategies could explore methods that make it harder for attackers to identify and exploit critical regions via CAM or similar interpretability tools.
3. Enhance Robustness to Transferable Attacks: Given CAM-EA's high transferability, defenses should be designed to be robust not just against attacks specifically trained on the target model, but also against attacks generated from diverse substitute models.
4. Consider Perceptual Robustness: Defenses should aim to maintain model integrity even when subtle, near-imperceptible perturbations are introduced, reflecting the stealthy nature of CAM-EA.

In essence, CAM-EA provides a valuable benchmark for assessing the true robustness of DNN models in real-world black-box scenarios, pushing the boundaries for the next generation of adversarial defenses.

Key Takeaways

• Novel Black-Box Attack: CAM-EA is the first adversarial attack method to simultaneously consider both attack transferability through CAM ensembling and perturbation weighting using Class Activation Maps in a black-box setting.
• Superior Stealthiness: By focusing perturbations on critical image regions identified by CAM, CAM-EA generates adversarial examples with significantly enhanced visual imperceptibility, reducing L2 norm by 24.08 and Low_fre by 13.91, and improving SSIM by 0.04 and PSNR by 0.69 compared to VMIFGSM.
• High Attack Efficacy & Fast Convergence: The method achieves the most vigorous attack ability and fastest convergence, especially in low-round scenarios. It outperforms VMIFGSM by 11.69% and TPGD by 37.15% in non-targeted attacks at the 10th epoch.
• Robust Transferability: Ensembling CAMs from multiple substitute models ensures that the generated adversarial examples are highly effective across a diverse range of unseen target models.
• Evasion of Defenses: CAM-EA demonstrates a notable ability to evade various state-of-the-art defense mechanisms, including Feature Squeezing and Label Smoothing, highlighting the limitations of current defense strategies.
• Tool for Robustness Assessment: This work provides a powerful new tool for evaluating the true security and robustness of Deep Neural Networks, emphasizing the need for more sophisticated and interpretability-aware defense mechanisms.

About the Speaker(s)

The talk was delivered by Hui Xia. Based on the provided metadata and transcript, Hui Xia is a researcher in the field of adversarial machine learning, focusing on enhancing the stealthiness and transferability of adversarial attacks against Deep Neural Networks. This work, presented at the prestigious NDSS Symposium, highlights their expertise in developing novel methods to probe the vulnerabilities of AI systems.