TextGuard: Provable Defense against Backdoor Attacks on Text Classification

Hengzhi Pei

Network and Distributed System Security (NDSS) Symposium 2024 · Day 2 · ML Security

Overview

In an era increasingly reliant on machine learning models for critical applications, the integrity and trustworthiness of these systems are paramount. This article delves into TextGuard, a groundbreaking approach presented at the NDSS Symposium by Hengzhi Pei, which introduces the first provable defense against backdoor attacks on text classification models. These insidious attacks allow an adversary to embed hidden triggers into a model during training, causing misclassification to a target class when the trigger is present at inference time, while otherwise preserving normal functionality on clean inputs. This poses a severe threat, particularly within supply chains where models or training data may originate from untrusted sources.

Watch on YouTube · Slides

Visual summary for TextGuard: Provable Defense against Backdoor Attacks on Text Classification by Hengzhi Pei
Visual summary for TextGuard: Provable Defense against Backdoor Attacks on Text Classification by Hengzhi Pei

Key moments

  1. 0:00 Introduction to TextGuard: Provable defense against backdoor attacks
  2. 2:00 Defining backdoor attacks and TextGuard's threat model
  3. 3:55 TextGuard's defense assumptions and provable robustness goal
  4. 4:10 TextGuard's key technical approach: ensemble and word-level partitioning
  5. 4:40 Detailed explanation of word-level partitioning using hashing
  6. 5:30 TextGuard's strategy for defending against structure-level attacks

TextGuard: Provable Defense against Backdoor Attacks on Text Classification

Speakers: Hengzhi Pei

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=kwIB2c_3_Yfo

Overview

In an era increasingly reliant on machine learning models for critical applications, the integrity and trustworthiness of these systems are paramount. This article delves into TextGuard, a groundbreaking approach presented at the NDSS Symposium by Hengzhi Pei, which introduces the first provable defense against backdoor attacks on text classification models. These insidious attacks allow an adversary to embed hidden triggers into a model during training, causing misclassification to a target class when the trigger is present at inference time, while otherwise preserving normal functionality on clean inputs. This poses a severe threat, particularly within supply chains where models or training data may originate from untrusted sources.

Existing defenses against backdoor attacks largely fall into two categories: empirical and provable. While empirical defenses offer practical solutions, they frequently lack formal security guarantees, leaving them vulnerable to adaptive and sophisticated adversaries. Provable defenses, on the other hand, provide mathematical assurances of robustness. However, those developed for image data often leverage its continuous nature, rendering them unsuitable for the discrete space of text. NLP-specific adaptations of provable defenses typically provide only weak guarantees, tolerating an unacceptably small fraction of backdoored texts. TextGuard addresses this critical gap by offering a novel framework that provides strong, formal security guarantees against both word-level and structure-level backdoor attacks in Natural Language Processing.

TextGuard's core innovation lies in its unique word-level partitioning strategy, which contrasts sharply with prior sample-level approaches. By dividing words within training inputs into multiple disjoint groups and training an ensemble of base classifiers, TextGuard ensures that a majority of these base classifiers remain untainted by backdoor triggers. This enables a final prediction through majority voting that is theoretically guaranteed to be unaffected by triggers, provided their length falls within a certain threshold. The research demonstrates TextGuard's superior performance across benchmark text classification tasks, significantly surpassing the certified accuracy of existing certified defenses and outperforming state-of-the-art empirical defenses against various backdoor attack types. Its introduction marks a significant leap forward in securing NLP models against a prevalent and dangerous class of attacks.

Background

▶ Watch: Introduction to TextGuard: Provable defense against backdoor attacks (0:00)

The proliferation of machine learning models in diverse applications, from sentiment analysis to malware detection, has simultaneously opened new vectors for sophisticated cyberattacks. Among these, backdoor attacks have emerged as a particularly stealthy and potent threat. In the context of text classification, a backdoored model is engineered to behave normally on benign inputs but to consistently misclassify inputs containing a specific, hidden trigger to an attacker-chosen target class. This malicious behavior can be exploited to bypass security filters, manipulate public opinion, or compromise critical decision-making systems.

Backdoor attacks are broadly categorized into data-poisoning attacks and model-poisoning attacks. TextGuard focuses specifically on data-poisoning attacks, where the adversary possesses the ability to manipulate the training data but not the model training process itself. In this scenario, the attacker injects a backdoor trigger into a subset of training samples and often relabels them to the target class. A model trained on such a poisoned dataset will subsequently exhibit the desired backdoored behavior.

Data-poisoning attacks on text can manifest in two primary forms based on their trigger patterns:

  1. Word-level backdoor attacks: These attacks utilize one or more specific words as the trigger. For instance, injecting the word "actually" into a positive sentence like "the film is full of charm" results in "the film is actually full of charm," without altering the original sentiment or semantics for clean inputs, but activating the backdoor for poisoned inputs.
  2. Structure-level backdoor attacks: These attacks leverage a particular sentence structure, often combined with fixed words, as the trigger. An example might involve transforming "the film is full of charm" into "When it comes to this film, it is full of charm," using an attributive clause structure to embed the trigger.

The threat model for TextGuard assumes an attacker can poison a fraction, p, of the training samples. This poisoning can occur under two conditions:

  • Mixed-label attacks: The attacker can freely poison p fraction of training samples from arbitrary classes, relabeling them to the target class.
  • Clean-label attacks: A more stealthy variant where the attacker only poisons samples originally belonging to the target class, preserving their original labels but injecting the trigger.

The attacker is assumed to have a trigger word set of size e, which they can inject for word-level attacks or use to manipulate word order for structure-level attacks.

The defense goal of TextGuard is to train a provably robust and secure text classifier f. This classifier guarantees that, as long as the trigger size e is below a certain threshold, its predicted label for any testing data (whether backdoored or not) remains unaffected by the presence of backdoored training samples. This is equivalent to achieving a worst-case classification accuracy guarantee against arbitrary data poisoning attacks.

Existing provable defenses, particularly those originating from the image domain, are largely unsuitable for NLP due to the fundamental difference between continuous image data and discrete text. Attempts to adapt these defenses, such as Deep Partition Aggregation (DPA) and Bagging, typically yield very weak guarantees, tolerating less than 1% of backdoored texts and resulting in negligible certified accuracy. This limitation stems from their approach of partitioning entire training samples into groups. If the poisoning rate is high, it becomes exceedingly difficult to ensure that a majority of these sample-level groups remain clean, thus compromising the provable guarantees. TextGuard distinguishes itself by addressing this critical flaw through its novel word-level partitioning strategy, which forms the bedrock of its superior certified robustness.

Key Findings

▶ Watch: TextGuard's defense assumptions and provable robustness goal (3:55)

TextGuard represents a significant advancement in the field of secure machine learning, offering the first comprehensive provable defense against backdoor attacks in Natural Language Processing. Its key findings and contributions can be summarized as follows:

  • First Provable Defense for NLP Backdoors: TextGuard is the pioneering work to provide formal security guarantees against both word-level and structure-level backdoor attacks in text classification. This addresses a critical gap where prior provable defenses were either inapplicable to discrete text or offered extremely weak guarantees.
  • Superior Certified Accuracy: Through extensive evaluations, TextGuard demonstrated significantly higher certified accuracy compared to existing certified defenses like DPA and Bagging. While baselines often yielded 0.0000 certified accuracy, TextGuard achieved meaningful positive accuracies (e.g., 0.7589 on AG's News for p=0.01, e=1), validating its robust theoretical framework.
  • Outperformance of State-of-the-Art Empirical Defenses: TextGuard consistently achieved the lowest Attack Success Rate (ASR) against various word-level (BadWord, AddSent) and structure-level (Hidden Killer/SynBkd) backdoor attacks, while maintaining competitive or superior Clean Accuracy (CACC). This empirical superiority highlights its practical efficacy in real-world threat scenarios.
  • Novel Word-Level Partitioning Strategy: The core innovation lies in dividing words within each input text into multiple disjoint groups using a hash function, rather than partitioning entire training samples. This ensures that a backdoor trigger, consisting of a bounded number of words, can only corrupt a limited number of groups, thus preserving the integrity of the majority of base classifiers.
  • Robustness Against Diverse Attack Vectors: TextGuard proved robust against varying poisoning rates (up to p=0.3) and an adaptive attack where trigger words were strategically assigned to unique groups. It also demonstrates an ability to defend against complex structure-level attacks, albeit with some limitations on specific datasets for style-based triggers.
  • Efficiency and Scalability: Despite training an ensemble of base classifiers, TextGuard exhibited significantly lower computational training times compared to DPA and Bagging (minutes versus hours), making it a more practical solution for real-world deployments.
  • Empirical Enhancements for Utility: The introduction of Semantic Preserving and Potential Trigger Word Identification techniques further enhances TextGuard's practical utility. These strategies mitigate the trade-off between strict provable robustness and clean accuracy, allowing users to fine-tune the defense based on application-specific priorities.
  • Agnosticism to Model Architecture and Training Algorithm: TextGuard's provable guarantees are independent of the underlying training algorithm (A) and model architecture (e.g., BERT), making it broadly applicable across different NLP model implementations.

These findings collectively establish TextGuard as a highly effective and theoretically sound defense mechanism, offering a crucial layer of security for machine learning models in security-critical NLP applications.

Technical Deep Dive

▶ Watch: TextGuard's key technical approach: ensemble and word-level partitioning (4:10)

TextGuard's innovative architecture is centered around an ensemble text classifier that leverages a novel word-level partitioning strategy to achieve provable robustness against backdoor attacks. The fundamental insight is that by ensuring a majority of constituent base classifiers remain untainted by triggers, their collective decision through majority voting can effectively neutralize the backdoor.

The core technical approach can be broken down into several key components:

  1. Word-level Partitioning:

Unlike previous provable defenses that partition entire training samples, TextGuard operates at the word level. For any given input text x = x_1, ..., x_d (a sequence of d words), each individual word x_k is assigned to one of m disjoint groups. This assignment is deterministic, using a hash function H (e.g., MD5): the group ID for x_k is computed as H(x_k) % m + 1. This ensures that the same word, regardless of its position or the text it appears in, is always mapped to the same group. This consistency is crucial because it localizes the impact of any trigger word; a trigger word can only "corrupt" the single group it belongs to, preventing it from influencing other groups. When the number of words in a backdoor trigger is bounded, the number of corrupted groups is similarly bounded.

  1. Defense against Structure-level Attacks:

To specifically counter structure-level backdoor attacks, which often manipulate word order, TextGuard introduces an additional step. After words are assigned to their respective groups, the sequence of words within each group g_j(x) is sorted based on a pre-defined word ID (e.g., BERT's word ID). This sorting process effectively destroys the original syntactic and semantic order of words within each sub-text, making the sequence of words within g_j(x) independent of their original arrangement in x. This independence is vital for thwarting structure-level triggers that rely on specific sentence constructions.

  1. Training Phase:

The word-level partitioning is applied to both training and testing texts. For each original training instance (x_i, y_i) in the dataset D, the text x_i is divided into m sub-sequences: g_1(x_i), ..., g_m(x_i). Crucially, for each sub-sequence g_j(x_i), a (sub-text, label) pair (g_j(x_i), y_i) is created, assigning the original ground truth label y_i to all m sub-texts. These pairs are then aggregated to form m distinct sub-datasets, D_j, where D_j contains all (g_j(x_k), y_k) pairs for group j across the entire training set. Finally, a base text classifier f_j is trained independently on each sub-dataset D_j.

  1. Testing Phase:

Given a testing text x_test, the same word-level partitioning and sorting mechanisms are applied to divide its words into m groups g_j(x_test). Each trained base classifier f_j then makes a prediction for its corresponding sub-text g_j(x_test). The final prediction for x_test by the ensemble classifier f(x_test; D) is determined by a majority vote over the m predicted labels from the base classifiers. Ties are typically broken by selecting the label with the smaller index.

Deriving Certified Accuracy

The strength of TextGuard lies in its provable robustness, formalized by Theorem 1. The goal is to guarantee that the ensemble classifier's prediction for a backdoored text x'_test (generated by trigger T_e) is identical to its prediction for the clean text x_test when trained on a backdoored dataset D(T_e).

Let M_c be the number of base classifiers predicting label c when trained on a clean dataset D(Ø), and M'_c be the count when trained on D(T_e). The key insight for certification is that each trigger word is assigned to a single group. Therefore, a backdoor trigger of size e can affect at most e of the m base classifiers. This implies that for any label c, the count M'_c can deviate from M_c by at most e (i.e., M_c - e <= M'_c <= M_c + e).

The ensemble classifier f will predict the same label y (the correct label from D(Ø)) even in the presence of D(T_e) if the number of base classifiers predicting y in the backdoored scenario (M'_y) is strictly greater than the number of base classifiers predicting any other class c (M'_c). This condition is formalized as M'_y > max_{c!=y} M'_c. Substituting the bounds, this simplifies to M_y - e > max_{c!=y} (M_c + e). Rearranging this inequality yields the condition for robustness: e < (M_y - max_{c!=y} M_c) / 2.

The certified size s(x_test) for an individual test sample x_test is thus defined as (M_y - max_{c!=y} M_c) / 2. Theorem 1 formally states that the predicted label of f for x_test is provably unaffected by the backdoor trigger T_e when the actual trigger size e is less than or equal to s(x_test).

Key observations from Theorem 1:

  • TextGuard is agnostic to the specific training algorithm A and model architecture.
  • It provably resists arbitrary word-level or structure-level backdoor attacks as long as the trigger size e is bounded.
  • The certified size s(x_test) is larger when there is a greater margin between the number of base classifiers predicting the correct label y and the maximum number of base classifiers predicting any other incorrect label.

TextGuard employs two strategies for calculating certified accuracy:

  • Individual Certification: This method calculates s(x_test) for each individual test sample x_test independently.
  • Joint Certification: This more advanced technique considers that the e corrupted groups might be the same across all testing inputs. By analyzing combinations of e groups out of m and deriving a tighter lower bound for M'_c, joint certification provides a more accurate and often higher certified accuracy, as it leverages collective information.

Empirical Extension of TextGuard

While TextGuard's provable guarantees are robust, the explicit word sorting for structure-level defense and strict word-to-group mapping can sometimes reduce the semantic context available to base classifiers, potentially affecting clean accuracy. To mitigate this trade-off and enhance empirical performance, TextGuard proposes two optional empirical techniques:

  1. Semantic Preserving:

The provable defense sorts words within groups, which sacrifices the original semantic order. For improved empirical utility, this strict sorting can be relaxed.

  • During the testing phase, instead of partitioning x_test into m groups and sorting, the original, unpartitioned x_test can be fed to each base classifier f_j for prediction. This provides f_j with the full semantic context, potentially leading to more accurate predictions on clean inputs.
  • Similarly, during the training phase, the original word order can be maintained within the sub-texts g_j(x_i) when constructing D_j.

This technique trades some of the strict provable security against structure-level attacks for better empirical performance and overall utility, allowing base classifiers to leverage richer semantic information.

  1. Potential Trigger Word Identification (Omega):

The default TextGuard design assigns every word to a single group. However, trigger words typically constitute a small fraction of the vocabulary. This technique aims to differentiate between potential trigger words and benign words.

  • First, a classifier f' is trained on a potentially backdoored training dataset D'.
  • For each word x in a training text x_train, its influence score is computed as the L2-norm of the difference between the latent feature vectors produced by f' for x_train and for x_train with x removed (x'_train).
  • A word x is deemed influential for x_train if its influence score is among the top-5 for that text.
  • If a word x is influential for at least K training texts in D', it is added to a set of potential trigger words, Omega.

Once Omega is identified, only words within Omega are mapped to a single group using the hash function. All other words (presumed non-trigger words) are assigned to all m groups. This allows non-trigger words to contribute information to all base classifiers, significantly improving clean accuracy without compromising robustness against triggers in Omega. The hyperparameter K introduces a trade-off: a larger K generally improves utility (higher CACC) but might reduce security if actual trigger words are missed by Omega. Users can tune K based on their specific application requirements.

These empirical extensions provide flexibility, allowing TextGuard to strike an optimal balance between rigorous provable guarantees and practical performance, making it adaptable to a wider range of real-world scenarios.

Demo / Proof of Concept

▶ Watch: Detailed explanation of word-level partitioning using hashing (4:40)

While the talk did not feature a live software demonstration, TextGuard's effectiveness and robustness were thoroughly validated through extensive certified and empirical evaluations. These experiments served as the primary proof of concept, comparing TextGuard against both state-of-the-art provable defenses and empirical baselines across various text classification tasks and attack scenarios.

Experiment Setup

The evaluations utilized three benchmark text classification datasets:

  • SST-2: For sentiment analysis (6,920 training, 1,821 testing samples, binary labels).
  • HSOL: For toxic classification (5,823 training, 2,485 testing samples, binary labels).
  • AG's News: For topic classification (108,000 training, 7,600 testing samples, 4 topics).

The underlying model architecture was BERT, a widely adopted language model, trained using the AdamW optimizer. MD5 was the default hash function for word partitioning. Backdoored training sets were constructed using both mixed-label attacks (poisoning rate p=0.1) and clean-label attacks (poisoning rate p=0.2).

TextGuard's performance was benchmarked against:

  • Direct Training (DT): A baseline without any defense, used to establish the upper bound of attack effectiveness.
  • Provable Defenses: DPA (Deep Partition Aggregation) and Bagging, both generalized from the image domain and operating by partitioning training samples.
  • Empirical Defenses: R-Adapter, BKI (Backdoor Keyword Identification), ONION, STRIP, and RAP (Robustness-Aware Perturbations), representing state-of-the-art empirical approaches.
  • Backdoor Attacks: BadWord (word-level, single irregular word), AddSent (word-level, trigger sentence), and Hidden Killer (SynBkd) (structure-level, syntactic paraphrasing).

Certified Evaluation Results

The certified evaluations demonstrated TextGuard's unparalleled robustness:

  • TextGuard vs. Direct Training: While DT models quickly succumbed to attacks, exhibiting near 0% classification accuracy even for a trigger size e=1, TextGuard consistently provided positive certified accuracy. For instance, on HSOL with m=5, TextGuard maintained 0.66 certified accuracy for e=1 under clean-label attacks and 0.54 under mixed-label attacks. This clearly verified TextGuard's meaningful certification guarantee. The results also showed that a larger number of groups (m) generally allowed TextGuard to tolerate larger trigger sizes, confirming theoretical expectations.
  • TextGuard vs. DPA and Bagging: This comparison starkly highlighted TextGuard's superiority. DPA and Bagging consistently failed to provide any meaningful certified accuracy, often reporting 0.0000, even at a low poisoning rate of p=0.01 and trigger size e=1. This failure was attributed to their sample-level partitioning, which necessitates an impractical number of groups to ensure cleanliness, thereby degrading base model performance. In contrast, TextGuard's word-level partitioning decoupled the number of clean groups from the poisoning rate, achieving significantly higher certified accuracies (e.g., 0.3904 on SST-2, 0.6232 on HSOL, and 0.7589 on AG's News for p=0.01, e=1).
  • Computational Efficiency: TextGuard also proved to be substantially more efficient. For e=1, its training times ranged from 1.9 minutes (HSOL) to 17.9 minutes (AG's News), whereas DPA and Bagging often required hours. This efficiency stems from TextGuard needing to train fewer base models due to its word-level strategy.
  • Robustness to Poisoning Rate Variations: TextGuard maintained decent and meaningful certified accuracy even at a high poisoning rate of p=0.3. For p=0.3 and e=1 on HSOL, TextGuard achieved 0.2246 certified accuracy, dramatically outperforming DT's 0.0008.
  • Ablation Studies: The choice of hash function (MD5, SHA1, SHA256) had only a minor impact on certified accuracy, demonstrating TextGuard's insensitivity to this detail. Critically, joint certification consistently outperformed individual certification (e.g., 0.6417 vs. 0.6143 for e=1 on HSOL), validating the benefit of considering all testing texts collectively for tighter bounds.

Empirical Evaluation Results

The empirical evaluations confirmed TextGuard's practical efficacy against various real-world attacks:

  • TextGuard vs. Comparison Baselines: TextGuard generally achieved the lowest Attack Success Rate (ASR) across most word-level and structure-level attacks under both mixed-label and clean-label setups. For example, on SST-2 under a clean-label attack, TextGuard's ASR was 0.1754, while all other empirical defenses had ASRs exceeding 0.90. This superior defense efficacy is attributed to TextGuard's fundamental design, which bypasses the often-unreliable step of trigger detection by directly training a robust classifier. TextGuard also maintained a decent Clean Accuracy (CACC), comparable to or better than most baselines (e.g., 0.8929 on SST-2 compared to DT's 0.9176), demonstrating effective preservation of model utility on clean inputs. Its computational cost was also comparable to other empirical defenses.
  • Robustness against Attack Variations: TextGuard showed greater stability against word-level attacks. While ASR for structure-level attacks (SynBkd) increased with higher poisoning rates, TextGuard still offered a strong defense. Against an adaptive attack where each trigger word was assigned to a unique group (to bypass the defense), TextGuard still forced a significantly low ASR (0.2319 for mixed-label, e=3) compared to undefended models, due to its group size being much larger than the trigger size.
  • Connection with Certified Evaluation: As expected, empirical accuracy was consistently larger than the corresponding certified accuracy (e.g., on HSOL for e=1 mixed-label, empirical CACC was 0.9275 while certified was 0.5620). This difference is natural, as certified accuracy provides a lower bound against arbitrary attacks, while empirical results are against specific attack instances.
  • Ablation Studies (Empirical):
  • Semantic Preserving: Discarding this strategy led to a drop in both CACC and ASR (e.g., CACC dropped from 0.9115 to 0.8141, and ASR increased from 0.1208 to 0.2198 for BadWord on HSOL), validating the importance of preserving semantic context for utility.
  • Potential Trigger Word Identification: A larger K (number of texts a word must influence to be in Omega) generally improved CACC and reduced ASR for word-level attacks. However, for structure-level attacks, ASR sometimes increased with K, indicating a trade-off that users must manage based on their priorities.
  • Style Backdoor Attack (SynBkd): The style-based SynBkd attack proved particularly challenging. While TextGuard performed well on AG's News, its efficacy was limited on SST-2 and HSOL. This suggests that highly complex, style-based triggers can still mislead base models even after partitioning, highlighting an area for future research.

In summary, the comprehensive evaluations unequivocally demonstrate that TextGuard provides a strong, meaningful, and efficient certified defense against backdoor attacks in NLP, significantly outperforming both existing provable and empirical baselines across various metrics and attack scenarios.

Defensive Implications

▶ Watch: TextGuard's strategy for defending against structure-level attacks (5:30)

TextGuard offers profound implications for defenders operating in environments where text classification models are deployed, particularly in security-critical applications or within untrusted machine learning supply chains. Its primary contribution is providing provable robustness, a feature largely absent or extremely weak in prior NLP defense mechanisms.

  1. Formal Security Guarantees: Defenders can leverage TextGuard to deploy text classifiers with formal security guarantees against data-poisoning backdoor attacks. This is invaluable in domains like content moderation, spam detection, fraud detection, or critical infrastructure monitoring, where the consequences of a backdoored model can be severe. The ability to mathematically certify that a model's prediction remains unaffected by triggers up to a certain size e (the certified size s(x_test)) provides a level of assurance previously unavailable.
  1. Mitigating Supply Chain Risks: In scenarios where training data cannot be fully trusted (e.g., obtained from third-party vendors, scraped from the internet, or crowdsourced), TextGuard provides a robust mechanism to train models resilient to hidden backdoors. Even if a significant fraction of the training data is poisoned, TextGuard's word-level partitioning ensures that the majority of base classifiers remain clean, effectively neutralizing the attack.
  1. Versatility Against Attack Types: TextGuard's design specifically counters both word-level and structure-level backdoor attacks. This broad coverage means defenders are better equipped against a wider spectrum of adversarial manipulations, including sophisticated attacks that alter sentence syntax rather than just injecting isolated words. The sorting mechanism within groups is a key component for this resilience.
  1. Practicality and Performance: Despite its provable guarantees, TextGuard demonstrates competitive or superior empirical performance (low ASR, high CACC) against state-of-the-art empirical defenses. Furthermore, its computational efficiency (minutes of training time compared to hours for other certified defenses) makes it a practical solution for real-world deployment, avoiding prohibitive overheads.
  1. Configurable Trade-offs: The empirical extensions, such as Semantic Preserving and Potential Trigger Word Identification, offer defenders crucial flexibility. They can tune parameters like the number of groups (m) and the K value for trigger identification to strike a desired balance between stringent provable guarantees and optimal clean accuracy/utility for their specific application and threat model. For applications where strict semantic context is paramount, the semantic preserving technique can be employed, albeit with a slight trade-off in provable robustness against structure-level attacks.
  1. Awareness of Limitations: Defenders should also be mindful of TextGuard's current limitations. While robust, its provable guarantee is bounded by trigger size. Very large triggers might still pose a challenge, requiring a greater number of groups and potentially impacting computational costs and base classifier accuracy. Additionally, while effective against many adaptive attacks, highly sophisticated style-based backdoor attacks (like SynBkd on some datasets) remain a harder challenge, suggesting areas where further research and complementary defenses might be needed. For NLP tasks heavily reliant on precise inter-word context (e.g., question answering), the inherent context loss from word partitioning might be a consideration.

In essence, TextGuard provides a robust, provably secure, and practically viable framework for building trustworthy text classification models. It empowers defenders to significantly elevate the security posture of their NLP systems against a critical class of adversarial attacks, fostering greater confidence in the integrity of AI-driven decisions.

Key Takeaways

  • First Provable Defense for NLP Backdoors: TextGuard is the pioneering solution offering formal security guarantees against both word-level and structure-level backdoor attacks in text classification, addressing a critical gap in existing defenses.
  • Word-Level Partitioning is Key: Unlike prior sample-level approaches, TextGuard's novel word-level partitioning strategy ensures that a bounded trigger size can only corrupt a limited number of base classifiers, allowing majority voting to maintain robustness.
  • Superior Certified and Empirical Performance: TextGuard significantly outperforms existing certified defenses (DPA, Bagging) in certified accuracy and state-of-the-art empirical defenses in Attack Success Rate (ASR), while maintaining high Clean Accuracy (CACC) and comparable computational efficiency.
  • Robustness to Diverse Threats: The defense is effective against various word-level and structure-level attacks, varying poisoning rates, and even adaptive attacker strategies, making it a strong general-purpose solution.
  • Flexible Utility-Robustness Balance: Empirical extensions like semantic preserving and potential trigger word identification allow defenders to strategically balance strict provable guarantees with practical model utility and clean accuracy based on application needs.
  • Critical for Secure NLP Supply Chains: TextGuard provides a vital mechanism for deploying trustworthy text classification models in security-critical applications where training data sources may be untrusted, significantly mitigating supply chain risks.

About the Speaker(s)

The talk was presented by Hengzhi Pei. The provided transcript and metadata do not include additional details about the speaker's title or affiliation.

All talks from Network and Distributed System Security (NDSS) Symposium 2024