Inaudible Adversarial Perturbation: Manipulating the Recognition of User Speech in Real Time

Inaudible Adversarial Perturbation: Manipulating the Recognition of User Speech in Real Time

Speakers: Xinfeng Li

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=VD_KAU-s1oA

Overview

This talk introduces VRIFLE, a novel inaudible adversarial perturbation (IAP) attack designed to manipulate Automatic Speech Recognition (ASR) systems in real-time, even while a user is speaking and without their awareness. Presented by Xinfeng Li, this research addresses a significant gap in the field of adversarial examples against ASR. Previous attacks often suffered from audibility, limited universality, or failed in the presence of user speech and potential remedy commands. VRIFLE aims to overcome these limitations by leveraging ultrasound modulation to deliver stealthy, potent perturbations.

The core problem VRIFLE tackles is the vulnerability of ASR models to misinterpretation when subtle, inaudible signals are injected into the audio stream. This is particularly critical in user-present scenarios, where voice assistants, smart devices, and customer service systems rely on accurate speech recognition. The ability to covertly alter or silence user commands poses a severe security risk, enabling adversaries to inject misinformation, bypass authentication, or disrupt services without the user ever noticing the interference. This research is crucial for understanding and mitigating a sophisticated new class of threats to voice-controlled systems.

Background

▶ Watch: Introducing VRIFLE: Inaudible real-time ASR manipulation (0:00)

ASR systems convert raw audio into linguistic units through a pipeline involving spectrum feature extraction, neural acoustic models, and decoders, often utilizing Connectionist Temporal Classification (CTC) for alignment. The vulnerability of these systems to adversarial examples (AEs) has been consistently demonstrated. AEs are inputs designed to sound benign to humans but cause misrecognition by models. The primary challenge for AEs has always been stealthiness – ensuring they are not detected by the user.

Early adversarial attacks on ASR, often referred to as audible AEs, focused on reducing audibility by making perturbations sound like music, noise, or short pulses. Examples include Carlini's work (10), Abdullah's work (11), and Schönherr's psychoacoustic hiding (3). However, these methods are still susceptible to human detection, especially in "user-present scenarios" where users are actively interacting with an ASR system. Such attacks often fail if a user notices unexpected ASR behavior (e.g., false wake-ups) and issues remedy commands. Furthermore, their attack distances are typically limited to a few meters or less.

A more stealthy approach involves inaudible attacks, which modulate malicious audio onto high-frequency carriers (above 20 kHz) that are imperceptible to human ears. These attacks exploit the nonlinear vulnerability of microphones, which demodulate the high-frequency carrier, recovering the malicious audio back into the audible band. Prior inaudible attacks, such as DolphinAttack (5) using ultrasound or laser-based attacks (6), primarily focused on command injection (e.g., activating a voice assistant with a hidden command) rather than the real-time manipulation of ongoing user speech. These attacks also faced significant challenges in user-present scenarios, which VRIFLE specifically addresses:

1. Achieving universal, inaudible adversarial perturbations: The trade-off between universality (ability to attack arbitrary speech) and stealthiness (inaudibility) is a long-standing problem. Audible AEs are often short and noise-like, limiting their universality. VRIFLE aims for truly inaudible perturbations using ultrasound, which removes the audibility constraint and opens up a vast optimization space for more universal attacks.
2. Altering user speech in real-time despite disruption: Even if inaudible, user disruption (e.g., continuous speech, remedy commands) is a major hurdle. Superimposing IAPs on user speech can lead to interference, and the unpredictable nature of user speech (content, timing, length, loudness, environment) makes real-time manipulation extremely challenging.
3. Guaranteeing physical effectiveness after ultrasonic delivery: The ultrasound channel is inherently lossy and distorted (15). Maintaining the effectiveness of fine-grained IAPs after modulation, transmission, and demodulation in the physical world is non-trivial. Factors like high-directional propagation, varying soundfields, nonlinear distortion, anomalous noises, and hardware-induced instability make existing acoustic channel modeling methods (like Room Impulse Response for audible bands) inapplicable.

Key Findings

▶ Watch: Limitations of traditional audible adversarial attacks on ASR (2:00)

VRIFLE presents a groundbreaking approach to manipulating ASR systems, demonstrating several key findings that significantly advance the state of the art in adversarial audio.

First, VRIFLE achieves unprecedented universality in targeted adversarial attacks against ASR. By leveraging the broad optimization space afforded by inaudible perturbations (where the perturbation magnitude ε can be set to 1 without audibility concerns), VRIFLE successfully altered 18,946 speech samples into a desired target command ("open the door") and muted 27,531 user commands into a blank transcription. This level of universal effectiveness, especially for targeted attacks on arbitrary user speech, was previously unattainable with audible AEs.

Second, the system demonstrates remarkable physical robustness and long-range effectiveness. VRIFLE maintains 100% Success Rate (SR) and 0% Character Error Rate (CER) for target commands over distances ranging from 1m to 9m. It remains highly effective up to 10m (88.7% SR). The attack is also robust across various attack angles (15 to 150 degrees at 100% SR), diverse environmental scenes (SRs from 92.5% to 100% in offices, lounges, labs, corridors), and under significant ambient noise levels (average SR of 97.65% at 65 dB). This broad operational capability in real-world physical environments is a critical advancement.

Third, VRIFLE's ultrasonic transformation modeling is proven to be essential for physical effectiveness. Ablation experiments showed that without this precise modeling, direct ultrasound attacks or optimization without transformation resulted in a 0% SR. Only VRIFLE, with its sophisticated modeling of the nonlinear ultrasound channel, achieved 100% SR and 0% CER in physical tests, validating the critical role of its divide-and-conquer strategy for handling ultrasound complexity.

Finally, VRIFLE exhibits strong resilience against existing defenses. It largely bypasses common audio pre-processing methods like Quantization, Voice Activity Detection (VAD), and Opus Compression (all maintaining 95.93% SR or higher). Even against more challenging defenses like band-pass filters and down-sampling, an adaptive adversary (one who integrates defense mechanisms into the perturbation crafting process) can significantly boost VRIFLE's performance. Furthermore, VRIFLE compromises a dedicated inaudible attack detection method, LipRead (21), reducing its detection accuracy to as low as 30.55% for an adaptive adversary, indicating that current detection methods are insufficient.

Technical Deep Dive

▶ Watch: Overview of inaudible ultrasound attacks and VRIFLE's key challenges (4:00)

VRIFLE's sophisticated design addresses the complex challenges of inaudible, real-time adversarial attacks in user-present scenarios.

Threat Model:

The threat model assumes user-present scenarios where the victim is alert and can issue arbitrary or remedy commands. The adversary prepares IAPs offline for specific intents and delivers them in real-time using an ultrasonic transmitter, either physically close (e.g., handheld device) or at a distance (e.g., through a window). Adversary goals include providing wrong information to intelligent voice customer service or compromising voice assistants. Two covert strategies are considered:

1. No-feedback Attack: Muting VA vocal prompts and turning off WiFi to prevent user awareness.
2. Man-in-the-middle Attack: Replaying user commands after satisfying the adversary's intent, making the attack less suspicious.

Assumptions include the adversary having prior knowledge of the target ASR model for gradient information and access to a recording device (e.g., a smartphone of the same brand) to model ultrasonic transformation. The transmitter is deployed within line of sight.

Preliminary Investigation and Challenges:

Initial investigations confirmed the failure of traditional inaudible attacks in user-present scenarios. While ultrasound was chosen over laser due to visibility and channel distortion issues, direct ultrasound-based attacks still severely degraded when coinciding with user voice, with CERs up to 96.01%. This is because the human voice significantly affects the malicious signal, causing ASR acoustic features to deviate from the adversary's anticipation.

Attempts to model ultrasound delivery using Room Impulse Response (RIR) failed because RIR assumes a linear time-invariant (LTI) system, whereas ultrasound attacks exploit the nonlinear vulnerability of microphones. A Multi-layer Perception (MLP) also failed due to its extreme position-dependency; even slight displacements (e.g., 3 cm) introduced anomalous noises and caused the network to fail. These failures highlighted four root causes for the difficulty in modeling ultrasonic delivery:

• Ultrasound-induced Noise: Continuous vibration of the microphone diaphragm by the ultrasound carrier generates abnormal noise varying with injection angle and distance.
• Nonlinear Distortion: The input/output relationship of nonlinear demodulation involves unknown and varying factors.
• Varying Soundfield: Ultrasound fields (25 kHz) are significantly more directional and change dramatically compared to audible waves (1 kHz) due to shorter wavelengths.
• Hardware-induced Instability: Signal processing and hardware imperfections introduce instability.

VRIFLE System Design and Architecture:

VRIFLE's design aims to create universal IAPs that manipulate ASRs in real-time, overcome user disruption, and ensure physical effectiveness. The core objective is to fool ASRs into recognizing the superimposed speech of user voice and perturbation (x + δ) as the adversary-desired transcription (y_t), formalized by minimizing L(f(x + δ), y_t) subject to δ and x + δ within [-1, 1].

1. Ultrasonic Transformation Modeling: This is VRIFLE's first major component, designed to estimate the recorded audio m' = H(f)m(f) + n, which is highly similar to the actual recorded audio m. It employs a divide-and-conquer strategy:

• Tackling Anomalous Noises: A lightweight noise dataset (U_n) of 25 pieces of 10-second noises is collected by simply emitting the ultrasonic carrier at various angles and distances without modulation.
• Ultrasonic Frequency Response (UFR): The sine sweep technique (36) is used to characterize the inaudible acoustic energy conversion across frequencies, decoupling linear and nonlinear terms. This yields H(f), a holistic frequency-domain UFR of the received perturbations.
• Enabling Location-Variable Attacks: To adapt UFR to arbitrary positions, sine sweeps are conducted at different angles to obtain a complete set of UFRs (U_H) and anomalous noises (U_n). During optimization, UFR and noise pairs are randomly selected from these sets to mimic real-world delivery at different locations. The optimization objective is modified to:

argmin_δ E_{h_o∈U_H, n∈U_n} L(f(x + h_o(d) * δ + n), y_t)

where h_o(d) * δ represents the time-domain transformed perturbation.

1. Silence Perturbation (ξ): To counter long user instructions and potential remedy commands, VRIFLE introduces Silence Perturbation. These perturbations alter arbitrary user instructions to a blank transcription, rendering the ASR system unresponsive. This prevents remedy commands and can induce a denial-of-service. An empirically set robust silence perturbation is 5 seconds, repeatable for longer utterances.

1. Universal Perturbation (δ): VRIFLE integrates silence perturbation into a Universal Perturbation strategy. The inaudible delivery allows for unconstrained perturbation length and amplitude, maximizing the optimization space. A universal perturbation (δ) is crafted to alter short utterances to a desired intent. For longer speech or remedy commands, δ is combined with ξ in an alter-and-mute manner. The optimization objective is further refined:

argmin_δ E_{h_o∈U_H, n∈U_n, x∈U_x} L(f(x + S(h_o(d) * δ : ξ + n)), y_t)

Here, S(.) randomly shifts user utterances (x) within a preset time range (T=100 ms) to introduce randomness, making the attack synchronization-aided. Voice Activity Detection (VAD) triggers perturbation emission in real-time, accounting for pipeline delays.

1. Physical Robustness:

• Loudness Adaptive and Hardware Instability: To account for varying user speech loudness and hardware instability, relative volume augmentation (β) (0.5 to 1.5) is introduced during crafting, adding randomness to the mutual energy relationship.
• Attack at Different Environments: To counter multi-path reflections and ambient noises, random Room Impulse Response (RIR) and noise clips from the Aachen Impulse Response (AIR) Database (33) are used for user speech augmentation during training.
• Single-Sideband Extension: To enable attacks with portable devices and off-the-shelf loudspeakers (which often have poor suppression of intermodulation with Double-Sideband AM), Upper Single-Sideband Amplitude Modulation (USB-AM) is adopted. USB-AM has half the bandwidth, leading to higher transmission efficiency and reduced intermodulation. Mathematically, S_USB(t) = mcosω_c t msinω_c t + cosω_c t.

Implementation Details:

VRIFLE was implemented using PyTorch. The target ASR model was the end-to-end DeepSpeech2 (20). The Fluent Speech Command Dataset (29) (30,046 samples) was used for training and evaluation. Hardware included a SIGLENT SDG6032X signal generator, NF HSA4015 power amplifier, and a custom ultrasound transducer array. Recording devices tested included Google Pixel 3aXL, iPhone14 pro, MI Mix2s, OPPO Reno5 pro, and ReSpeaker Mic array v2.0. Metrics used were Success Rate (SR), Character Error Rate (CER), Signal-to-Noise Ratio (SNR), and L2-distortion.

Demo / Proof of Concept

▶ Watch: Threat model and failure of direct ultrasound attacks with user speech (6:00)

The efficacy of VRIFLE was rigorously evaluated through extensive digital and physical experiments, demonstrating its practical applicability and robustness.

Digital Attack Performance:

VRIFLE's inaudibility allowed for a perturbation upper bound (ε) of 1, significantly boosting universality. With ε=1.0, VRIFLE successfully altered 18,946 samples into "open the door" and muted 27,531 user commands into blank, showcasing its highly universal capability. Compared to classical audible AEs (CW 7, Qin 4, SpecPatch 8), VRIFLE converged significantly faster (1.63 min for 5 commands vs. 6.52-35.38 min for others) due to its broad optimization space and reduced complexity, all while avoiding the high audibility costs (low SNRs, high L2-distortion) inherent in audible AEs.

Physical Attack Performance:

• Different Target Commands: VRIFLE demonstrated consistent high performance across 10 typical malicious commands (e.g., "open the door," "turn on the light"). Most targets achieved 100% SR and 0% CER (7 out of 10), with the lowest SR still at 92.82% and CER down to 0.50%.
• Ablation Experiments: Validation of the ultrasonic transformation modeling (G4 in the talk) showed its critical role. Without this modeling (G1, G2), SR was 0%. With a low-pass filter approximation (G3), SR was 21.67%. Only VRIFLE (G4) achieved 100% SR and 0% CER, proving the necessity of the proposed modeling.
• Attack Distances: VRIFLE was highly effective within 1m-9m, achieving 100% SR and 0% CER. At 10m, SR was 88.7% with 3.25% CER, with performance decreasing beyond 10m due to ultrasound attenuation.
• Attack Angles: The attack maintained 100% SR between 15 and 150 degrees from the recording device, demonstrating broad angular robustness.
• Different Scenes: VRIFLE performed robustly in various environments (small office, medium lounge, large laboratory, narrow corridor), with SRs ranging from 92.5% to 100% and CERs from 0% to 1.04%, indicating resilience to environmental reverberation.
• Ambient Noises: Under typical ambient noises (cafeteria, office, lab, outdoor) at 50-65 dB, VRIFLE maintained an average SR up to 97.65%. A slight drop for office noise (keyboard typing) suggested high-frequency noise might reduce performance.
• Different Recording Devices: VRIFLE achieved an average SR of 96.8% and CER of 0.50% across various smartphones (Pixel 3aXL, iPhone14 pro, MI Mix2s, OPPO Reno5 pro) and a ReSpeaker Mic array. It also showed transferability, with perturbations optimized for Mix2s transferring to Pixel 3aXL (94.2% SR) and Reno5 pro (83.3% SR).
• Speech & Perturbation Loudness: VRIFLE achieved over 55% SR when perturbation volume matched benign audio, and up to 95.5% SR when 5 dB higher. When 10 dB higher, it dominated all user commands.
• Portable Device and Off-the-shelf Loudspeaker: A self-made portable device achieved 100% SR within 150 cm and 78% SR at 180 cm. Using a high-end Hivi loudspeaker with embedded USB-AM perturbations, an effective attack distance of approximately 20 cm with a low CER of 11.07% was achieved.

Anti-Defense Experiment:

VRIFLE was tested against six representative defenses, considering both naive and adaptive adversaries.

• Audio Pre-processing Methods:
• Quantization, VAD, Opus Compression: These defenses barely affected VRIFLE's attack success rate (all 95.93% SR). VAD even increased benign CER while failing to lower attack performance.
• Band-pass Filter: Naive VRIFLE maintained effectiveness (97.96% SR) with a 50-7000 Hz filter. An adaptive adversary significantly boosted performance, achieving 78.88% SR and 5.34% CER even under a 50-3000 Hz filter.
• Down-sampling: Naive VRIFLE maintained effectiveness (98.98% SR) for a 0.9 rate. An adaptive adversary achieved 81.42% SR and 4.38% CER against a 0.4 down-sampling rate.
• Inaudible Attack Detection Method (LipRead 21): LipRead, which extracts features like sub-50Hz power, correlation coefficient, and amplitude skew, showed a detection accuracy down to 45.07% against naive VRIFLE. An adaptive adversary, by constraining these features during perturbation generation, further reduced LipRead's detection accuracy to 30.55%. VRIFLE's compact skewness around 1.0 (due to its symmetrical waveform) makes its distribution closer to normal speech, compromising LipRead.

Defensive Implications

▶ Watch: Challenges in modeling nonlinear ultrasonic delivery for attacks (8:00)

VRIFLE's demonstrated effectiveness and resilience against existing countermeasures present significant implications for defenders of ASR systems. The traditional approaches to mitigating adversarial audio, such as common audio pre-processing techniques (quantization, VAD, compression, simple band-pass filters, down-sampling), are largely ineffective against VRIFLE, especially when faced with an adaptive adversary. This suggests that current defense strategies are insufficient to protect against sophisticated inaudible attacks.

The research highlights the critical need for more advanced and potentially hardware-dependent defenses. Defenses that explicitly track the unique nature of ultrasound, such as those mentioned in prior work (49, 50), might be effective, but their hardware dependency can limit their broad applicability and increase deployment costs. Furthermore, while ML-based defenses (15, 51) could be considered, the study notes that an adaptive adversary could potentially integrate these defenses into their perturbation crafting process, thereby reducing attack universality but still maintaining effectiveness.

Defenders should recognize that the adversary can model the physical channel and adapt their perturbations to bypass detection. This implies that static, pre-defined defense mechanisms will likely fail. Future defensive strategies must consider dynamic, adaptive, and possibly hardware-integrated solutions that can specifically detect and mitigate the effects of nonlinear microphone demodulation and the subsequent low-frequency acoustic features induced by ultrasound. Monitoring for anomalous noise patterns or subtle spectral shifts that are not characteristic of normal speech, even after demodulation, could be a starting point. Additionally, continuous research into understanding the full spectrum of microphone vulnerabilities and developing robust signal processing techniques to filter out or neutralize these specific inaudible interferences is paramount.

Key Takeaways

• VRIFLE introduces a novel inaudible adversarial perturbation (IAP) attack that can manipulate ASR systems in real-time, even during active user speech, without user awareness.
• It overcomes significant challenges of prior work, including audibility, limited universality, and user disruption, by precisely modeling the nonlinear ultrasonic transformation process.
• VRIFLE achieves high universality, capable of altering arbitrary user commands to a desired intent or silencing them (alter-and-mute strategy), demonstrating robustness across various environments, distances (up to 10m), angles, and ambient noise levels.
• The attack is highly resilient against common audio pre-processing defenses and significantly compromises a dedicated inaudible attack detection method, LipRead (21), especially when the adversary adapts their strategy.
• VRIFLE's ability to operate with portable devices and even off-the-shelf loudspeakers, combined with its complete inaudibility, reveals a potent and practical new attack surface against ASR models.
• Defensive strategies must evolve beyond simple audio pre-processing to consider hardware-dependent solutions or sophisticated adaptive ML-based defenses to counter these advanced, stealthy threats.

About the Speaker(s)

The talk was presented by Xinfeng Li. Based on the provided transcript, Xinfeng Li is a researcher actively involved in the field of adversarial machine learning and security, specifically focusing on the vulnerabilities of Automatic Speech Recognition systems. No further biographical details are available in the provided text.