FlyTrap: Physical Distance-Pulling Attack Towards Camera-based Autonomous Target Tracking Systems

FlyTrap: Physical Distance-Pulling Attack Towards Camera-based Autonomous Target Tracking Systems

Speakers: Shaoyuan Xie

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=tgJCtZCeMi0

Overview

FlyTrap is the first distance-pulling attack against camera-based autonomous drone tracking systems. By printing adversarial patterns on an ordinary umbrella ($20 on Amazon), an attacker can trick a tracking drone into progressively moving closer to the target by spoofing a smaller bounding box -- causing the drone's control loop to misinterpret the target as moving farther away. The attack achieves 100% success rate at pulling drones within 0.5 meters in closed-loop experiments and transfers to three commercial drones: DJI Mini4 Pro, DJI Neo, and Hover Air X1.

The research addresses three key challenges: designing a physically deployable outdoor attack vector, ensuring effectiveness throughout the drone's closed-loop control logic, and bypassing existing spatial-temporal defenses. The progressive distance-pulling (PDP) design simulates the entire closed-loop control process during optimization, significantly outperforming baseline approaches by 48-58% in attack success rate. Responsible vulnerability disclosure was conducted with DJI and Hover Air.

Background

▶ Watch: How autonomous drone tracking works: distance and lateral control (0:00)

Autonomous target tracking drones are widely deployed for both legitimate purposes (law enforcement tracking suspects) and potentially malicious uses (illegal stalking). Understanding their security vulnerabilities is critical for public safety. These systems operate through a two-part control loop:

Distance control: The drone maintains a target distance by monitoring the area of the tracking bounding box. If the bounding box shrinks (indicating the target is moving away), the drone moves forward to compensate. If it grows, the drone retreats.

Lateral control: The drone keeps the bounding box centered in the image frame to maintain stable tracking.

This work focuses on distance control because it enables the most severe attack consequences: pulling the drone within range of a net gun (9 meters), within range of sensor attacks (like acoustic DoubleStar attacks), or within physical crash distance (0.5 meters).

Prior adversarial attacks on object tracking focused only on the perception model in isolation, ignoring the control loop. They also used impractical attack vectors (TV screens, projectors) unsuitable for outdoor deployment. Existing defenses include PerceptGuard (using LSTM to analyze bounding box trajectories) and a pose estimation defense (checking for human pose within the bounding box).

Key Findings

▶ Watch: Live video demo of adversarial umbrella pulling drone closer (4:00)

Adversarial umbrella patterns achieve reliable distance-pulling. By printing optimized adversarial patterns on an umbrella, the attacker can continuously shrink the drone's perceived bounding box while the actual physical distance remains unchanged, causing the drone to approach the target.

Progressive Distance Pulling (PDP) improves success rate by 58%. By simulating the closed-loop control process during attack optimization -- zooming into images to simulate the drone's progressive approach -- the attack maintains effectiveness throughout the entire distance-pulling trajectory, not just at the initial distance.

100% success rate at pulling drones within 0.5 meters. In real-world closed-loop experiments with implemented drone systems, the PDP design achieved 100% success at bringing drones within physical crash distance.

Transfer to three commercial drones. Without any knowledge of the commercial systems' internal models, the adversarial umbrella achieved over 60% success rate pulling DJI Neo and Hover Air X1 within 0.5 meters, and approximately 30% success rate pulling DJI Mini4 Pro within 6 meters.

Attack generalizes beyond training data. The adversarial pattern works on unseen backgrounds and person identities not present during optimization, and generalizes beyond 30 meters even though training data only included distances up to 20 meters.

The attack bypasses existing defenses. By carefully controlling bounding box size to maintain injected visual features (human pose, trajectory consistency), the attack circumvents both PerceptGuard and pose estimation defenses.

Technical Deep Dive

▶ Watch: Progressive Distance Pulling: simulating closed-loop control (10:00)

The attack pipeline consists of several sophisticated components:

Umbrella rendering: A UV mapping establishes the relationship between 2D adversarial patch coordinates and 3D vertices of an umbrella mesh model. Differentiable rendering simulates how the umbrella appears from the drone's camera viewpoint at various angles and distances. The rendered umbrella is cropped and composited into training images at the correct position.

Progressive Distance Pulling (PDP): The initial drone-target distance is estimated using the pinhole camera model. Intermediate distances (from the initial distance down to zero) are simulated by zooming into the image, approximating the drone's progressive approach. At each simulated distance, a theoretically derived formula determines the required bounding box size -- small enough to trigger forward movement but large enough to preserve visual features needed to bypass defenses.

Defense bypass: At each simulated distance frame, the attack objective ensures: (1) the bounding box is predicted at the calculated target size, (2) a human pose can still be estimated within the bounding box (bypassing pose estimation defense), (3) the bounding box aspect ratio and trajectory remain consistent with real human tracking (bypassing LSTM-based trajectory defense).

Robustness: Expectation over Transformation (EOT) is applied during optimization to ensure the adversarial pattern withstands varying lighting conditions, viewing angles, and environmental factors in real-world deployment.

The evaluation dataset contains 16 combinations (4 individuals x 4 backgrounds), tested against both convolutional-based and transformer-based tracking models. Both white-box (full knowledge) and black-box (surrogate model transfer) threat models are evaluated.

Demo / Proof of Concept

▶ Watch: Evaluation: 100% success rate at 0.5m, transfers to commercial drones (12:00)

A live video demonstration showed the complete FlyTrap attack in action, with a person carrying the adversarial umbrella causing a tracking drone to progressively approach and come within close physical proximity.

Physical world evaluation results:

• Net gun range (9m): Successfully pulled drones within range for net capture
• Sensor attack range (DoubleStar): Successfully pulled drones within sensor attack distance
• Crash range (0.5m): 100% success rate with PDP design at pulling drones within physical striking distance

Commercial drone transfer:

• DJI Neo: Over 60% success rate pulling within 0.5 meters
• Hover Air X1: Over 60% success rate pulling within 0.5 meters
• DJI Mini4 Pro: ~30% success rate pulling within 6 meters

Cost: The adversarial umbrella costs approximately $20, created by uploading the trained pattern to an Amazon umbrella customization service.

Limitation noted in Q&A: The attack only targets camera-based tracking systems. Drones equipped with LiDAR sensors for distance measurement would not be affected, though the researchers note that most consumer-grade tracking drones rely solely on cameras.

Defensive Implications

▶ Watch: Q&A: LiDAR defense and autonomous driving applicability (18:00)

This research has significant implications for both drone manufacturers and organizations that rely on drone tracking systems:

Camera-only tracking is fundamentally vulnerable. Any tracking system that infers distance solely from bounding box area is susceptible to adversarial perception attacks. Drone manufacturers should consider multi-modal distance estimation (camera + LiDAR, camera + acoustic) to create sensor redundancy that adversarial patches cannot defeat.

Existing defenses are insufficient. Both PerceptGuard and pose estimation defenses are bypassed because the attack carefully maintains the visual features these defenses check. More robust defenses need to verify physical consistency across multiple independent sensors rather than relying on additional neural network analysis of the same camera feed.

Law enforcement implications. Police departments using autonomous drones for suspect tracking should be aware that a $20 adversarial umbrella could compromise their tracking systems. This has direct operational implications for pursuit scenarios.

Autonomous driving parallel. The Q&A confirmed that the same attack principle applies to autonomous vehicles that maintain following distance based on bounding box area. This extends the attack surface significantly beyond drones.

Adversarial training and certified robustness for single-object tracking models are suggested as future defenses, but must satisfy real-time efficiency requirements for autonomous drone operations.

Key Takeaways

• A $20 adversarial umbrella can trick tracking drones into approaching the target, achieving 100% success rate at pulling within 0.5 meters in closed-loop experiments
• The Progressive Distance Pulling (PDP) design improves attack success by 58% by simulating the entire closed-loop control process during optimization
• The attack transfers to three commercial drones (DJI Mini4 Pro, DJI Neo, Hover Air X1) in black-box settings
• Existing defenses (PerceptGuard, pose estimation) are bypassed by maintaining visual feature consistency while shrinking the bounding box
• The attack generalizes to unseen persons, backgrounds, and distances beyond the training data
• Multi-modal sensing (camera + LiDAR) would defend against this attack, but most consumer drones use camera-only tracking
• The same attack principle applies to autonomous driving following-distance systems

About the Speaker(s)

Shaoyuan Xie is a PhD student in the ASCAR group at UC Irvine. Collaborators include Ju, Faiza, Ningfei, Takami, Hale Lima, Professor Avaruk, and Professor Chafan. The research group focuses on adversarial machine learning attacks against autonomous systems, with expertise in physical-world adversarial attacks, drone security, and perception-control loop vulnerabilities.