noRecognition: Could a Pattern on Clothing Fool Facial Recognition?

noRecognition: Could a Pattern on Clothing Fool Facial Recognition?

Speakers: Bill Swearingen (Security Researcher, SecKC)

Conference: DEF CON

YouTube: https://www.youtube.com/watch?v=WyPmt8CE5L4

Overview

In his compelling DEF CON talk, "noRecognition: Could a Pattern on Clothing Fool Facial Recognition?", Bill Swearingen, a security researcher from SecKC, delves into the critical and rapidly escalating issue of mass surveillance in the United States. Swearingen meticulously details his year-long research into adversarial patterns designed to disrupt and confuse modern AI-powered camera systems, specifically targeting facial recognition and person/face detection. The core premise explores whether a strategically designed pattern, when worn on clothing, can render an individual effectively invisible or unidentifiable to these pervasive surveillance technologies.

Swearingen's motivation stems from profound concerns regarding privacy erosion, the lack of informed consent in data collection (such as driver's license photos being enrolled in facial recognition databases), and the documented biases inherent in these systems. He highlights the alarming rate of false positives for darker-skinned individuals (10 to 100 times higher than for white males), which exacerbates issues of over-policing and misidentification. The talk focuses heavily on the ubiquitous Flock Safety cameras, initially marketed as Automated License Plate Recognition (ALPR) systems but now functioning as comprehensive mass surveillance tools, collecting extensive metadata and often being misused by law enforcement without proper warrants.

This research matters because it positions the hacker community as a vital counterbalance against unchecked technological power. By demonstrating practical methods to subvert these systems, Swearingen not only exposes their vulnerabilities but also empowers individuals with potential tools for privacy protection. The article will explore the technical intricacies of AI detection, the speaker's innovative methodology involving Deep Reinforcement Learning (DRL) and community collaboration, the demonstrated proof of concept, and the significant defensive implications for both individuals and the industry.

Background

▶ Watch: Introduction: Mass surveillance and facial recognition bias (0:00)

The United States is increasingly becoming a mass surveillance state, a reality that often goes unnoticed or unacknowledged by the public. Swearingen initiates his talk by revealing that many citizens are unknowingly enrolled in facial recognition systems when they obtain a driver's license – a process for which explicit permission is rarely sought. This involuntary enrollment forms the bedrock of a burgeoning surveillance infrastructure that presents significant ethical and societal challenges.

A critical issue highlighted by Swearingen is the inherent bias within current facial recognition technologies. While these systems perform "pretty good" for individuals matching the demographic profile of a 50-year-old white male, darker-skinned individuals experience anywhere from 10 to 100 times more false positives. This alarming disparity carries severe consequences, particularly for communities already subject to disproportionate policing, where incorrect identification can lead to wrongful accusations and severe infringements on civil liberties.

A central antagonist in Swearingen's narrative is Flock Safety, a private company that has deployed over 120,000 cameras across the U.S., performing an astonishing 20 billion vehicle scans per month. These cameras were initially sold to neighborhoods and homeowner associations as ALPR (Automated License Plate Recognition) systems, ostensibly to track stolen cars. However, Swearingen presents compelling evidence, including video footage extracted by Ben Jordan, demonstrating that Flock cameras capture far more than just license plates. They track individuals, zoom in on phones with enough resolution to potentially read screen content, and collect extensive metadata on vehicles, including color, condition, bumper stickers, and damage. This data, owned by a private company, is made available to police forces nationwide, often without requiring a warrant, leading to widespread concerns about privacy and potential abuse. Swearingen cites instances of law enforcement officers misusing the system, such as a police chief in Wichita, Kansas, who queried his former partner and her new boyfriend over 200 times.

Historically, independent researchers and the hacker community have served as a crucial counterbalance to vendor claims of security and governmental assurances of oversight. From exposing vulnerabilities in telephone signaling and electronic voting machines to vehicle CAN bus systems, conferences like DEF CON have repeatedly demonstrated the true security posture of technologies. Swearingen frames his research within this tradition, aiming to expose the real-world vulnerabilities of AI cameras and provide a pushback against unchecked surveillance. While acknowledging prior work in adversarial pattern generation—such as Adam Harvey's groundbreaking CV Dazzle from 2010, which used makeup to confuse AI models—Swearingen notes that much of this previous research was conducted on "extremely old models" and "extremely small models" that do not accurately represent the sophisticated cameras deployed today. This observation provided a strong foundation for his work, prompting him to develop techniques effective against contemporary, commercially deployed systems.

Key Findings

▶ Watch: Flock's massive scale: 120k cameras, 20 billion scans (4:00)

Bill Swearingen's research yielded several critical findings that underscore the fragility of modern AI-powered surveillance systems and offer tangible pathways for privacy protection:

• Digital Patterns Disrupt AI Detection: The most significant finding is that digitally generated patterns, when applied to clothing, can dramatically reduce the confidence scores of AI models in detecting persons and faces. This doesn't make the person disappear but rather makes the AI uncertain of what it's seeing.
• Face Detection is the Achilles' Heel of Facial Recognition: A crucial realization was that facial recognition systems are fundamentally dependent on successful face detection. If a pattern can prevent a camera from detecting a face, it inherently prevents the system from identifying who that face belongs to. This insight allowed Swearingen to focus his attack vector more surgically.
• Cross-Model Effectiveness: Through extensive testing, Swearingen demonstrated that certain adversarial patterns are not only effective against single models but can work across multiple, diverse camera models (e.g., person detectors, face detectors, and facial recognition systems from different manufacturers). Out of 31.7 million patterns tested, 480,000 worked across two or more models, and 85 were effective across three or more.
• Vulnerable Body Areas: The research identified specific regions on the human body that, when adorned with an adversarial pattern, are particularly effective at confusing detectors. The chest area is critical for person detection, while the neck region proved uniquely vulnerable, as disrupting its detection could simultaneously impair both person and face detection.
• All Tested Models Defeated (in simulation): In digital simulations, every single one of the 11 modern AI camera models integrated into Swearingen's testing gauntlet—comprising five person detectors, four face detectors, and two facial recognition systems—was successfully defeated by his generated patterns. While he refrains from publicly disclosing the most potent patterns to prevent immediate retraining by camera manufacturers, this finding highlights the widespread vulnerability.
• Deep Reinforcement Learning (DRL) for Pattern Generation: Swearingen's implementation of a DRL system proved exceptionally effective, generating new, improved adversarial patterns at an astonishing rate (approximately 1,000 patterns per minute). This adaptive and continuously evolving engine represents a significant leap in the automated discovery of anti-surveillance patterns.

Technical Deep Dive

▶ Watch: Flock cameras track people, not just license plates (5:30)

Swearingen approaches the problem of AI camera evasion from a hacker's perspective, viewing the camera as a parser. Just as an attacker seeks to control the input to a software parser to manipulate its output, he aims to control the visual input to a camera to dictate its detection outcome. His initial methodology, though rudimentary, laid the groundwork for more sophisticated techniques. He started with a simple Python script, generating six distinct personas, which were digitally imprinted with 61 different pattern generators (including QR codes and injection strings) onto green-screened t-shirts. The script then queried an AI model, asking "how many people are in this image?", "how many faces?", and "have you seen this person before?" The discovery of "anomalies" – extra people, no people, or incorrect counts – from merely changing the digital pattern on clothing was the first indication of a vulnerability.

The core of AI detection in cameras, as Swearingen explains, typically involves a pipeline of three types of detectors:

1. Person Detector: Identifies the presence and count of people in an image.
2. Face Detector: Identifies the presence and count of faces in an image.
3. Facial Recognition: Identifies specific individuals based on detected faces.

A critical insight gained from his research was that facial recognition systems are fundamentally chained to face detection. If a face detector fails to identify a face, the facial recognition component cannot operate. This allowed Swearingen to focus his efforts on disrupting face detection, knowing it would cascade into facial recognition failure.

Cameras don't "see" in a human sense; they process images and assign confidence scores. For instance, a camera might analyze an image and determine, "I think this is a person, and I am 82% sure." Swearingen identified two primary attack surfaces within this confidence calculation, particularly for systems like the Flock Falcon camera. These cameras shrink images (e.g., to 320x320 pixels), overlay a grid, and then make guesses based on two factors:

• Objectness: A binary determination of whether something exists in a grid cell (e.g., "something" vs. "pavement").
• Class Probability: The likelihood that the detected "something" belongs to a specific class (e.g., "I see a motorcycle at 80% confidence").

These two probabilities are multiplied to yield the final confidence score. Swearingen realized that attacking either the objectness or the class probability could effectively lower the overall confidence, thus creating two distinct opportunities for subversion.

To identify the underlying models used in commercial cameras, Swearingen undertook an extensive effort to gather SBOMs (Software Bill of Materials) and licensing information. This allowed him to deduce common models like YOLO or SSD MobileNet used in systems such as Hikvision cameras. With this intelligence, he constructed a "gauntlet" of models for testing. Initially comprising 10 models, this gauntlet expanded to include five person detectors, four face detectors, and two facial recognition systems. Running tests through this comprehensive gauntlet was computationally intensive, dropping his initial 700 tests per minute to just four.

This computational bottleneck was overcome through a remarkable display of hacker community collaboration. Volunteers from around the world, including individuals like "Coupe" from Corn Con, donated access to their high-powered GPUs, some even disassembling their gaming computers to send components. This collective effort enabled Swearingen and his team to test an astonishing 31.7 million patterns, leading to the discovery of 534,000 anomalies. Of these, 480,000 patterns were effective across two or more models, and 85 patterns demonstrated effectiveness across three or more – a small but significant fraction (0.0016%) indicating a viable attack surface.

Recognizing the inefficiency of brute-force pattern generation, Swearingen developed an evolution engine. This system "mated" successful patterns from the person and face detectors, aiming to evolve more potent, cross-functional patterns. The true breakthrough, however, came with the adoption of Deep Reinforcement Learning (DRL). Using the 31 million previously generated patterns as training data, Swearingen "taught the DRL to paint." This DRL system continuously generates new patterns, learning from both successes and failures, at a rate of roughly 1,000 improved patterns per minute. To ensure the robustness and generalizability of his findings, he significantly expanded his persona image dataset to include a diverse range of body types, races, sexes, and skin colors.

The mechanism by which these adversarial patterns work is not to make the wearer "disappear." Instead, the patterns inject digital noise in specific, repeating ways that disrupt the features the AI model was trained to recognize. For example, a human body has distinct angles, curves, and contrast differences (e.g., between the shoulder and background). The patterns are designed to interfere with these expected features, causing the model to lower its confidence from "I am 80% sure this is a person" to "I don't know what that is." Swearingen found that while more pattern coverage was initially thought to be better, specific areas like the chest for person detection and the neck for both person and face detection proved to be highly effective targets for pattern placement.

Demo / Proof of Concept

▶ Watch: Police chief misused Flock data to stalk ex-partner (7:00)

Swearingen's talk included a live demonstration of his adversarial patterns in action, which served as a compelling proof of concept for his research. Early in the presentation, he attempted to show the audience how a pattern on his clothing would affect the detection confidence of a Flock camera positioned on stage. While initially hampered by lighting conditions, the camera eventually began to detect him. Once the pattern was introduced into the camera's view, the confidence score for person detection visibly dropped, eliciting cheers from the audience. This real-time interaction, despite its initial hiccups, vividly illustrated the core premise: a specially designed pattern can indeed confuse and reduce the certainty of a live AI surveillance system.

Beyond the live demonstration, Swearingen also referenced earlier, simulated proofs of concept. His initial Python script digitally imprinted patterns onto green-screened t-shirts worn by personas. This simulated environment, running on an M1 MacBook at 700 tests per minute, consistently produced "anomalies," such as the detection of "extra people" or "no people" where a person was clearly present. These early digital experiments provided the foundational evidence that patterns alone, without any physical alteration to the subject, could significantly disrupt AI detection models.

It is important to note that while the live demonstration showed the pattern's immediate effect on confidence, Swearingen acknowledges that the research is still progressing towards full physical testing. As of the talk, the patterns had primarily existed in digital simulation on GPUs. The next critical step, which he is funding through a Kickstarter campaign, involves physically printing these optimized patterns onto fabric and conducting real-world tests in various environments. This future work will further validate the practical efficacy of his findings beyond digital simulations and controlled live stage demonstrations.

Defensive Implications

▶ Watch: Hackers as counterbalance against mass surveillance systems (8:40)

The research presented by Bill Swearingen has profound implications for both individuals seeking to protect their privacy and for the developers and deployers of AI surveillance technologies.

For individuals and privacy advocates, Swearingen's work offers a tangible, albeit developing, tool for evading mass surveillance. The concept of wearing specially designed clothing to confuse facial recognition and person detection systems provides a potential means to reclaim anonymity in public spaces increasingly monitored by AI cameras. This could be particularly impactful for individuals in communities disproportionately affected by biased facial recognition systems, offering a layer of protection against misidentification and unwarranted scrutiny. The "noRecognition" t-shirt, once physically realized, could become a symbol and a practical method of digital self-defense.

For camera manufacturers and AI developers, Swearingen's findings represent a critical challenge to the robustness and reliability of their models. The fact that all 11 tested modern AI models were defeated in simulation indicates a widespread vulnerability. This necessitates a fundamental re-evaluation of model training, adversarial robustness, and deployment practices. While manufacturers could attempt to retrain their models on Swearingen's patterns, his Deep Reinforcement Learning (DRL) system continuously generates new, improved patterns at a rapid pace (1,000 per minute), creating an ongoing adversarial arms race. This suggests that simply retraining might only offer temporary fixes, pushing the industry towards more sophisticated, multi-modal, and truly robust detection architectures that are inherently harder to fool.

For law enforcement and government agencies that rely on systems like Flock Safety, the research highlights the inherent limitations and potential for subversion of their surveillance tools. The reliance on AI for identification and tracking becomes less reliable when individuals can adopt simple, accessible countermeasures. This should prompt a re-evaluation of policies regarding the deployment and use of such technologies, emphasizing the need for warrant requirements and stricter oversight to prevent misuse. The ease with which these systems can be confused also raises questions about the validity of evidence derived from them, especially in legal contexts where accuracy and reliability are paramount.

Beyond direct evasion, the research also contributes to a broader understanding of AI ethics and accountability. By publicly demonstrating these vulnerabilities, Swearingen compels a necessary societal conversation about the trade-offs between security, privacy, and the unchecked proliferation of surveillance technology. It reinforces the argument that AI systems, particularly those with significant societal impact, must be subject to independent security audits and rigorous testing against adversarial attacks, rather than relying solely on vendor assurances. The work underscores the importance of the hacker community's role in holding powerful institutions accountable and ensuring that technological advancements do not inadvertently erode fundamental rights.

Key Takeaways

• Pervasive Mass Surveillance: Systems like Flock Safety have created a widespread mass surveillance state, collecting vast amounts of data (e.g., 20 billion vehicle scans/month) often without public consent or adequate oversight, leading to privacy concerns and documented misuse by law enforcement.
• AI Bias and Disproportionate Impact: Facial recognition systems exhibit significant biases, with darker-skinned individuals facing 10 to 100 times more false positives, exacerbating issues of over-policing and misidentification in vulnerable communities.
• Adversarial Patterns are Effective: Digitally designed patterns, when applied to clothing, can significantly reduce the confidence of modern AI person and face detection models, effectively preventing facial recognition in simulated environments.
• Hacker Community's Role and Advanced Techniques: Collaborative efforts from the hacker community, combined with sophisticated AI techniques like Deep Reinforcement Learning (DRL), are crucial for continuously generating and discovering new, potent adversarial patterns at scale (e.g., 1,000 new patterns per minute).
• Vulnerability of Current AI Models: All 11 tested modern AI camera models (person, face, and facial recognition detectors) were defeated in simulation, highlighting a widespread vulnerability in current commercial surveillance technologies.
• Need for Real-World Validation and Ethical Scrutiny: While highly successful in simulation, the next critical step is physical testing of these patterns on fabric. This research underscores the urgent need for greater transparency, accountability, and robust security measures in the development and deployment of AI-powered surveillance systems.

About the Speaker(s)

Bill Swearingen is a dedicated Security Researcher affiliated with SecKC. His presentation at DEF CON highlights his passion for addressing critical issues at the intersection of technology, privacy, and civil liberties, particularly concerning mass surveillance. Swearingen describes himself as a hacker, emphasizing the community's traditional role in independently evaluating and challenging the security claims of systems.

Despite initially having no prior experience in machine vision, Swearingen embarked on this year-long research project with a keen interest in understanding and subverting AI cameras. His journey reflects a self-taught, iterative approach, starting from basic Python scripting and evolving to sophisticated Deep Reinforcement Learning. He is a strong advocate for community collaboration, crediting numerous individuals from the hacker community for contributing compute power and support to his extensive pattern testing efforts. Swearingen's work exemplifies the independent, analytical, and often challenging spirit of security research within the hacker ethos.