On Precisely Detecting Censorship Circumvention in Real-World Networks

On Precisely Detecting Censorship Circumvention in Real-World Networks

Speakers: Ryan Wails

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=8uAycFpJ2_A

Overview

This talk, presented by Ryan Wails at the NDSS Symposium, addresses a critical and often underestimated challenge in the cat-and-mouse game of internet censorship: the precise detection of censorship circumvention systems (CRSes) in real-world networks. The core motivation stems from the widespread use of internet censorship by over 60 countries, impacting fundamental human rights and freedom of communication, as evidenced by recent events in China, Iran, and Ukraine. The research aims to advance a free and open internet by providing a deeper understanding of how sophisticated censors can effectively identify and block circumvention traffic, thus enabling the design of more resilient CRSes.

The central problem for censors is the base rate challenge: the vast majority of network traffic is benign, making it incredibly difficult to distinguish rare circumvention flows without generating an overwhelming number of false positives that would block legitimate traffic and cause unacceptable collateral damage. Traditional machine learning approaches, even with high reported accuracy, fall short under realistic conditions. Wails' work introduces a novel perspective by shifting the detection focus from individual circumventing flows to the detection of circumventing hosts. This host-centric approach, which accumulates information from multiple flows over time, significantly enhances precision, offering a more viable and scalable strategy for real-world censors. The findings provide crucial insights for both censors seeking to improve detection and CRS developers striving to build more robust systems.

Background

▶ Watch: Introduction to censorship problem and base rate challenge (0:00)

Internet censorship is a multifaceted problem where state-level adversaries employ various techniques, including IP filtering, DNS hijacking, active probing, and deep packet inspection (DPI), to restrict access to information and services. The talk specifically focuses on fingerprinting—the process of classifying traffic based on its unique features—as a primary method for censors to identify and block prohibited flows. In response, researchers and developers have created CRSes designed to evade such fingerprinting through techniques like protocol mimicry, tunneling, and randomization.

The research centers on Tor, the most widely used CRS framework. Tor operates by routing internet communication through a network of volunteer proxy servers (entry, middle, and exit relays) to obscure a user's origin and destination. Censors often target Tor by identifying and blocking these proxy IP addresses. To counter this, Tor utilizes bridge relays, whose addresses are not publicly advertised, and pluggable transports, which obfuscate the traffic between the client and the bridge. The study specifically investigates obfs4 and Snowflake, two popular pluggable transports. Obfs4, part of the obfsproxy family, aims to make traffic indistinguishable from random bytes using authenticated Diffie-Hellman key exchange. Snowflake, a unique WebRTC-based proxy, leverages volunteers' web browsers as ephemeral proxies, with clients locating them via a domain-fronted broker and establishing connections using public STUN servers.

The adversary model assumes a stateful, on-path censor capable of storing information about flows and hosts over time. While IP addresses and ports are not used as features for flow detection, they are crucial for host identification. The evaluation relies on standard classifier metrics: true positives (TP), false positives (FP), true negatives (TN), false negatives (FN), precision, and recall. A paramount focus is placed on minimizing false positives to reduce collateral damage, as a censor's tolerance for blocking legitimate traffic is extremely low due to the base rate challenge—the overwhelming prevalence of benign traffic compared to circumvention traffic. Prior work in this domain has explored various fingerprinting techniques, including website fingerprinting using deep learning and application-based fingerprinting, which shares conceptual similarities with the proposed host-based approach. However, many existing evaluations of CRS detection methods suffer from unrealistic assumptions regarding base rates and benign traffic diversity, leading to overly optimistic performance claims.

Key Findings

▶ Watch: Summary of research contributions and novel evaluation methods (1:45)

The research yielded several critical findings that challenge existing assumptions about censorship circumvention detection and propose a more effective paradigm:

1. Limitations of State-of-the-Art (Classical ML): Current classical machine learning methods for detecting obfuscated protocols like obfs4 are fundamentally flawed under realistic conditions. Even a replicated decision tree classifier achieved 98% True Positive Rate (TPR) but a 6% False Positive Rate (FPR). More critically, the FPR significantly worsened in "open-world" scenarios (14-200% higher) and especially in the "long tail" of less common protocols (71-333% higher), where circumvention traffic often tries to hide. Furthermore, the precision and F1-score of these classifiers approached zero at even a modest base rate of λ = 1,000, becoming effectively useless at realistic base rates like λ 10⁶. These classifiers were also easily evaded by trivial modifications to the protocol, such as the obfs* variant, which completely degraded performance to 0% precision and recall.

1. Deep Learning Improvements, but Still Insufficient for Flow-Based Detection: Deep learning techniques, specifically a Convolutional Neural Network (CNN) adapted from website fingerprinting research, demonstrated significantly improved performance over classical methods. For obfs4, the CNN achieved a near-perfect TPR of 1.0 and an FPR of 2.9 × 10⁻³, an order of magnitude improvement in false positives. For obfs*, performance was even better, with 100% recall and an FPR of 7 × 10⁻⁴. While impressive, these FPRs are still prohibitively high for real-world censors operating at internet scale with extremely low base rates. At λ 10⁶, the precision for deep learning classifiers also approached zero, indicating an unacceptable level of collateral damage.

1. Host-Based Analysis as a Game Changer: The most significant finding is that shifting the detection target from individual flows to circumventing hosts dramatically enhances precision and makes detection feasible for censors. By accumulating observations from multiple flows associated with a specific host over time, the system can achieve a much lower effective base rate and significantly higher confidence. The proposed host-based meta-classification algorithm, which tracks the fraction of positive flow classifications for an (IP, Port) pair, effectively reduces false positives to near-zero levels. For obfs4, the total number of false positives dropped to less than 1 on average after observing no more than 30 flows. For obfs*, false positives reached 0 after 38 flows. The actual host FPRs achieved were 2.4 × 10⁻⁶ for obfs4 and 1.5 × 10⁻⁶ for obfs*—two orders of magnitude smaller than the flow error rates and matching the desired error rate α = 1 × 10⁻⁶. All 15 self-hosted bridges in the experiment were successfully identified.

1. Computational Feasibility for Nation-State Adversaries: While deep learning requires more computational resources (e.g., 50 GPUs costing ~$250,000 for an internet backbone link), this cost is deemed feasible for powerful nation-state censors. The subsequent host-based analysis is computationally inexpensive, requiring minimal state (e.g., 2 log₂(η) bits per host) and storage (e.g., 500 TiB for all IPv4 addresses and TCP ports, which can be further optimized).

Technical Deep Dive

▶ Watch: Background on Internet censorship and Tor's role (2:00)

The research meticulously details the methodology for evaluating censorship circumvention detection, starting with data collection, moving through classical and deep learning flow-based classification, and culminating in the novel host-based analysis.

Data Collection:

To achieve realism, a private dataset of network flow statistics was collected from a university-hosted network observation point, capturing traffic from its WiFi network. The setup included two 10 Gbps NICs, PF_RING with zero-copy support, and a custom C++ packet processing program. Only IPv4 TCP and UDP packets were captured, with flows defined by their five-tuple and up to 5,000 packets recorded per flow. Crucially, raw packet data was not stored; instead, various statistics (general info, summary statistics like entropy and timing, individual packet payload sizes) were computed. IPv4 addresses were anonymized using HMAC with a discarded key to ensure privacy.

Alongside benign background traffic, obfuscated protocol flows were generated using 8 crawler machines on the same WiFi network. These crawlers accessed popular webpages via Selenium-driven Tor Browser instances, randomly selecting sites. Each crawler used either obfs4, a tweaked obfs* protocol (modifying per-packet entropy and sizes), or Snowflake. Self-operated obfs4 and obfs* bridges were hosted on Google Cloud. This co-located client setup minimized artificial distinctions between obfuscated and benign traffic.

The two-week collection in March-April 2022 yielded:

• 54,355,226 background flows to ~600,000 unique destination pairs.
• 83,002 obfs4 flows.
• 207,975 obfs* flows.
• 5,894,149 Snowflake flows.

The dataset revealed a high prevalence of HTTPS (TCP 443) and DNS (UDP 53) traffic, a significant "long tail" of flows to less popular ports, and typical diurnal patterns.

Flow-Based Classification (Classical ML):

The problem was framed as a supervised binary classification task. The dataset was split 40% for training and 60% for testing, ensuring no destination host overlap. Classifiers were trained on 1,500 randomly sampled negative and 1,500 positive flows, stratified by destination port.

Prior work by Wang et al. (79) on obfs4 detection using decision trees with manually defined features (entropy, packet header, timing from the first 30 packets) was replicated. Using entropy-based features, 98% TPR and 6% FPR were achieved. However, several pitfalls were identified:

1. FPR Misrepresentation: FPRs were significantly higher in "open-world" scenarios (14-200% higher) and the "long tail" (71-333% higher) than reported, meaning classifiers performed worst where evasive protocols often hide.
2. Scalability at Low Base Rates: Precision and F1-score plummeted to near-zero at modest base rates (λ = 1,000), becoming unusable at realistic base rates (λ 10⁶).
3. Lack of Generalizability: The obfs* protocol, a trivial modification to obfs4, completely degraded the hand-tuned classifier's performance to 0% precision and recall, demonstrating vulnerability to evasion.

Flow-Based Classification (Deep Learning):

To overcome classical ML limitations, three neural networks previously used for website fingerprinting were adapted: a stacked denoising autoencoder (SDAE) and two convolutional neural networks (CNNs) by Rimmer et al. (66) and Sirinam et al. (68). Input consisted of sequences of normalized packet sizes and directions (p_i in -1,1), where the sign indicated direction. Packet sizes, not just directions, improved performance. Various input dimensions (n=100, 500, 1000, 5000 packets per flow) were tested.

The Sirinam et al. CNN model consistently performed best. For obfs4, it achieved a TPR of 1.0 and an FPR of 2.9 × 10⁻³, an order of magnitude improvement. For obfs*, it yielded 100% recall and an FPR of 7 × 10⁻⁴. Snowflake data connections showed the highest performance (F1-score of 1.0 at λ=1k), likely due to longer flow characteristics. Snowflake broker connections performed least well (FPR = 0.18) as they are genuine TLS connections.

Despite these improvements, deep learning's FPRs remained prohibitively high for realistic base rates (precision near-zero at λ 10⁶).

Computational cost: Training was fast (seconds per epoch) on a server with 2 Intel Xeons and 4 NVIDIA Tesla V100 GPUs. Inference throughput for the best CNN was 3,500 flows/sec on one GPU, and 13,000 flows/sec with four GPUs. Monitoring an internet backbone link (150,000 flows/sec) would require ~50 GPUs, costing ~$250,000, which is deemed feasible for a nation-state censor.

Demo / Proof of Concept

▶ Watch: Defining the stateful, on-path censor adversary model (3:40)

The most compelling demonstration of this research is the host-based analysis methodology and its results, which serve as a proof of concept for precise censorship detection. The core idea is to bootstrap from a flow-based classifier and track the behavior of destination hosts (IP, Port pairs) over time.

Host-Based Classification Scheme:

The algorithm operates as follows:

1. For each host, it tracks the total number of flow classifications (m) and the number of positive (circumventing) classifications (p).
2. It calculates the fraction of positive classifications (p/m).
3. Based on the flow classifier's validated TPR and FPR, and a desired low error probability (α), a minimum number of observations (η) and a threshold (τ) are computed. The theoretical basis for this relies on Hoeffding's inequality, which bounds the probability that p/m deviates significantly from the flow classifier's FPR for benign hosts.
4. A host is classified as Obfuscated if m ≥ η AND p/m > τ.
5. A host is classified as Benign if m ≥ η AND p/m ≤ τ.
6. If m < η, the host classification is Rejected, meaning not enough information has been gathered yet. This "reject option" strategy is key to reducing false classifications.

The minimum observations (η) required scales logarithmically with α. For example, for a target error probability α = 1 × 10⁻⁶, η = 32 for the CNN detecting obfs4. This means that after just 32 flows, the host classifier can make a decision with very high confidence. The denominator (TPR - FPR)² in the η calculation highlights that classifiers maximizing Youden's J statistic (TPR - FPR) require the least state.

Implementation and Results:

The team implemented Algorithm 1 to detect obfs4 and obfs* on their test dataset, which contained 410,911 unique destination hosts not used during flow classifier training.

The results were striking:

• Dramatic Reduction in False Positives: The number of false positive host classifications dropped exponentially as the number of flows observed for a host increased. For obfs4, the total number of false positives decreased to less than 1 on average after observing no more than 30 flows. For obfs*, false positives reached 0 after 38 flows.
• Achieving Target Error Rates: The actual host false positive rates achieved were 2.4 × 10⁻⁶ for obfs4 and 1.5 × 10⁻⁶ for obfs*. These rates nearly matched the desired error rate α = 1 × 10⁻⁶, representing a two-order-of-magnitude improvement over the deep learning flow error rates.
• Perfect Recall for Known Bridges: All 15 of the self-hosted bridges used in the experiment were successfully identified, demonstrating perfect recall for actual circumventing hosts.
• Feasible State Requirements: Even for an extremely low error probability (α = 1 × 10⁻¹⁵), the better-performing classifiers required no more than 128 observations, implying minimal state (e.g., 2 log₂(η) bits) per host. While tracking all 2³² IPv4 addresses and 2¹⁶ TCP ports would theoretically require 500 TiB of storage, this is well within a censor's capabilities and can be further optimized using dynamic allocation and data structures like counting Bloom filters.

This host-based approach effectively shifts the burden from identifying rare individual flows in a sea of benign traffic to identifying a much smaller set of consistently misbehaving hosts. This aligns perfectly with a censor's objective of curating a list of IP addresses for wholesale blocking, rather than blocking individual connections. The demonstration clearly shows that such a technique is not only theoretically sound but also practically implementable and highly effective against bridge-based circumvention systems with static ingress points.

Defensive Implications

▶ Watch: Methodology for collecting realistic network traffic data (4:00)

The research provides critical insights for developers of Censorship Resistant Systems (CRSes), highlighting vulnerabilities and suggesting new directions for defense:

1. Avoid Static Ingress Points: CRSes relying on non-publicly advertised, static ingress points (like Tor bridges) and fixed circumvention protocols are highly vulnerable to host-based analysis. The ability to accumulate evidence over time against a persistent IP address makes detection highly precise. Future CRSes should move away from this model.

1. Embrace Ephemeral Bridges: To mitigate host-based analysis, circumvention systems should utilize multiple, ephemeral bridges that change frequently, similar to aspects of Snowflake's design. This strategy would distribute client connections across many short-lived hosts, preventing censors from accumulating sufficient m (number of observations) for any single host to trigger detection thresholds. Rapid switching mechanisms would further enhance resilience if a bridge is blocked.

1. Client-Side Obfuscation for Diversity: Client-side host-based analysis is considerably more challenging for censors due to the client's diverse background traffic and dynamic nature. While the talk focuses on detecting servers, future CRS designs could explore ways to make client-side circumvention traffic harder to attribute to a specific host's behavior.

1. Develop Polymorphic and Programmable Protocols: The ease with which obfs* evaded classical classifiers, and the general challenge of detecting diverse traffic, suggests that CRSes could benefit from polymorphic or programmable circumvention protocols (e.g., Marionette, Proteus). These protocols dynamically alter their traffic patterns, forcing adversaries to develop classifiers for entire families of protocols, a significantly more difficult task than targeting a single, fixed protocol. This approach could leverage adversarial examples to confuse machine learning classifiers, making them misclassify circumvention traffic as benign.

1. Focus on Elusive Transports: The field of obfuscated protocol development needs renewed focus, potentially drawing inspiration from advancements in website fingerprinting. Research into "elusive, hard-to-detect transports" is crucial to stay ahead of sophisticated detection methods.

1. Realism in CRS Evaluation: The findings underscore the paramount importance of evaluating CRSes under realistic conditions, including accurate base rates and diverse background traffic. Unrealistic assumptions lead to inflated security claims and a false sense of robustness.

1. Privacy-Preserving Data Sharing: The lack of realistic background network traffic datasets is a significant impediment to CRS research. The community should invest in developing privacy-preserving methodologies for collecting and sharing such datasets to accelerate the development and evaluation of stronger circumvention systems.

Key Takeaways

• Classical machine learning methods for censorship circumvention detection are ineffective at scale: They are overwhelmed by false positives (exceeding 94%) even at optimistic base rates and are easily evaded by minor protocol modifications.
• Deep learning improves flow-based detection, but still falls short: While CNNs offer an order of magnitude improvement in false positive rates (e.g., FPR of 2.9 × 10⁻³ for obfs4), their precision still vanishes at realistic internet-scale base rates (λ 10⁶), leading to unacceptable collateral damage.
• Host-based analysis is a game-changer for censors: By accumulating evidence from multiple flows for a specific (IP, Port) host over time, censors can dramatically reduce the effective base rate and achieve extremely high precision (FPRs of 10⁻⁶) with perfect recall for known bridges, requiring only a small number of observations (e.g., 30-40 flows).
• Static bridge designs are vulnerable: CRSes that rely on long-lived, non-publicly advertised static ingress points are highly susceptible to host-based detection.
• Future CRSes must be dynamic and ephemeral: Defenses against host-based analysis include using multiple ephemeral bridges, rapidly switching connections, and developing polymorphic or programmable protocols that dynamically change traffic patterns to confuse classifiers.
• Realism is paramount in CRS research: Evaluations of circumvention systems must incorporate realistic adversary models, base rates, and diverse background traffic to provide meaningful security assessments.

About the Speaker(s)

Ryan Wails is the researcher behind this detailed technical article on detecting censorship circumvention in real-world networks. His work, presented at the NDSS Symposium, highlights a deep understanding of network traffic analysis, machine learning applications in security, and the complex dynamics of internet censorship. While specific affiliations (Title, Company) are not provided in the metadata or transcript, the talk describes the collection of a private dataset from a "university-hosted network observation point" and mentions "our campus dataset collection," implying his involvement in academic research, likely at a university. His research demonstrates a commitment to advancing a free and open internet by rigorously evaluating the capabilities of sophisticated censors to enable the development of more resilient censorship-resistant systems.