Wallbleed: A Memory Disclosure Vulnerability in the Great Firewall of China

Wallbleed: A Memory Disclosure Vulnerability in the Great Firewall of China

Speakers: Shencha Fan, University of Colorado Boulder

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=_emoqfRXx2s

Overview

This article delves into "Wallbleed," a significant memory disclosure vulnerability discovered in the Great Firewall of China (GFW)'s DNS injection system. Presented at the NDSS Symposium by Shencha Fan (who introduced himself as Jackson), this research uncovers a critical flaw that allowed researchers to leak sensitive network traffic and internal system information from the GFW's infrastructure. The talk highlights how a seemingly minor manipulation of a DNS query's length field could lead to the exposure of hundreds of gigabytes of data, impacting not only users within China but also individuals connecting through its network perimeter globally.

The discovery of Wallbleed is paramount for several reasons. Firstly, it provides unprecedented insight into the internal workings and architecture of one of the world's most sophisticated state-sponsored censorship systems. By reverse engineering aspects of the GFW's code and observing its patching behavior, the researchers gained valuable knowledge about its operational resilience and deployment strategies. Secondly, the vulnerability underscores the severe privacy risks associated with state-level internet censorship, demonstrating how systems designed for control can inadvertently expose the very data they aim to monitor or block.

Ultimately, Wallbleed serves as a stark reminder of the potential for even highly resourced adversaries to harbor critical security flaws. The ethical considerations surrounding the exploitation and disclosure of such a vulnerability in a system like the GFW were also a central theme, emphasizing the complex dilemmas faced by security researchers when dealing with systems that are both a source of harm and a target for investigation. The talk meticulously details the technical mechanics of Wallbleed, its broad impact, and the fascinating timeline of its discovery and subsequent patching.

Background

▶ Watch: Introduction to DNS injection and the Great Firewall (0:00)

To understand Wallbleed, it's essential to first grasp the fundamentals of DNS (Domain Name System) and the specific censorship technique known as DNS injection, which forms a core component of the Great Firewall of China (GFW). When a client wants to access a website, it first sends a DNS query to a DNS resolver to obtain the corresponding IP address. The resolver then responds with this address, allowing the client to connect directly to the host.

The GFW employs DNS injection as a primary method of censorship. A sensor (the GFW's DNS injector) sits on the network path between the client and the resolver. When this sensor detects a query for a "disapproved" domain (e.g., Facebook), it immediately injects a forged DNS response containing an incorrect or unrelated IP address. Because the sensor is typically closer to the client than the legitimate resolver, its injected packet often arrives first. The client, receiving this malicious response, attempts to connect to the wrong IP address and fails, effectively blocking access to the intended website. The legitimate response from the resolver, arriving later, is usually discarded.

The concept of Wallbleed draws a direct parallel to the infamous Heartbleed bug, a memory disclosure vulnerability in the OpenSSL cryptographic library. The speaker used a modified XKCD cartoon to illustrate this: in Heartbleed, a server trusts a client-provided length field for data. If the client sends a small amount of data but claims a much larger length, a vulnerable server will respond with the original data followed by additional bytes read directly from its memory, leading to a memory disclosure. This allows an attacker to potentially retrieve sensitive information such as private keys, user credentials, or other critical data residing in the server's memory.

In the context of DNS, a standard query for a domain name (the Qname) follows a specific structure. For example, "facebook.com" would be encoded as 08 (length of "facebook") followed by facebook, then 03 (length of "com") followed by com, and finally a null terminator (00) to mark the end of the Qname. This precise structure, particularly the length fields, became the Achilles' heel of the GFW's DNS injector.

Key Findings

▶ Watch: Wallbleed discovery: triggering the memory leak (3:45)

The discovery of Wallbleed originated from a simple yet ingenious manipulation of the standard DNS query structure. Researchers found that by altering the final length field in the Qname from its legitimate value (e.g., 03 for "com") to an illicitly large value, specifically FF (255 in decimal), and sending this modified query across the GFW, the DNS injector would respond with a standard DNS response plus over 100 bytes of leaked memory. This confirmed a classic memory overread vulnerability, akin to Heartbleed, but within a critical state-level censorship system.

To thoroughly investigate what was being leaked, the researchers conducted an extensive study, sending millions of Wallbleed queries and collecting hundreds of gigabytes of leaked memory. Through rigorous analysis using regular expressions, they classified the disclosed data, revealing several critical insights:

1. Live Network Traffic: A significant portion of the leaked data comprised live network traffic. Notably, they observed UPnP (Universal Plug and Play) protocol traffic, which operates over HTTP (port 80) and TCP. This was a crucial finding because DNS typically uses UDP on port 53. The presence of TCP/HTTP traffic indicated that the GFW's injector was not performing any pre-filtering and was observing all network traffic passing through its network tap or interface, regardless of protocol or port. This demonstrated a far broader surveillance capability than previously understood.

1. Internal Communications and Architecture: The leaked data also contained HTTP date headers, which allowed researchers to infer the cache duration of traffic on the GFW's systems – approximately 1 second. Beyond this, they identified Linux stack frames, providing strong evidence that the GFW's injectors run on an x86 architecture and utilize the Linux operating system. Furthermore, IP headers revealed private source IPs connecting to public destination IPs, indicating that the researchers were not just observing external traffic but also internal communications between different GFW injectors, offering a glimpse into their distributed architecture.

1. User Privacy Risk: Most alarmingly, the leaked memory included highly sensitive user data such as HTTP cookies and password fields. This unequivocally demonstrated the severe risk Wallbleed posed to user privacy, as it could expose credentials and session information of individuals whose traffic traversed the GFW.

1. Proof of Live Leakage: To definitively prove that the leaked data was live network traffic and not static memory artifacts, the team set up a controlled experiment. Two hosts were connected across the GFW: one sent a unique "magic sequence" (a custom, identifiable string), and the other simultaneously sent Wallbleed queries. When Wallbleed responses were received, they contained the unique magic sequence, confirming that the GFW was indeed leaking live data that had just passed through its systems.

1. Reverse Engineering and Code Analysis: A key contribution of the research was the successful reverse engineering of the C equivalent code responsible for the DNS injector's functionality. This allowed the researchers to pinpoint the exact memory overread vulnerability within the code, providing a concrete understanding of how the flaw occurred. While the full code snippet isn't provided in the talk, the ability to reconstruct it highlights a deep technical understanding of the GFW's internal logic.

1. Global Impact: The vulnerability was not confined to users within mainland China. By sending Wallbleed requests from a client in the US to the entire IPv4 space, researchers identified hosts globally that triggered an injection, even without a direct relation to China. This demonstrated that users around the world, whose internet traffic might be routed through or intercepted by GFW infrastructure, were also susceptible to data leakage.

1. Patching Behavior and Resilience: The researchers established an experiment to monitor the GFW's patching behavior, sending Wallbleed queries every 15 minutes to selected Chinese IPs. This unique approach revealed a multi-stage patching process, initially involving testing (a drop in monitored IPs), followed by a phased rollout across approximately five stages, which further analysis indicated was based on geographical region.

• Wallbleed V1 Discovery: October 2021
• Wallbleed V1 Patch: November 2023 (after over two years of monitoring)
• Wallbleed V2 Discovery: A few months after V1 was patched, researchers discovered a new variant (V2) with slight query modifications, allowing data leakage to resume.
• Wallbleed V2 Patch: April 2024. As of the talk, no further vulnerabilities had been found.

This detailed timeline and observation of patching behavior provided invaluable insights into the GFW's operational response capabilities and its ability to adapt to discovered vulnerabilities.

Technical Deep Dive

▶ Watch: Analysis of leaked data: network traffic, cookies, passwords (4:40)

The core of the Wallbleed vulnerability lies in a subtle yet critical manipulation of the DNS Qname structure, combined with an underlying memory handling flaw in the GFW's DNS injector. A standard DNS query's Qname field encodes the domain name by segmenting it into "labels," each prefixed by a single byte indicating its length. For instance, facebook.com is represented as 08facebook03com00. Here, 08 signifies "facebook" is 8 bytes long, 03 for "com" is 3 bytes, and 00 acts as a null terminator, signaling the end of the domain name.

The researchers discovered that if they modified the last length field in this sequence from its legitimate value (e.g., 03) to FF (hexadecimal for 255), the GFW's injector failed to properly validate this length. Instead of rejecting the malformed query or truncating the read, it would attempt to read 255 bytes from memory starting from the position where the domain name label should have been. Since the actual label was much shorter (e.g., "com" is only 3 bytes), the injector would overread beyond the intended buffer, effectively scooping up adjacent memory contents and including them in its injected DNS response. This behavior is a direct analogue to the Heartbleed vulnerability, where a lack of bounds checking on a client-provided length led to arbitrary memory disclosure.

The implications of the leaked data were technically profound. The discovery of UPnP network traffic (a protocol typically used for device discovery on local networks) alongside HTTP headers was particularly telling. These protocols operate over TCP and often on port 80 (for HTTP) or other high ports, whereas DNS fundamentally uses UDP on port 53. The fact that the GFW's injector was leaking non-DNS, non-UDP, and non-port-53 traffic strongly suggested that the injector was deployed at a very low level in the network stack. It implies that the GFW infrastructure was not merely inspecting DNS packets but was likely operating as a full packet inspection (FPI) or deep packet inspection (DPI) engine, tapping into a broad spectrum of network traffic before any protocol-specific filtering or parsing occurred. This broad visibility is what allowed the memory overread to disclose data from unrelated network streams.

Further technical analysis of the leaked memory fragments provided architectural clues. The presence of Linux stack frames was a clear indicator that the GFW's injection systems run on the Linux operating system. Additionally, the structure of these stack frames, combined with other memory artifacts, allowed the researchers to infer that the underlying hardware architecture was x86. This level of detail about the GFW's software and hardware stack is highly unusual for an opaque, state-controlled system and represents a significant intelligence gain.

The reverse engineering of the C equivalent code provided the definitive technical explanation for the vulnerability. While the specific code snippet was not presented in the transcript, the speaker confirmed that the full code could be found in the paper's appendix. This process allowed the researchers to identify the precise location where the length validation was missing or incorrectly handled, leading to the buffer overread when processing the manipulated Qname. The ability to reconstruct this code demonstrates a deep understanding of the GFW's internal logic and programming practices.

The GFW's patching behavior also offered a technical glimpse into its operational strategies. The observation of a phased rollout, initially with a testing phase and then regional deployment, suggests a sophisticated and organized approach to system maintenance and vulnerability remediation, rather than an ad-hoc fix. This indicates a well-structured engineering team behind the GFW, capable of coordinating updates across a distributed infrastructure. The rapid re-discovery of Wallbleed V2 after V1 was patched highlights the iterative nature of vulnerability research and the continuous cat-and-mouse game between researchers and system operators. Wallbleed V2 likely required only "slight modifications to the initial query," suggesting that the initial patch might have been narrowly focused, leaving other related code paths vulnerable or introducing a new edge case.

Demo / Proof of Concept

▶ Watch: Reverse engineering the GFW injector's C code (7:20)

The talk included a robust proof of concept (PoC) demonstration designed to unequivocally prove that the Wallbleed vulnerability was leaking live network traffic, rather than static or cached memory contents. This was crucial for establishing the real-time privacy implications of the flaw.

The PoC involved setting up two distinct hosts, both configured to connect across the Great Firewall (GFW):

1. Host 1 (Magic Sequence Sender): This host was configured to continuously send a unique, identifiable sequence of data. The researchers referred to this as a "magic sequence" in their paper. The sequence was designed to be easily recognizable if encountered in any leaked memory. This host essentially generated "bait" traffic that would traverse the GFW's network.

1. Host 2 (Wallbleed Query Sender): Simultaneously, this host was responsible for sending the specially crafted Wallbleed DNS queries. These queries, as detailed in the "Technical Deep Dive" section, contained the manipulated FF length field in the Qname, designed to trigger the memory overread vulnerability in the GFW's DNS injector.

The core of the demonstration was the observation that when a Wallbleed response was received from Host 2, it frequently contained the unique magic sequence that had just been sent by Host 1. This direct correlation served as irrefutable evidence. The time delay between sending the magic sequence and seeing it leaked in a Wallbleed response was remarkably short, often less than one second. This quick turnaround further solidified the conclusion that the GFW was disclosing data from actively processed network streams.

The speaker clarified that while they did not see all of the magic sequences they sent, and often had to send a high volume of both magic sequences and Wallbleed queries, the successful capture of even a few instances was sufficient to validate the live leakage hypothesis. This experiment provided concrete, empirical evidence to support the claims of real-time memory disclosure, moving the findings from theoretical possibility to demonstrated reality.

Defensive Implications

▶ Watch: Global impact: Wallbleed affects users worldwide (8:00)

The discovery of Wallbleed carries significant defensive implications, particularly for individuals, organizations, and service providers whose traffic might traverse or be influenced by the Great Firewall of China. While it's not a vulnerability that can be directly patched by typical users, understanding its existence and nature is crucial for mitigating risks.

1. Heightened Awareness of GFW Risks: Wallbleed underscores that the GFW, beyond its primary function of censorship, also poses a substantial privacy risk. Users and organizations sending traffic through or originating from regions where GFW infrastructure is present should be aware that their data, including potentially sensitive information like HTTP cookies and password fields, could be inadvertently exposed due to internal system flaws. This reinforces the need for a "zero-trust" mentality when dealing with state-controlled network infrastructure.

1. Encryption is Paramount: The fact that Wallbleed leaked raw network traffic, including HTTP headers and password fields, highlights the critical importance of end-to-end encryption for all communications. While encryption (like HTTPS) would not prevent the memory disclosure itself, it would render the content of the leaked data unintelligible to an adversary. Any unencrypted traffic (e.g., plain HTTP, unencrypted email protocols) is inherently vulnerable to exposure through such flaws, regardless of the GFW's intent. Defenders should prioritize enabling HTTPS for all web services and using encrypted protocols for all other data transfers wherever possible.

1. Anonymization and Circumvention Tool Security: Users relying on VPNs, Tor, or other circumvention tools to bypass the GFW should be acutely aware that these tools become even more critical. However, the security of these tools themselves is paramount. If a circumvention tool's traffic somehow becomes unencrypted or is compromised before it reaches its secure tunnel, it could still be subject to leakage. Users should ensure they are using reputable, well-configured, and updated circumvention technologies.

1. Internal Network Security Best Practices: The observation of internal IP headers and the inference of x86 Linux architecture suggest that even state-sponsored systems have common attack surfaces. While this specific vulnerability targets the GFW, it serves as a reminder for all network operators to adhere to robust security best practices:

• Input Validation: Thoroughly validate all incoming data, especially length fields, to prevent buffer overflows and overreads.
• Secure Coding Practices: Implement memory-safe coding practices to avoid vulnerabilities like Heartbleed and Wallbleed.
• Regular Audits: Conduct regular security audits and penetration testing of critical network infrastructure.
• Patch Management: Implement a robust and timely patch management process, as demonstrated by the GFW's multi-stage rollout.

1. Ethical Considerations for Researchers: The talk prominently featured a discussion on the ethics of exploiting systems like the GFW. Researchers contemplating similar work should consider:

• Source of Harm: Is the target system itself a source of harm? If so, does understanding its flaws contribute to reducing harm?
• Impact of Disclosure: Will disclosing the vulnerability help the system's operators cause further harm, or will it simply reduce existing harms (e.g., by forcing a patch that makes the system less leaky)?
• Responsible Disclosure: Even in politically charged contexts, the intent for responsible disclosure, as demonstrated by the Wallbleed researchers, is important. Planning for notification, even if circumstances prevent it, reflects ethical conduct.

In summary, for individuals and organizations, the primary defensive strategy against vulnerabilities like Wallbleed is to maximize encryption, minimize the transmission of sensitive unencrypted data, and be aware of the inherent risks when operating within or connecting through controlled network environments. For system operators, it's a call to meticulous input validation and secure memory handling in all network-facing applications.

Key Takeaways

• Wallbleed is a memory disclosure vulnerability in the Great Firewall of China (GFW)'s DNS injection system, allowing the leakage of live network traffic and internal system information.
• The vulnerability is triggered by manipulating the final length field of a DNS query's Qname to FF, causing an overread of adjacent memory due to insufficient input validation, similar to the Heartbleed bug.
• Researchers collected hundreds of gigabytes of leaked data, revealing live HTTP and UPnP traffic (TCP, port 80), indicating the GFW's deep packet inspection capabilities across various protocols, not just DNS.
• Leaked data included sensitive user information like HTTP cookies and password fields, posing a significant privacy risk to individuals whose traffic traverses the GFW.
• Analysis of leaked data provided insights into the GFW's architecture, including Linux stack frames (suggesting an x86 architecture) and internal IP headers (revealing communications between GFW injectors).
• The GFW exhibited a multi-stage, regionally-based patching behavior for Wallbleed V1 (patched November 2023) and Wallbleed V2 (patched April 2024), demonstrating an organized operational response.
• The research highlighted complex ethical considerations regarding exploiting systems that are a source of harm, emphasizing the goal of reducing existing harms and the intent for responsible disclosure.

About the Speaker(s)

The talk "Wallbleed: A Memory Disclosure Vulnerability in the Great Firewall of China" was presented by Shencha Fan. While introducing himself as "Jackson" at the beginning of the presentation, the official metadata identifies Shencha Fan as the speaker. He is affiliated with the University of Colorado Boulder, and the research presented was joint work with GFW Report, the University of Massachusetts Amherst, and the Shinomi lab. His work focuses on understanding and analyzing the Great Firewall, contributing significant insights into its operational mechanisms, vulnerabilities, and broader implications for internet freedom and user privacy.