Break the Wall from Bottom: Automated Discovery of Protocol-Level Evasion Vulnerabilities

Break the Wall from Bottom: Automated Discovery of Protocol-Level Evasion Vulnerabilities

Speakers: Chiwan (Aki)

Conference: Black Hat USA

YouTube: https://www.youtube.com/watch?v=RgBf7P2BkJM

Overview

In a critical presentation at Black Hat USA, a researcher known as Chiwan, or Aki, unveiled groundbreaking work on the automated discovery of protocol-level evasion vulnerabilities in Web Application Firewalls (WAFs). Titled "Break the Wall from Bottom," the talk delved into a persistent and often overlooked class of bypasses that undermine the very foundation of WAF protection. The research highlights how fundamental discrepancies in how WAFs and target web applications parse HTTP requests can create blind spots, allowing malicious payloads to bypass even the most stringent security rules.

WAFs are a cornerstone of modern web application security, designed to inspect HTTP traffic, detect malicious inputs like SQL injection or Cross-Site Scripting (XSS), and block them before they reach the backend application. They offer a flexible and faster alternative to waiting for vendor patches or reconfiguring applications. However, as the speaker emphasized, the deployment of a WAF does not guarantee impregnable security. The ongoing "cat and mouse" game between attackers and defenders means WAFs are constantly under threat of evasion, a challenge underscored by the persistent attempts to bypass WAFs during major incidents like the Log4j vulnerability.

This talk is particularly significant because it moves beyond traditional payload-level evasion tactics—where attackers simply mutate malicious strings to avoid detection—to focus on a more subtle and powerful vector: the protocol level. By exploiting differences in HTTP parsing, attackers can render WAFs completely "silent" without altering the core malicious payload, making these vulnerabilities far more resilient to typical rule updates. The research introduces an automated framework to systematically uncover these critical parsing mismatches, providing a vital tool for both WAF vendors and web application defenders to strengthen their security posture against sophisticated attacks.

Background

▶ Watch: Introduction to protocol-level WAF evasion vulnerabilities (0:00)

The ubiquitous nature of web applications, which frequently process user inputs embedded in HTTP request parameters, has created a fertile ground for cyberattacks. Malicious inputs, such as a SQL injection payload targeting a product ID parameter, can hijack application control flows, leading to data theft or arbitrary command execution. To counter these threats, Web Application Firewalls (WAFs) were developed as an essential layer of defense. WAFs operate by monitoring and filtering HTTP requests, acting as an intermediary between clients and web servers.

The working principle of a WAF can be broken down into three primary stages:

1. Parsing HTTP Messages: The WAF first parses incoming HTTP requests to extract various components, including headers, body content, and crucially, HTTP parameters.
2. Matching Parameters with Rules: Extracted parameters are then compared against a predefined set of security rules. These rules are designed to identify known attack patterns for vulnerabilities like SQL injection, XSS, or command injection.
3. Taking Action: If any part of the request matches a malicious rule, the WAF takes a configured action, such as blocking the request, alerting administrators, or logging the incident. This prevents the malicious request from ever reaching the backend web application.

Historically, attackers have focused on payload-level evasion tactics. This involves mutating the malicious payload itself to circumvent WAF rules. For instance, an attacker might change an '=' sign to '>' or alter the case of keywords (SELECT to select) to bypass a signature-based rule. Tools like Auto Spear and WAF-bypass-tool are designed to automatically generate such mutated payloads. While effective against naive rules, this approach has inherent limitations. It is often specific to a particular vulnerability type (e.g., SQL injection) and can be quickly mitigated by WAF administrators updating their rules with new signatures. The speaker highlighted how, in the wake of the Log4j vulnerability, the Cloudflare WAF team observed attackers continuously experimenting with various encoding schemes and Log4j-specific features to bypass WAFs once initial payloads were blocked.

However, the talk introduces a more insidious form of evasion: protocol-level evasion. This class of vulnerabilities arises not from the content of the payload, but from fundamental differences in how the WAF and the actual web application interpret the HTTP protocol itself. If a WAF fails to correctly parse an HTTP request in the same way the backend application does, it can effectively become "silent" to an attack. The WAF might not extract the parameters, or it might extract them incorrectly, thus never matching the malicious payload against its rules, while the backend application processes the malicious input as intended. Such tactics, while seemingly novel, have historical roots, with similar concepts having been discussed at Black Hat over a decade ago. The continued discovery of new cases underscores that this is a persistent and systemic problem, often rooted in the complexities and permissiveness of HTTP parsing across different implementations and frameworks.

Key Findings

▶ Watch: WAF principles and the continuous battle with attackers (2:00)

The central discovery presented in this research is the existence and automated discovery of protocol-level evasion vulnerabilities that exploit parsing discrepancies between Web Application Firewalls (WAFs) and target web applications. Unlike payload-level evasions, which focus on altering the malicious string itself, these vulnerabilities leverage how HTTP requests are interpreted at a foundational level.

The core finding can be summarized as follows: A WAF may fail to correctly parse an incoming HTTP request in a way that matches the parsing logic of the actual web application it is protecting. This mismatch creates a critical blind spot. When the WAF incorrectly parses or completely ignores a part of the request containing a malicious payload, it will not apply its security rules, allowing the attack to pass through undetected to the application, which does parse it correctly.

The speaker presented a compelling example involving the Content-Type header:

• An attacker crafted a request with a malicious SQL injection payload within the JSON body.
• Instead of a standard Content-Type: application/json, the attacker used an unconventional but valid variant, specifically Content-Type: X-whatever+JSON.
• The WAF, encountering this non-standard Content-Type, failed to recognize it as a JSON body. Consequently, the WAF did not attempt to parse the body content to extract parameters. Without extracted parameters, its security rules could not be applied, and the malicious payload remained undetected.
• Crucially, the web application framework (as illustrated by example code in the talk) was designed to accept any content type that starts with application/ and ends with +JSON as a valid JSON body. Therefore, it correctly parsed the X-whatever+JSON content, extracted the malicious SQL injection payload, and processed it, leading to a successful attack.

This finding demonstrates that even with "really mad" (i.e., extremely strict) payload-level rules that restrict parameters to numeric values, protocol-level evasion can render those rules irrelevant. The WAF is effectively "silenced" at the parsing stage, making the payload content irrelevant to its detection logic. This represents a significant contribution because it highlights a systemic weakness in WAF design and implementation—the inherent difficulty in perfectly synchronizing HTTP parsing behavior across diverse WAF products and the myriad web application frameworks they are meant to protect. The research also introduces an automated framework for discovering these subtle yet critical discrepancies, shifting the burden from manual, ad-hoc discovery to systematic, scalable analysis.

Technical Deep Dive

▶ Watch: Demonstrating payload-level evasion tactics against WAFs (3:00)

The technical underpinning of protocol-level WAF evasion lies in the subtle, yet critical, differences in HTTP parsing logic between WAFs and web application frameworks. While WAFs aim to emulate application behavior to detect threats, achieving perfect synchronization across all possible permutations of HTTP requests and application-specific parsing rules is an incredibly complex challenge.

A WAF's detection mechanism, as described, operates in three sequential stages:

1. HTTP Message Parsing: The WAF first dissects the raw HTTP request into its constituent parts: headers, method, URI, and body. During this stage, it identifies the Content-Type header, which dictates how the request body should be interpreted (e.g., application/x-www-form-urlencoded, multipart/form-data, application/json).
2. Parameter Extraction: Based on the parsed Content-Type and other factors, the WAF attempts to extract user-supplied parameters from various locations within the request, such as query strings, request body, or even custom headers. For JSON bodies, this involves parsing the JSON structure to identify key-value pairs.
3. Rule Matching and Action: Once parameters are extracted, they are matched against a database of security rules (signatures, heuristics, anomaly detection). If a match occurs, the WAF takes a predefined action, typically blocking the request.

The vulnerability arises when the web application's parsing logic diverges from the WAF's at stage one or two. The speaker's primary example, Content-Type: X-whatever+JSON, perfectly illustrates this. Modern web frameworks, often designed for flexibility and ease of development, may implement more permissive parsing rules for Content-Type headers. For instance, a framework might use a regular expression or a broad matching logic that accepts any Content-Type starting with application/ and ending with +JSON as a valid JSON payload. This allows developers to use custom or vendor-specific JSON media types (e.g., application/vnd.api+json) without explicit configuration.

The critical mismatch occurs because:

• WAF's Strictness (or Lack Thereof): A WAF, designed for security, might employ a more rigid or limited set of recognized Content-Type values. If X-whatever+JSON is not explicitly recognized as a JSON type, the WAF might default to not parsing the body for parameters, or it might attempt to parse it as plain text or application/x-www-form-urlencoded, failing to extract the JSON-encoded malicious data. The WAF becomes "silent" to the attack.
• Application's Permissiveness: The backend web application, using its framework's more flexible parsing, correctly identifies X-whatever+JSON as JSON. It then proceeds to parse the body, extracts the malicious parameter, and processes it, leading to a successful attack.

This discrepancy effectively bypasses the WAF's security rules entirely, regardless of how robust they are. The attack doesn't rely on obfuscating the payload (which is payload-level evasion), but on making the WAF ignore the payload due to a protocol interpretation difference. This is why protocol-level evasion is considered more potent; it can bypass WAFs even when they have "strict rules at the payload level" and can be "utilized to load arbitrary attack vectors," not just specific vulnerability types.

The research also touched upon the methodology for discovering these vulnerabilities, emphasizing an automated approach. The tool "instruments the code of HTTP parser" (likely referring to the WAF's parser or a simulated environment) to observe its behavior under various malformed or unconventional HTTP requests. By systematically generating and testing a vast array of HTTP request permutations and comparing how a WAF processes them against how a target application framework would, the system can pinpoint these critical parsing discrepancies. This fuzzing-like approach, coupled with an understanding of common web framework parsing behaviors, allows for the systematic discovery of new evasion vectors that have historically been found through manual, ad-hoc efforts.

Demo / Proof of Concept

▶ Watch: Introducing protocol-level evasion via Content-Type manipulation (4:00)

The speaker provided a concrete demonstration of their framework's capability to discover protocol-level evasion cases, showcasing a vulnerability in a popular open-source WAF.

The setup for the demonstration involved:

1. Target WAF Configuration: The chosen WAF for the demo was SafeLine, described as "one of the most popular open source WAF in the GitHub." The speaker explicitly stated that the issue demonstrated had already been fixed in the latest version of SafeLine at the time of the talk, indicating responsible disclosure. For the demo, SafeLine was configured in its "most strict mode" with "all of the rules" enabled. This ensured that the WAF was operating at its highest possible security posture, making any successful bypass particularly impactful.
2. Initial WAF Behavior: Before initiating the fuzzing process, the speaker verified SafeLine's expected behavior. It correctly allowed benign HTTP requests to pass through while effectively blocking standard SQL injection payloads, confirming its operational status and rule enforcement.
3. Automated Discovery Framework: The core of the demonstration was the automated tool designed to "fast the evasion cases" (likely meaning 'fuzz' or 'find' evasion cases).

• Seed Generation: The process began by generating "initial seeds," which are baseline HTTP requests that the fuzzer would then mutate.
• Instrumentation: A key technical detail mentioned was that the tool "will first instrument the code of HTTP parser." This implies that the framework either hooks into or analyzes the WAF's internal HTTP parsing functions. By observing how the WAF processes various malformed or non-standard requests, the tool can identify when the WAF's parsing logic deviates from a known, standard application's parsing. This instrumentation allows for a more targeted and efficient fuzzing process, focusing on the parsing pipeline rather than just arbitrary input mutations.
• Fuzzing and Evasion Detection: The framework then systematically mutated the initial seeds, generating a wide range of unconventional HTTP requests. These requests were sent through the configured SafeLine WAF. The goal was to find a request that would bypass SafeLine (i.e., not be blocked) but would be correctly interpreted and processed by a simulated or actual backend web application, leading to a successful attack (e.g., SQL injection).

While the specifics of the discovered SafeLine vulnerability were not detailed beyond the general Content-Type manipulation discussed earlier, the demonstration successfully illustrated the automated framework's ability to identify scenarios where the WAF's parsing logic failed to align with that of a typical web application. This practical proof of concept reinforced the theoretical arguments about the existence and persistence of protocol-level evasion vulnerabilities, even in highly secured WAF configurations. The fact that the issue was already patched in SafeLine underscores the ongoing nature of this problem and the value of such research in improving WAF resilience.

Defensive Implications

▶ Watch: Summarizing payload vs. protocol-level WAF evasion approaches (4:50)

The findings presented in "Break the Wall from Bottom" carry significant implications for defenders responsible for securing web applications, necessitating a shift in mindset beyond merely updating WAF signatures.

1. WAF Parsing Alignment is Critical: The most crucial takeaway is that WAFs must parse HTTP requests identically to the web applications they protect. Any discrepancy, however minor, creates an exploitable blind spot. Defenders should:

• Regularly Audit WAFs: Conduct thorough testing to ensure WAFs correctly interpret all HTTP protocol variations, especially Content-Type headers and other parsing-sensitive elements, in the same way the backend application (and its framework) does. This goes beyond simple payload-level testing.
• Understand Application Frameworks: Gain a deep understanding of how the specific web application frameworks (e.g., Spring, Node.js Express, Django, Ruby on Rails) parse incoming HTTP requests, particularly regarding ambiguous or non-standard headers and body formats. This knowledge is vital for configuring WAFs appropriately.

1. Beyond Payload-Level Rules: Relying solely on signature-based rules for malicious payloads is insufficient. Attackers can bypass these rules entirely by exploiting parsing differences, making the WAF "silent." Defenders should:

• Embrace Behavioral and Anomaly Detection: While challenging, WAFs should incorporate more advanced behavioral analysis that can detect anomalous request structures or parsing outcomes, rather than just matching known malicious strings.
• Focus on Input Validation at the Application Layer: Robust, comprehensive input validation within the application itself remains the last line of defense. Even if a WAF is bypassed, strong application-level validation can mitigate the impact of malicious inputs.

1. Continuous Testing and Fuzzing: The automated discovery framework presented in the talk highlights the need for continuous, automated testing of WAFs against protocol-level evasion.

• Integrate Evasion Testing into CI/CD: Incorporate tools and methodologies similar to the one demonstrated into continuous integration/continuous deployment (CI/CD) pipelines to proactively identify parsing discrepancies as applications evolve or WAF configurations change.
• Collaborate with WAF Vendors: WAF users should actively engage with their vendors to share insights from such research and push for improved parsing robustness and alignment with common application frameworks.

1. Developer Education: Developers need to be aware of the nuances of HTTP parsing and the potential for discrepancies.

• Standardize Content-Type Usage: Encourage the use of standard and well-defined Content-Type headers in API designs to minimize ambiguity.
• Beware of Overly Permissive Parsing: Understand the security implications of frameworks that are overly permissive in their interpretation of HTTP headers and body formats.

In essence, defending against protocol-level evasion requires a multi-layered approach that combines meticulous WAF configuration and auditing, robust application-level security controls, continuous automated testing, and a deep understanding of how both WAFs and applications interpret the HTTP protocol. The battle against WAF bypasses is not just about detecting malicious content, but about ensuring the "wall" itself is built on a consistent and unexploitable foundation.

Key Takeaways

• Protocol-level WAF evasion is a persistent and critical threat: Attackers can bypass WAFs by exploiting subtle differences in how WAFs and web applications parse HTTP requests, even with strict payload-level rules.
• Parsing discrepancies are the root cause: Vulnerabilities arise when a WAF fails to interpret HTTP requests (e.g., Content-Type headers) in the exact same way the backend application framework does, rendering the WAF "silent."
• Payload-level security is insufficient alone: While important, signature-based detection of malicious payloads can be completely circumvented if the WAF's parsing logic is flawed, allowing untouched malicious data to reach the application.
• Automated discovery is crucial for proactive defense: Tools and frameworks that can systematically fuzz and compare HTTP parsing behaviors between WAFs and applications are essential for identifying these complex vulnerabilities at scale.
• The Content-Type: X-whatever+JSON example is a classic illustration: This specific evasion tactic highlights how an unconventional but valid Content-Type can confuse WAFs while still being processed correctly by permissive application frameworks.
• Defenders must ensure WAFs align with application parsing: Comprehensive testing, deep understanding of application framework behavior, and continuous auditing are necessary to ensure WAFs provide effective protection against these foundational bypasses.

About the Speaker(s)

The talk was delivered by Chiwan, who also goes by Aki. Beyond his name, the transcript and metadata do not provide further details about his title or company affiliations. He presented research findings on the automated discovery of protocol-level evasion vulnerabilities in web application firewalls.