Automatic Policy Synthesis and Enforcement for Protecting Untrusted Deserialization

Automatic Policy Synthesis and Enforcement for Protecting Untrusted Deserialization

Speakers: Quan Zhang

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=UNQXy0bPWVU

Overview

Java deserialization vulnerabilities represent a persistent and critical security threat to modern applications. Attackers exploit these flaws by injecting meticulously crafted malicious objects, leveraging existing methods within an application's classpath to construct "gadget chains" that can lead to severe consequences, including Remote Code Execution (RCE), Denial of Service (DoS), and Server-Side Request Forgery (SSRF). This is not a theoretical concern; over the past five years, approximately 800 vulnerabilities, categorized under CWE 502 (Deserialization of Untrusted Data), have been reported in the Common Vulnerabilities and Exposures (CVE) database, underscoring the widespread impact and gravity of the problem. Existing mitigation strategies, such as blocklist or manually-crafted allowlist policies, have proven insufficient due to their reactive nature, the continuous discovery of new bypasses, and the immense manual effort and expertise required for their accurate formulation.

This talk introduces DESERIGUARD, an innovative framework designed to automatically synthesize and enforce precise deserialization policies. Developed by Quan Zhang, DESERIGUARD addresses the fundamental challenges of manual policy creation by leveraging semantic-aware dataflow analysis to deduce the legitimate types of objects an application expects to deserialize. It constructs a Semantic-Aware Property Tree (SAPT) to capture the hierarchical structure of deserialized objects, from which it generates a stringent allowlist policy. This policy is then seamlessly enforced at runtime using a Java agent, providing robust protection across various deserialization libraries without requiring source code modifications.

The significance of DESERIGUARD lies in its proactive and automated approach to a problem that has long plagued the security community. By eliminating the reliance on human expertise and manual maintenance, DESERIGUARD offers a scalable and effective defense mechanism. Its ability to generate policies that are both highly precise and comprehensive, while incurring negligible performance overhead, marks a significant advancement in securing applications against the evolving landscape of deserialization attacks. The extensive evaluation presented in the talk demonstrates DESERIGUARD's capability to defend against real-world vulnerabilities, outperform state-of-the-art policy learning methods, and provide significantly stricter policies than those typically designed by developers, ultimately making it far more challenging for attackers to exploit these pervasive flaws.

Background

▶ Watch: Introduction to Java deserialization vulnerabilities and impact (0:00)

Serialization and deserialization are ubiquitous mechanisms in software development, enabling the transformation of complex objects into byte streams for purposes like network transmission, inter-process communication, and persistent storage. Popular libraries such as XStream for XML and FastJson for JSON facilitate these operations across diverse applications. However, the convenience offered by these mechanisms comes with a profound security risk when untrusted data is deserialized. CWE 502, "Deserialization of Untrusted Data," encapsulates this danger, with its criticality evidenced by the roughly 800 related CVEs identified in the last five years.

The root cause of deserialization vulnerabilities stems from applications attempting to reconstruct objects from untrusted input without adequate validation. Attackers exploit this by crafting a malicious byte stream representing a carefully structured, nested object. This object leverages classes already present in the victim application's classpath. During deserialization, the application is compelled to recursively process these properties in an attacker-controlled order. This process inadvertently triggers a sequence of method invocations—known as a gadget chain—that ultimately hijacks the application's execution flow. A classic example involves a PriorityQueue object whose comparator property is set to a TransformingComparator. Within this, an array of Transformer objects, composed of ConstantTransformer and InvokeTransformer, can be constructed to instantiate a Runtime object and invoke its exec method, leading to RCE even before the entire deserialization process completes. Crucially, every method in such a gadget chain must belong to a class available in the victim application's classpath.

In response to these threats, the security community has focused on implementing policies to restrict the types of objects that can be deserialized. Java Enhancement Proposal 290 (JEP 290) introduced a native feature for specifying deserialization policies, and many deserialization libraries, including XStream and FastJson, provide interfaces for defining either blocklist or allowlist policies.

• Blocklists aim to forbid known exploitable classes, such as those from CommonsCollections or CommonsBeanutils. While effective against known threats, blocklists are inherently reactive. Attackers constantly discover new gadget chains and bypasses, rendering existing blocklists obsolete. A notable instance is CVE-2017-1000353, where attackers bypassed Jenkins's blocklist to achieve RCE.
• Allowlists, which permit only a strictly defined set of necessary classes, are theoretically the most effective defense. However, their practical implementation is fraught with challenges. Developers often lack the deep security expertise and comprehensive understanding of an application's full deserialization requirements. This leads to:
• Manual Effort and Error-Proneness: Formulating a precise allowlist requires extensive manual analysis, debugging, and maintenance, which is both time-consuming and prone to human error.
• Overly Permissive Policies: To avoid disrupting normal application functionality, developers frequently resort to loose policies. A striking example is the Ofbiz application, which initially had no deserialization defense (leading to CVE-2019-0189). A subsequent allowlist policy broadly permitted java.. and org.apache.ofbiz.. patterns. This overly broad allowance enabled bypasses via java.rmi.server.RemoteObject, resulting in CVE-2021-26295. Even after adding a blocklist, it was bypassed by RMIConnectionImpl_Stub (CVE-2021-29200). Further, regex matching flaws in the allowlist were exploited (CVE-2021-30128). This iterative patching cycle vividly illustrates the difficulty and inadequacy of manual policy formulation.
• Insufficient Learning Data: Policy learning approaches, such as Trusted Execution Path (Trusted), attempt to learn policies from benign workloads. However, these methods often suffer from insufficient or incomplete learning data, leading to high false alarms or an inability to cover all legitimate deserialization paths.

DESERIGUARD distinguishes itself from existing related work. While gadget mining tools like GadgetInspector, SerHybrid, Tabby, GCMiner, and ODDFuzz focus on discovering gadget chains, DESERIGUARD is a defense tool designed to block them. Similarly, existing deserialization defense mechanisms (e.g., JEP 290's look-ahead defense, RASP frameworks) primarily focus on enforcement and low overhead, but they do not address the critical challenge of automatically synthesizing precise policies. DESERIGUARD aims to overcome these limitations by providing an automated, semantic-aware policy synthesis and enforcement framework that minimizes false alarms and offers robust, proactive protection.

Key Findings

▶ Watch: DESERIGUARD: Automatic policy synthesis and enforcement (2:00)

DESERIGUARD's evaluation against a comprehensive set of real-world scenarios and existing tools yielded several critical findings that underscore its effectiveness and superiority:

• 100% Defense Against Real-World Vulnerabilities: DESERIGUARD successfully resisted deserialization attacks on all 12 real-world vulnerabilities selected for their high impact and recent CVEs. These applications, averaging 1.11 million lines of code and 20.68 thousand classes, represent significant complexity, yet DESERIGUARD provided complete protection against diverse gadget chains including CommonsCollections, RMI, and Groovy.
• No False Alarms: Crucially, DESERIGUARD incurred no false alarms during extensive unit and integration tests across all 12 applications. Further validation on Jenkins and Ofbiz, monitoring 301,452 and 465,167 deserialization instances respectively, also revealed no false alarms, demonstrating its conservative yet accurate policy synthesis.
• Superior Policy Strictness: Policies automatically synthesized by DESERIGUARD are significantly stricter and more precise than 109 developer-designed policies from 40 popular GitHub projects. DESERIGUARD achieved a median compression rate of 0.04% (permitting a tiny fraction of total classes) compared to the developers' median of 19.02%. On average, DESERIGUARD permits 90 times fewer classes, restricting 99.12% more classes than manual policies. This drastically reduces the attack surface.
• Negligible Performance Overhead: DESERIGUARD operates with minimal performance impact. Static analysis, a one-time preprocessing step, averaged 36.1 seconds. The Java agent initialization averaged 46.08 milliseconds. Runtime auditing averaged a mere 0.039 milliseconds per deserialization operation. The overall application slowdown averaged a negligible 2.168%, with some applications experiencing less than 1%.
• Robust Against Evolving Gadget Chains: DESERIGUARD successfully blocked all potential gadget chains discovered by GadgetInspector (ranging from 2 to 20 per application) and all 33 Ysoserial gadget chains. This demonstrates its ability to defend against both known and rapidly evolving, previously unknown gadget chains.
• Outperformance of State-of-the-Art Policy Learning: When compared to the state-of-the-art policy learning tool, Trusted Execution Path (Trusted), DESERIGUARD exhibited superior reliability. While both mitigated all 12 vulnerabilities, Trusted incurred false alarms on 8 vulnerabilities due to inadequate benign deserialization workloads. DESERIGUARD, in contrast, produced no false alarms, highlighting the advantages of its semantic-aware synthesis over learning-based approaches.
• Blocklist Inadequacy Confirmed: A straw-man experiment confirmed the inherent weakness of traditional blocklist policies. A blocklist effective against pre-2021 Ysoserial gadget chains was bypassed by the AspectJWeaver gadget chain (discovered in 2021) on applications like Shiro, Apereo CAS, and Tomcat, while DESERIGUARD successfully resisted it on all 12 applications. This reinforces the necessity of proactive allowlist strategies.

Technical Deep Dive

▶ Watch: Three key contributions of DESERIGUARD framework (2:45)

DESERIGUARD is architected around two primary modules: the Policy Synthesis module and the Policy Enforcement module. Together, they provide a comprehensive solution for automatically generating and enforcing stringent deserialization allowlist policies.

Threat Model

DESERIGUARD operates under a specific threat model. It assumes that attackers can access a deserialization entry point within the target application and manipulate its input. Attackers are also presumed to have knowledge of all classes and their member methods within the victim application's classpath, enabling them to construct sophisticated malicious gadget chains. DESERIGUARD requires access to the application's source code for static analysis and must be launched with a Java agent. It assumes an uncompromised machine and a trusted Java Virtual Machine. Attacks originating from other vectors, such as OS kernel vulnerabilities or JVM memory errors, fall outside its scope. A crucial limitation is that if attackers discover exploitable gadgets within the necessary classes that an application must deserialize for legitimate functionality, DESERIGUARD cannot prevent that specific attack, as its core goal is to restrict unnecessary classes.

Policy Synthesis Module

The Policy Synthesis module is responsible for designing a precise allowlist policy for each deserialization entry point, encompassing all legitimate object types that could be deserialized. This process is driven by the construction of a Semantic-Aware Property Tree (SAPT) using dataflow analysis.

1. SAPT Construction: The SAPT records the potential complex hierarchical structures of deserialized objects. Each node nx in the SAPT represents a class Cx and its property set Px. The tree utilizes two types of edges:

• Property Edges (solid arrows): Connect a parent node nx to a child node ny if nx's property p is of class Cy.
• Inference Edges (dotted arrows): Represent relationships inferred from inheritance or dataflow, indicating that an object of class Cy can be assigned to a property of class Cx.

1. Root Node Identification: The first step is identifying the root node n0 of the SAPT, which corresponds to the class of the object returned by a deserialization entry. DESERIGUARD identifies these entries from known libraries (e.g., ObjectInputStream.readObject, XStream.fromXML). Rather than broadly assuming Object or Serializable, DESERIGUARD traces dataflow from these entries to identify more precise typecastings. For example, if a deserialized object is consistently cast to Profile or Session, then n1: Profile and n2: Session become children of the initial root n0.

1. Property Edges Connection: This step recursively analyzes the properties of each node. For instance, if n2: Session has properties like user, values, expirationTime, and id, a property edge ep(n2, n4) would connect Session to User because Session.user is of type User. This recursive analysis continues until basic types (e.g., String, primitives) or classes without complex inheritance relations are encountered.

1. Inference Edges Solvement: This critical phase handles inheritance and dataflow-inferred relationships. If class Cy is a subclass of Cx, an object of Cy can be assigned to a property of Cx. For example, if User has subclasses Admin and Guest, inference edges ei(n4, n7) and ei(n4, n8) would connect User to Admin and Guest respectively. A particular challenge arises with generic classes like Object, Serializable, or Comparable. Indiscriminately allowing all their subclasses would permit a vast number of potentially exploitable classes. DESERIGUARD addresses this by performing meticulous dataflow tracing from property access operations to infer the actual specific classes required by the application. This involves identifying explicit typecasting operations (Class.cast()) and type comparisons (instanceof, isAssignableFrom). For example, if the id property of Session is declared Comparable, but dataflow analysis reveals it is consistently checked against IntID, AuthKey, and HashVal, then inference edges ei(n6, n9), ei(n6, n10), and ei(n6, n11) are added.

1. Permitted Classes Identification: Finally, DESERIGUARD generates the allowlist policy by traversing the constructed SAPT and collecting all reachable class nodes. For generic classes, it delves into their subtrees to deduce stricter policies. If dataflow cannot be fully resolved due to insufficient semantics (e.g., object sent over network, stored on disk), DESERIGUARD adopts the currently resolved classes to over-approximate the necessary classes in the subsequent dataflow and alerts developers, prioritizing application functionality over absolute minimal policy. To synthesize a concise policy, if a parent class is included in the allowlist, its subclasses are also implicitly permitted during enforcement, simplifying the policy representation. DESERIGUARD synthesizes a customized allowlist policy for each deserialization entry in the application, even protecting entries not immediately exposed to attackers (e.g., deserialization from a trusted database, as seen in CVE-2022-40955), due to the difficulty of accurately identifying all exposed entries and the minor overhead involved.

Policy Enforcement Module

The Policy Enforcement module ensures real-time safeguarding of the deserialization process without requiring source code modifications. DESERIGUARD achieves this using a Java agent to instrument the application's bytecode when classes are loaded by the JVM.

1. Type Auditing Activation and Policy Specification: DESERIGUARD instruments deserialization entry points to set and unset a flag. Before a deserialization entry is invoked, the flag is set, and the specific allowlist policy for that entry is loaded. Immediately after the entry returns, the flag is unset. When the flag is set, DESERIGUARD actively monitors the deserialization procedure against the customized policy.

1. Auditing Position Identification: To perform type auditing, the Java agent must identify the precise position to check the class of the deserialized object. This position is critical: it must be after the object's type is resolved from the input byte stream but before any potential gadget chain can be triggered. Deserialization libraries typically resolve the object's class using a resolving method (e.g., desc.forClass() in ObjectInputStream) and then construct the object using constructors or reflection. DESERIGUARD first statically identifies all methods that resolve Class objects from byte streams. It then performs runtime validation by generating a deserialization driver that triggers the relevant deserialization entry with a workload object of a known class (e.g., Session). It monitors the candidate positions to determine which one is activated and confirms that the Class object produced by the resolving method corresponds to the object provided in the driver. This process accurately identifies the proper invocation position of the resolving method for type auditing (e.g., after desc.forClass() within readOrdinaryObject).

1. Actual Type Auditing: At the identified auditing position, DESERIGUARD loads the corresponding Class object of the deserialized type. It then uses the isAssignableFrom method to determine if this deserialized class is a subclass of any class present in the allowlist. If the deserialized class falls outside the policy, DESERIGUARD immediately blocks the deserialization process by throwing an exception, effectively preventing gadget chain execution. To minimize runtime overhead, DESERIGUARD caches the Class objects of allowlisted classes, significantly reducing lookup times during repeated deserializations.

Implementation Details

For policy synthesis, DESERIGUARD leverages CodeQL to perform its static analysis on the program's source code. CodeQL's dataflow analysis capabilities are tailored to locate typecasting and type comparison statements, which are crucial for identifying potential types. From these initial types, DESERIGUARD recursively traces dataflow on property access statements and inheritance relations to construct the SAPT. During this tracing, careful filtering is applied to avoid over-permission, such as ignoring implicit typecastings to generic superclasses (e.g., Object in toString(Object o)). When objects are pushed into containers (e.g., HashMap), DESERIGUARD traces the dataflow of their elements, referring to the original object's class to infer corresponding types if an element can be cast to various classes. Challenges posed by Java's complex mechanisms like reflection and Java Native Interface (JNI) invocations are addressed by performing self-referencing reflection analysis to gather potentially invoked methods and by modeling frequently used JNIs with CodeQL to connect their dataflow. In cases where dataflow remains unresolved (e.g., object sent over network or stored on disk), DESERIGUARD adopts the currently solved classes to over-approximate the necessary classes, ensuring that legitimate functionality is not disrupted, and alerts developers to potential areas of ambiguity.

For policy enforcement, DESERIGUARD is implemented as a Java agent, attaching to the application during its initialization phase. It utilizes the ASM library for efficient bytecode instrumentation. The agent dynamically loads the allowlist policies and performs real-time type auditing.

Demo / Proof of Concept

▶ Watch: Ofbiz case study: Flawed manual policy failures (3:40)

While the talk does not describe a live, interactive demonstration in the traditional sense, the evaluation section serves as a robust and empirical proof of concept for DESERIGUARD's capabilities. The speaker presented extensive experimental results validating DESERIGUARD's effectiveness against real-world vulnerabilities and its performance advantages over existing approaches.

The evaluation included:

• Defense against 12 real-world vulnerabilities: DESERIGUARD was deployed on complex applications, averaging 1.11 million lines of code and 20.68 thousand classes, known to be vulnerable to specific deserialization attacks (e.g., CommonsCollections, RMI, Groovy gadget chains). In every instance, DESERIGUARD successfully blocked the exploitation attempts. For example, it generated policies for different versions of Ofbiz (e.g., 623 rules for 17.12.05, 392 for 17.12.06), showcasing its adaptive synthesis for complex, evolving applications.
• Resistance to known gadget chains: DESERIGUARD successfully blocked all 33 Ysoserial gadget chains and all potential gadget chains discovered by the advanced gadget mining tool GadgetInspector across the 12 vulnerable applications. This demonstrated its proactive defense against a broad spectrum of known and newly discovered attack vectors.
• Comparison with developer-designed policies: The talk presented visual and quantitative comparisons (Figure 9 and 10) showing DESERIGUARD's ability to synthesize policies that are vastly stricter—permitting 90 times fewer classes—than those manually crafted by developers. This effectively demonstrated that DESERIGUARD's automated approach yields more secure policies in practice.
• Performance and false alarm validation: The extensive testing, including monitoring hundreds of thousands of deserialization instances on Jenkins and Ofbiz, confirmed DESERIGUARD's negligible overhead (average 2.168% slowdown) and, critically, zero false alarms, proving its suitability for production environments.
• Outperformance of policy learning: A direct comparison with the state-of-the-art policy learning tool, Trusted Execution Path (Trusted), highlighted DESERIGUARD's robustness. While both mitigated the vulnerabilities, Trusted suffered from false alarms on 8 applications due to its reliance on benign workloads, a limitation DESERIGUARD's semantic-aware synthesis avoids entirely.
• Blocklist bypass demonstration: A straw-man experiment vividly illustrated the inadequacy of traditional blocklists. A blocklist designed to counter pre-2021 Ysoserial chains was bypassed by the 2021-discovered AspectJWeaver gadget chain on multiple applications, while DESERIGUARD remained effective, serving as a strong proof point for the necessity of allowlist-based approaches.

These rigorous evaluations across diverse metrics and scenarios collectively serve as a compelling proof of concept for DESERIGUARD's practical utility and effectiveness in securing applications against deserialization attacks.

Defensive Implications

▶ Watch: DESERIGUARD's approach to overcome policy challenges (4:15)

DESERIGUARD offers profound implications for how developers and organizations approach the defense against Java deserialization vulnerabilities, shifting the paradigm from reactive, manual efforts to proactive, automated security.

For Developers:

• Abandon Manual Blocklists and Loose Allowlists: The talk unequivocally demonstrates that manually maintained blocklists are inherently insufficient against the continuous discovery of new gadget chains (e.g., the AspectJWeaver bypass). Similarly, loosely defined allowlists (like Ofbiz's java..* pattern) are prone to bypasses and introduce significant attack surface. Developers should transition away from these error-prone strategies.
• Embrace Automated Policy Synthesis: The sheer complexity of modern applications makes it infeasible for human developers to accurately identify all necessary deserialized classes. Tools like DESERIGUARD, which leverage semantic-aware dataflow analysis to automatically synthesize precise allowlist policies, are essential. This reduces the burden on developers, allowing them to focus on application logic rather than intricate security policy formulation.
• Understand Dangerous Generic Types: Developers should be acutely aware of the risks associated with permitting generic types like Object, Serializable, or Comparable in deserialization contexts without highly specific type inference. DESERIGUARD's approach of tracing dataflow to infer precise types (e.g., IntID for Comparable) highlights the necessary granularity.
• Protect All Deserialization Entries: The finding that vulnerabilities can arise even from "trusted" deserialization sources (e.g., a database in CVE-2022-40955) underscores the need to protect all deserialization entry points, not just those directly exposed to external network input. DESERIGUARD's strategy of protecting all entries with minor overhead is a best practice.

For Organizations:

• Integrate Static Analysis into SDLC: Organizations should integrate advanced static analysis tools, such as CodeQL, into their Software Development Life Cycle (SDLC) to facilitate the automatic generation of security policies. This shifts security left, enabling proactive defenses from the development stage.
• Adopt RASP-like Solutions: DESERIGUARD functions as a powerful Runtime Application Self-Protection (RASP) framework for deserialization. Organizations should consider deploying such solutions that leverage precise, automatically generated policies to provide real-time protection without code modifications. The negligible runtime overhead (average 2.168% slowdown) makes DESERIGUARD a practical choice for production environments.
• Prioritize Allowlist Over Blocklist Strategies: The evidence strongly favors allowlist policies as a more robust and proactive defense. Organizations should mandate the use of allowlist-based deserialization protection, ideally automated, across their application portfolios.
• Reduce Attack Surface: By restricting deserialization to only the absolutely necessary classes (DESERIGUARD restricts 99.12% more classes than manual policies), organizations drastically reduce the available gadget chains for attackers, making exploitation significantly more difficult.

General Implications:

DESERIGUARD represents a significant step forward in combating one of the most persistent and dangerous vulnerability classes in Java applications. Its automated, precise, and low-overhead approach provides a robust, proactive defense that is scalable across complex codebases and adaptable to the evolving threat landscape, making it a crucial tool in the modern cybersecurity arsenal.

Key Takeaways

• Java deserialization vulnerabilities remain a critical and widespread threat: Approximately 800 CVEs related to CWE 502 in the last five years underscore the pervasive risk of untrusted deserialization leading to RCE, DoS, and SSRF.
• Manual policy formulation is ineffective and error-prone: Both reactive blocklists (easily bypassed by new gadgets like AspectJWeaver) and manually crafted allowlists (often too broad, like Ofbiz's java..* pattern, leading to bypasses) consistently fail to provide robust protection due to complexity and human error.
• DESERIGUARD automatically synthesizes strict, semantic-aware allowlist policies: By leveraging dataflow analysis to construct a Semantic-Aware Property Tree (SAPT), DESERIGUARD precisely identifies legitimate deserialized types, including handling generics and inheritance, to generate highly restrictive allowlists.
• DESERIGUARD provides highly effective and reliable defense: It successfully blocked all 12 real-world deserialization vulnerabilities, all 33 Ysoserial gadget chains, and all GadgetInspector-discovered chains, critically achieving this with no false alarms during extensive testing.
• Policies are significantly more precise and secure than developer-designed ones: DESERIGUARD's synthesized policies restrict 99.12% more classes than typical developer-defined policies, drastically reducing the attack surface by permitting 90 times fewer classes on average.
• The system operates with negligible runtime overhead: With an average slowdown of only 2.168% and an average auditing cost of 0.039 milliseconds per deserialization, DESERIGUARD is an efficient and practical real-time defense framework suitable for production environments.

About the Speaker(s)

The talk "Automatic Policy Synthesis and Enforcement for Protecting Untrusted Deserialization" was presented by Quan Zhang. Based on the technical depth and research focus of the presentation at a prominent security conference like NDSS, Quan Zhang is likely a researcher or PhD candidate specializing in application security, particularly in areas related to static analysis, program semantics, and runtime protection mechanisms for Java applications. The work presented, DESERIGUARD, reflects expertise in addressing complex software vulnerabilities through automated policy generation and enforcement.