Unseen in the Stack: Mapping Hidden Java Dependencies for Real-World Defense

Unseen in the Stack: Mapping Hidden Java Dependencies for Real-World Defense

Speakers: Oron Gutman, Co-founder and CTO, Hopper Security

Conference: BSides NYC

YouTube: https://www.youtube.com/watch?v=Niv2L_e4dWI

Overview

This talk, presented by Oron Gutman, Co-founder and CTO of Hopper Security, unveils critical research into the pervasive yet often overlooked problem of shaded and repackaged libraries within the Java ecosystem. Based on an extensive analysis of real-world Maven repositories and build artifacts, Gutman demonstrates how these hidden dependencies introduce millions of undetectable vulnerabilities into production applications. The core message is stark: traditional Software Composition Analysis (SCA) tools and Software Bill of Materials (SBOMs), which primarily rely on declared dependencies in pom.xml files or dependency trees, are fundamentally blind to this issue, leaving organizations exposed to significant and exploitable risks.

The presentation meticulously details the mechanisms of shading and how it allows vulnerable code to persist undetected, even in seemingly clean and up-to-date applications. Gutman illustrates this with a concrete example, showing how standard vulnerability scanners fail to identify critical flaws in a widely used library. The research reveals the alarming scale of this problem, quantifying the sheer number of hidden vulnerabilities, including thousands of critical and CISA KEV (Known Exploited Vulnerabilities) entries. This talk is crucial for any organization developing or deploying Java applications, as it exposes a significant gap in current security practices and offers insights into more robust defensive strategies.

Background

▶ Watch: Introduction to talk on shaded/repackaged Java dependencies (0:00)

Managing open-source software (OSS) risk is a cornerstone of modern application security. The standard approach involves a four-step process: first, gaining visibility into all software components via an SBOM; second, identifying risk by cross-referencing components with vulnerability databases like NVD, OSV, or proprietary sources; third, prioritizing risk based on metrics like CVSS, EPSS, or CISA KEVs; and finally, remediation, typically through version bumps or backporting. However, this established methodology often falters in the complex landscape of Java development.

Java stands as the number one backend language for Fortune 500 companies, with nearly 50% of their applications built on it. Its ecosystem is notoriously complex, characterized by extensive frameworks like Spring and Jakarta that introduce deeply transitive dependencies. This complexity means a single direct dependency can pull in dozens of others, creating a vast and intricate dependency graph. The impact of vulnerabilities in Java is consequently immense, as a single flaw can affect hundreds of thousands of downstream libraries and applications, as famously demonstrated by the Log4j incident.

Java applications are primarily built using Maven, Gradle, or Bazel. Maven, the largest, uses pom.xml files to define project metadata, direct dependencies, and the build lifecycle. During a Maven build, dependencies are resolved from repositories like Maven Central or internal artifactories. Dependency conflicts, where different libraries require the same dependency in incompatible versions, are common. Maven typically resolves these by choosing the "nearest to the root" dependency, but this can lead to runtime errors if the chosen version breaks functionality for other components. To mitigate these "dependency hell" scenarios and simplify deployment, especially for executable JARs, two primary techniques emerged: Uber JARs and Shading. Uber JARs bundle all application classes and their dependencies into a single JAR, often using plugins like Maven Shade Plugin or Gradle Shadow Jar Plugin. Shading takes this a step further by not only embedding dependencies but also relocating their packages (e.g., renaming com.apache.commons to com.myproject.shaded.apache.commons) to prevent classpath conflicts with other versions of the same library. A critical side effect of shading is the creation of a reduced POM, where the shaded dependencies are removed from the project's declared dependencies, making them invisible to standard dependency analysis.

Key Findings

▶ Watch: Why Java is a key focus for dependency analysis (3:00)

The central and most alarming finding of Oron Gutman's research is that millions of vulnerabilities are hidden within Java applications due to the widespread practice of shading and repackaging libraries. Traditional SCA tools, which rely on examining pom.xml files, Maven dependency trees, or publicly available vulnerability databases like NVD, OSV (via deps.dev), and even proprietary databases like Snyk, are fundamentally incapable of detecting these embedded, relocated dependencies. This creates a massive blind spot in an organization's security posture.

Gutman demonstrated this critical flaw using com.auth0.java-jwt version 1.0.0. Initial scans with mvn dependency:tree, checks against Maven Central's POM, deps.dev, and Snyk all reported no vulnerabilities for this specific version. However, a deeper, artifact-level scan using a tool like Trivy revealed 49 vulnerabilities, including 19 critical and 27 high-severity issues, specifically identifying Jackson-core and Jackson-databind at version 2.0.0 as the culprits. These vulnerable components were present because they had been shaded and embedded within com.auth0.java-jwt, making them invisible to metadata-based analysis.

The large-scale analysis performed by Hopper Security on over 16 million artifacts in Maven Central quantified the true impact:

• Over 2.5 million hidden vulnerabilities were discovered.
• Nearly 5,000 vulnerable packages were found to still contain shaded vulnerabilities even in their latest available versions, indicating a lack of available fixes.
• More than 8,000 CISA KEV (Known Exploited Vulnerabilities) were present, many of which have public Proof-of-Concepts (PoCs) and are actively exploited in the wild.
• Approximately half a million critical vulnerabilities were identified.
• Over 3,000 instances of Log4j vulnerabilities were found hidden in shaded dependencies, some even in the latest versions of certain packages.

These findings highlight that common, highly critical vulnerabilities related to libraries like Guava, Jackson Data Bind, and Log4j are frequently obscured by shading, posing a severe and unaddressed risk to countless Java applications.

Technical Deep Dive

▶ Watch: Understanding Java build systems and JAR packaging (4:40)

The technical core of the problem lies in how Java dependencies are managed and packaged, particularly the mechanism of shading. In a typical Maven build, direct and transitive dependencies are resolved based on the pom.xml. Maven's conflict resolution strategy, "nearest to the root," attempts to pick a single version for a shared dependency. However, when version incompatibilities are severe, or when a library developer wants to ensure a specific version of a dependency is always available without external conflicts, they resort to shading.

Shading involves two key steps, often facilitated by plugins like the Maven Shade Plugin:

1. Embedding: The complete bytecode of the dependency is copied directly into the consuming JAR file.
2. Relocation (or Repackaging): The package names of the embedded classes are systematically changed. For example, com.fasterxml.jackson might be relocated to com.auth0.jwt.internal.fasterxml.jackson. This ensures that if the main application or another direct dependency also pulls in a com.fasterxml.jackson library, there won't be a CLASSPATH collision, as the shaded version now exists under a unique, internal namespace.

The critical security implication of shading is the creation of a reduced POM. When a library is shaded and published to Maven Central, its pom.xml is modified to remove the entries for the dependencies that were embedded and relocated. From the perspective of Maven Central or any standard dependency analysis tool, these shaded dependencies simply do not exist. This metadata-level invisibility is precisely why tools relying on pom.xml or dependency trees fail to detect the actual components running within an application.

Gutman's example with com.auth0.java-jwt version 1.0.0 perfectly illustrates this.

• Initial Scan (Failure): Running mvn dependency:tree on a project consuming com.auth0.java-jwt v1.0.0 shows only JUnit as a test-scoped dependency, which is irrelevant for runtime. Checks on deps.dev and Snyk also report no vulnerabilities. This is because the published pom.xml for com.auth0.java-jwt v1.0.0 has a reduced POM, omitting its shaded dependencies.
• Deep Dive (Discovery): When the com.auth0.java-jwt v1.0.0 JAR is unzipped, an interesting file is found within the META-INF/maven/ directory: pom.properties. This file, while not a full POM, often contains clues about the original build, revealing the presence of Jackson-core (version 2.0.0) and Jackson-databind (version 2.0.0). These versions are known to have significant vulnerabilities.
• Source Code Confirmation: Inspecting the GitHub repository for com.auth0.java-jwt v1.0.0 reveals the actual build configuration. The pom.xml explicitly uses the maven-shade-plugin with the goal shade and includes <relocations> directives. These directives confirm that com.fasterxml.jackson packages are indeed being relocated, specifically to com.auth0.jwt.internal.fasterxml.jackson.
• Vulnerability Reachability: Hopper Security's platform, which performs static function-level reachability analysis on compiled bytecode, was able to confirm that five of the 49 detected vulnerabilities were reachable. By analyzing the call graph, it showed how a user-controlled input (e.g., verifying a JWT token) could trigger the decode and parse functions, eventually leading to the vulnerable Jackson-databind functions, even with the relocated package names.
• The Real Fix: The critical insight is that upgrading the directly declared vulnerable component (e.g., Jackson-databind) is impossible because it's not a direct dependency. The only effective remediation is to upgrade the parent library that performs the shading. In this case, upgrading com.auth0.java-jwt from version 1.0.0 to a later version (e.g., 3.0.1) that has internally updated its shaded Jackson dependencies to non-vulnerable versions.

This detailed technical analysis underscores that a complete understanding of an application's dependency graph requires inspecting the compiled artifacts themselves, not just their declared metadata.

Demo / Proof of Concept

▶ Watch: Explaining dependency conflicts and 'dependency hell' in libraries (9:00)

While the talk did not feature a live, interactive demo of a specific tool, Oron Gutman provided a highly effective, step-by-step walkthrough that served as a compelling proof-of-concept for the hidden dependency problem. He meticulously demonstrated how various standard security analysis methods fail to detect vulnerabilities introduced by shading, and then revealed how a deeper, artifact-centric approach uncovers them.

The demonstration began by creating a simple Java application with a single direct dependency: com.auth0.java-jwt version 1.0.0. Gutman then proceeded to:

1. Illustrate the "Blind Spot":

• He ran mvn dependency:tree, which showed only the direct dependency and no transitive ones of concern (only JUnit in test scope).
• He consulted Maven Central's pom.xml for com.auth0.java-jwt v1.0.0, which confirmed the lack of declared runtime dependencies.
• He checked deps.dev (Google's open-source inventory based on OSV.dev), which reported no vulnerabilities for com.auth0.java-jwt v1.0.0.
• He scanned the project with Snyk, a popular proprietary SCA tool, which also reported no vulnerabilities.

These steps clearly showed how all conventional, metadata-driven tools would falsely conclude the application was secure.

1. Reveal Hidden Vulnerabilities:

• Gutman then introduced the output from a compiled artifact scanner (implied to be Trivy, an open-source tool, though the exact tool for the specific output wasn't explicitly stated as Trivy in the talk, but it's a common example of such a scanner). This scan, performed on the compiled JAR artifact, shockingly revealed 49 vulnerabilities, including 19 critical and 27 high-severity issues, specifically mentioning Jackson-core and Jackson-databind at version 2.0.0. This immediately highlighted the discrepancy between metadata-based and artifact-based analysis.
• He then showed how attempting to resolve these newly found Jackson dependencies via mvn dependency:tree again yielded no results, reinforcing that they were not part of the declared dependency graph.

1. Uncover the Shading Mechanism:

• To explain how these vulnerabilities were hidden, Gutman manually inspected the com.auth0.java-jwt v1.0.0 JAR by unzipping it. He navigated to the META-INF/maven/com.auth0/java-jwt/pom.properties file, which contained references to the embedded Jackson libraries.
• He then went to the GitHub source code for com.auth0.java-jwt v1.0.0, demonstrating the maven-shade-plugin configuration in its pom.xml. This clearly showed the <relocations> directives, proving that Jackson packages were being embedded and renamed (e.g., to com.auth0.jwt.internal.fasterxml.jackson).

1. Demonstrate Reachability and Remediation:

• Finally, using a call graph visualization from Hopper Security's platform, Gutman illustrated how the vulnerable shaded Jackson functions could be reached from the application's entry points (e.g., JWT token verification), confirming the real-world risk.
• He concluded by emphasizing that the effective remediation was not to upgrade Jackson directly (which was impossible) but to upgrade the parent library, com.auth0.java-jwt, to a version that internally uses a patched Jackson dependency.

This comprehensive walkthrough effectively served as a proof-of-concept, demonstrating the existence, severity, and technical underpinnings of hidden shaded dependencies and their associated vulnerabilities.

Defensive Implications

▶ Watch: How shading embeds and renames dependencies to resolve conflicts (10:40)

The findings presented in "Unseen in the Stack" demand a significant shift in defensive strategies for organizations developing and deploying Java applications. The current reliance on traditional SCA tools and SBOMs, which are often generated from source code or pom.xml files, is demonstrably insufficient and creates a false sense of security.

Here are the key defensive implications:

1. Prioritize Binary/Artifact Scanning over Metadata Scanning: Defenders must move beyond scanning source code or pom.xml files alone. The primary focus should be on scanning the final compiled artifacts (JARs, WARs, EARs) that are deployed. Tools capable of performing static analysis on bytecode are essential to unpack, inspect, and identify all embedded and relocated classes, regardless of their original package names or whether they are declared in a pom.xml.
2. Rethink SBOM Generation: Current SBOM generation practices, especially those derived solely from build metadata, will be incomplete. To create an accurate SBOM, the process must incorporate analysis of the compiled binaries to enumerate all components, including those introduced through shading. This means adopting tools and methodologies that can generate SBOMs from the built artifact, reflecting its true composition.
3. Understand the Nuances of Dependency Resolution: Developers and security teams need a deeper understanding of Java's dependency resolution mechanisms, including Maven's "nearest to the root" and, critically, the implications of shading. Awareness that a library might internally package its own version of a common dependency is crucial for debugging and security analysis.
4. Focus Remediation on the Shading Parent: When a vulnerability is identified in a shaded dependency, attempting to upgrade that specific dependency directly will fail, as it's not a declared dependency. The correct remediation path is to identify and upgrade the parent library (the one performing the shading) to a version that has internally addressed the vulnerability by incorporating a patched version of its shaded component. This requires tools that can map a vulnerable shaded component back to its originating parent.
5. Be Skeptical of "Clean" Reports: Security teams should approach "no vulnerabilities found" reports from traditional SCA tools with extreme skepticism, especially for Java applications. These reports are likely missing an entire class of hidden vulnerabilities. This necessitates investing in advanced scanning capabilities or augmenting existing processes with artifact-level analysis.
6. Evaluate Tooling Capabilities: Organizations should critically evaluate their existing SCA and vulnerability management tools. Do they have the capability to:

• Unpack and analyze compiled Java bytecode?
• Detect relocated packages and their original identities?
• Identify vulnerabilities within these hidden components?
• Generate call graphs to determine reachability for shaded code?

If not, new tools or methodologies are required to fill this critical gap.

1. Consider the Impact of CISA KEVs: The finding of over 8,000 CISA KEVs hidden in shaded dependencies is particularly alarming. These are vulnerabilities known to be actively exploited. Organizations must assume that if they are running Java applications, they are likely exposed to these KEVs through shaded components and must prioritize their discovery and remediation immediately.

In summary, the talk highlights that Java's ecosystem complexity, exacerbated by practices like shading, demands a more sophisticated and granular approach to application security. Defenders must look deeper into the compiled binaries to truly understand their attack surface.

Key Takeaways

• Millions of Hidden Vulnerabilities: The practice of shading and repackaging Java libraries has introduced over 2.5 million vulnerabilities into the ecosystem, including thousands of critical and CISA KEVs, that remain undetected by traditional security tools.
• Traditional SCA Tools are Blind: Standard Software Composition Analysis (SCA) tools, SBOMs generated from source, and dependency tree analysis (e.g., mvn dependency:tree) cannot detect vulnerabilities within shaded dependencies because they rely on metadata (pom.xml) that omits these embedded components.
• Shading Alters Package Names: Shading involves embedding a dependency's bytecode directly into a JAR and relocating its package names (e.g., com.fasterxml.jackson becomes com.auth0.jwt.internal.fasterxml.jackson) to prevent classpath conflicts.
• Vulnerabilities are Reachable and Exploitable: Despite being hidden, vulnerabilities in shaded dependencies are often reachable through the parent library's functionality, posing real and exploitable risks to applications.
• Artifact-Level Scanning is Essential: To detect these hidden flaws, organizations must implement tools and processes that scan compiled artifacts (JARs, WARs) at the bytecode level, rather than solely relying on source code or build metadata.
• Remediate by Upgrading the Parent Library: The effective remediation for a vulnerability in a shaded dependency is to upgrade the parent library (the one performing the shading) to a version that has internally patched or updated its embedded components.

About the Speaker(s)

Oron Gutman is the Co-founder and CTO of Hopper Security, a company specializing in static function-level reachability analysis for C and container scanning. With a robust background spanning 15 years in vulnerability research, Oron has extensive experience managing both vulnerability researchers and software engineers. His expertise lies in uncovering deep-seated security issues within software architectures, a skill evident in his research on hidden Java dependencies. This talk at BSides NYC marked his first time speaking at the conference.