(Mis)adventures with Copilot+: Attacking and Exploiting Windows NPU Drivers

(Mis)adventures with Copilot+: Attacking and Exploiting Windows NPU Drivers

Speakers: A graduate student and their supervisor Gangur, UN Surirk University of Applied Science

Conference: Black Hat Asia

YouTube: https://www.youtube.com/watch?v=K_C5-wZCAN0

Overview

The advent of Copilot+ PCs marks a significant shift in Windows computing, deeply integrating Artificial Intelligence capabilities directly into the operating system. This talk, "(Mis)adventures with Copilot+: Attacking and Exploiting Windows NPU Drivers," presented by a graduate student from UN Surirk University of Applied Science and their supervisor Gangur, delves into the security implications of this new paradigm. The core focus is on the Neural Processing Unit (NPU) drivers, which are the backbone of Copilot+ features like local AI model execution, live translation, and the controversial Recall functionality.

The presentation highlights that while NPUs offer compelling benefits—addressing privacy concerns by keeping data local, reducing cloud inference costs, and improving availability—they also introduce a novel and complex attack surface. The speakers reveal how NPU drivers, by design, inherit architectural weaknesses and vulnerabilities from their GPU counterparts, leading to critical kernel-mode exploits. This research underscores the urgent need for a thorough security review of these nascent technologies to safeguard the future of AI-powered computing.

The talk provides a detailed exploration of two specific kernel vulnerabilities found in Qualcomm Snapdragon and AMD Ryzen NPU drivers, demonstrating how an attacker can leverage these flaws to achieve arbitrary kernel read/write primitives and ultimately escalate privileges to a system shell on Windows 24H2. The findings serve as a stark reminder that as new hardware and software layers are introduced for AI, established security principles, particularly input validation and robust isolation, must not be overlooked.

Background

▶ Watch: AI's privacy, cost, and availability challenges (0:35)

The current landscape of Artificial Intelligence is dominated by large language models (LLMs), presenting significant challenges related to privacy, cost, and availability. Cloud-based LLMs raise privacy concerns, as exemplified by Microsoft Recall's initial design to store user activity data in the cloud. The immense computational resources required for training and inference make cloud AI expensive, and dependency on external services introduces availability risks.

A partial solution to these challenges is local LLM execution. This can be achieved either on a GPU (a "sprinter" – fast, high power consumption) or an NPU (a "marathon runner" – lower power, sustained performance). NPUs are particularly well-suited for continuous, low-power AI tasks like camera background blur, making them ideal for integration into laptops. Key NPU suppliers for laptops include Intel, Qualcomm, and AMD, with AMD adopting a unique approach by incorporating FPGA (Xilinx) technology.

Copilot+ PCs are Microsoft's answer to integrating AI locally, featuring NPUs capable of 40 trillion operations per second (TOPS). These devices power features such as live translation, local Recall (now thankfully running on-device), camera blur, and upcoming functionalities like super resolution. The underlying architecture of this "magic AI machine" is described as an "onion" with multiple layers:

1. Application
2. ONNX Runtime
3. DirectML
4. DirectX12 User Mode Driver (UMD) or Kernel Mode Driver (KMD)

An additional upcoming feature, WebNN, allows LLMs to run directly in the browser, sharing a similar architecture through ONNX Runtime Web and the WebNN API to DirectML.

ONNX Runtime is a crucial component, providing a cross-platform, cross-language runtime for LLMs, simplifying deployment. Beneath this, DirectML serves as a COM-based DirectX extension, known for being particularly challenging to reverse engineer due to its C++ and COM intricacies.

The speaker's research focused on the kernel-mode driver path, considering it a definitive security boundary, unlike the user-mode driver DLLs which can be loaded within the same process. This path involves Device Driver Interface (DDI) functions, a well-known attack surface first highlighted in a Black Hat presentation in 2014, and now resurfacing with NPUs. The call chain for these DDI functions involves the application interacting with UMD vendor driver DLL or GDI32.DLL, which then performs a win32k.sys call into win32k.sys. This then passes parameters to dxgkrnl.sys, which performs some (but not all) validation, before forwarding them to the WDDDM or MCDM driver, which directly interacts with the GPU or NPU.

WDDDM (Windows Driver Device Model) is the standardized interface for GPUs in Windows, encompassing memory management and rendering. Since 2014, the complexity and number of functions available through WDDDM have grown significantly. In 2021, Microsoft introduced the MCDM (Microsoft Compute Driver Model) specifically for NPUs. MCDM is a subset of WDDDM, requiring 38 functions, making 42 optional, and disallowing 29. This shared interface between GPUs and NPUs is a critical point, as it means NPU drivers might inherit vulnerabilities present in the older, more complex GPU driver model. Notably, some DxgkDdiQueryAdapterInfo types, officially unsupported by MCDM, are still implemented by vendors, indicating that Microsoft's specifications are not always fully enforced.

Key Findings

▶ Watch: Running LLMs locally: GPU vs. NPU (2:15)

The research uncovered several critical findings regarding the security posture of Windows NPU drivers, particularly those underpinning Copilot+ features:

• Inherited Vulnerabilities from GPUs: The MCDM (Microsoft Compute Driver Model) for NPUs is a subset of the WDDDM (Windows Driver Device Model) used by GPUs. This architectural reuse means NPU drivers often implement interfaces and code paths similar to or directly derived from GPU drivers, inheriting long-standing vulnerabilities that have plagued GPU drivers for over a decade.
• Unvalidated DDI Parameters: A significant attack vector lies within DDI (Device Driver Interface) functions. Specifically, the PrivateDriverData and PrivateDriverDataSize parameters are often not validated by dxgkrnl.sys. Instead, they are directly passed from user mode to the WDDDM or MCDM driver. This "hands-off" approach by Microsoft makes these parameters highly attractive to attackers, as they can be used to smuggle malicious input into the kernel without prior sanitization.
• DirectX Escape Functions as a Critical Attack Surface: DirectX escape functions are designed to allow user-mode drivers to communicate custom, proprietary information or commands to their kernel-mode counterparts (e.g., vendor-specific anti-lag features). Because these functions handle custom code and often bypass standard interface validation, they represent a fertile ground for vulnerabilities. The shared interface between GPUs and NPUs extends this risk to NPU drivers.
• Potential for Process Interference: While currently hypothetical, the speaker raised concerns about process interference. If future Copilot+ features allow LLMs to modify system settings, a scenario could arise where one LLM process (e.g., belonging to Process 1) could interfere with or alter the output of another LLM process (e.g., belonging to Process 2), potentially leading to privilege escalation or data manipulation.
• NPU Firmware as a Research Target: The firmware itself, often loaded by the NPU driver (e.g., AMD's historical use of XCL bin for Xilinx FPGA firmware), presents another area for security research. Although vendors are moving away from loading entire binaries from disk, modifying certain firmware parts via device creation or escape functions remains a possibility.
• Ineffective Current Mitigations: Microsoft's primary mitigation, disabling win32k system calls, is a blunt instrument. It's an "all or nothing" switch that disables access to the GPU, NPU, user32.DLL, GDI32.DLL, and effectively breaks any GUI functionality. This makes it impractical for most applications. While Chrome employs a multi-process architecture to enable this mitigation for its renderer, WebNN's current developer preview still requires disabling the GPU sandbox.
• Specific Kernel Vulnerabilities:
• Qualcomm Snapdragon NPU Driver: A vulnerability (CVE-XXXX, as referenced by the speaker) in an escape function handler allows an arbitrary write-what-where. User-supplied PrivateDriverData is passed to MmProbeAndLockPages with AccessMode set to zero, enabling the locking of kernel virtual memory. Subsequent operations then dereference a user-controlled pointer and write a user-controlled value to that kernel address.
• AMD Ryzen NPU Driver: Another vulnerability (CVE-XXXX, as referenced by the speaker) involves an integer overflow and truncation within a private driver data validation function. This flaw allows an attacker to bypass size checks for subsequent data structures, leading to a dynamic heap or paged pool overflow, enabling multiple out-of-bounds writes at different locations.

These findings collectively highlight that despite being a new technology, NPU drivers are susceptible to classic kernel exploitation techniques, necessitating a renewed focus on fundamental security practices.

Technical Deep Dive

▶ Watch: Copilot+ key features and 'onion' architecture (3:55)

The technical deep dive into NPU driver exploitation begins with understanding how to interact with these devices from user mode. To obtain a handle to an NPU, one must first initialize a D3DKMT_ENUMADAPTERS2 structure, setting the IncludeComputeOnly option to 1. A D3DKMTEnumAdapters2 syscall is then made. Since this call returns handles for both GPUs and NPUs (due to their shared interface), a subsequent D3DKMTQueryAdapterInfo syscall, with the type set to DriverDescription, is used to inspect the resulting buffer and differentiate between the devices, ensuring the correct NPU handle is acquired.

The primary attack vectors identified by the speaker leverage weaknesses in the Device Driver Interface (DDI) functions, particularly concerning PrivateDriverData and PrivateDriverDataSize. These fields are part of the input structures for DDI calls and are often passed directly from user mode to the kernel-mode driver without explicit validation by dxgkrnl.sys. This creates a critical trust boundary issue, allowing an attacker to supply malicious data that the NPU driver's kernel component will process without prior sanitization.

A particularly potent class of vulnerabilities arises from DirectX escape functions. These functions are vendor-specific extensions to the standard DirectX API, allowing GPU and NPU manufacturers to implement proprietary features (e.g., anti-lag, custom performance optimizations) that don't fit into Microsoft's generalized DDI framework. Because they are custom implementations, they often lack the rigorous security scrutiny and validation applied to standard DDI functions, making them prone to bugs like buffer overflows, type confusion, and arbitrary memory access. The shared interface model (MCDM being a subset of WDDDM) means that NPU drivers are equally, if not more, susceptible to these types of vulnerabilities.

Beyond direct driver exploitation, the talk briefly touches on process interference as a future concern. As Copilot+ integrates LLMs that can modify system settings, an NPU driver vulnerability could potentially allow one process to manipulate the behavior or output of an LLM used by another, more privileged process, leading to unintended consequences or privilege escalation. The firmware itself, especially for AMD's NPUs which historically loaded Xilinx FPGA (XCL bin) firmware directly from disk, represents another avenue for attack. Although the loading mechanism has evolved, the possibility of modifying specific firmware components through device creation or other low-level interfaces remains an area for future research.

Regarding mitigations, Microsoft's win32k syscall disabling is a coarse-grained security feature. It's a binary switch: either all win32k syscalls are allowed, or none are. Disabling them effectively cripples any application requiring a GUI, as it blocks access to user32.DLL and GDI32.DLL, making it impractical for most real-world scenarios. While browsers like Chrome employ a multi-process architecture to isolate the renderer with this mitigation enabled, WebNN's current developer preview still requires disabling the GPU sandbox, indicating a lack of robust sandboxing for NPU-accelerated web content.

The speaker detailed two distinct kernel vulnerabilities:

1. Qualcomm Snapdragon NPU Arbitrary Write-What-Where:

• The vulnerability originates within an escape function handler.
• User-supplied PrivateDriverData is directly passed to a function named lock_virtual_address_size.
• Inside lock_virtual_address_size, the MmProbeAndLockPages kernel API is called with the AccessMode parameter set to 0 (UserMode). Crucially, if the AccessMode is 0, this function can be abused to lock kernel virtual address memory.
• The second buffer passed to MmProbeAndLockPages is subsequently dereferenced by write_value_to_out, which writes a user-controlled value (ValueToWrite) at a user-controlled address (VirtualAddressOne).
• This sequence of operations results in an arbitrary write-what-where primitive, allowing an attacker to write any 4-byte value to any valid kernel address.

1. AMD Ryzen NPU Dynamic Heap/Paged Pool Overflow:

• This bug resides in the validation logic for private driver data within an AMD NPU escape function.
• The validation function processes a series of internal "structs," each with an offset, minimal size, offset of the next struct, and data size.
• The vulnerability is an integer overflow and truncation occurring when uint32 + uint32 is compared to a uint32 size, and a uint64 value is truncated to uint32.
• This flaw allows an attacker to bypass the size check for subsequent structs. By manipulating the offset of the second struct and its size, the attacker can cause a dynamic heap or paged pool overflow.
• The "last byte written" primitive, controlled by the attacker, enables multiple out-of-bounds writes at different locations within the same allocation. This makes it a powerful and flexible primitive for exploitation.

These vulnerabilities underscore the critical need for comprehensive input validation and boundary checks in kernel-mode drivers, especially for new hardware interfaces that often reuse older, less secure design patterns.

Demo / Proof of Concept

▶ Watch: DirectML and choosing kernel mode for security analysis (6:10)

The core of the demonstration showcased a privilege escalation (privesc) exploit, leveraging the identified AMD NPU driver vulnerability to gain system privileges on Windows 24H2. The exploit chain is a data-only technique, which is considered highly stable for Windows kernel exploitation.

The steps for the privilege escalation are as follows:

1. Heap Spray WNF State Data: The first step involves "heap spraying" the kernel's paged pool with numerous WNF state data objects. WNF state data is described as a "Swiss Army knife" for paged pool exploitation because it allows control over allocated_size, data_size, and a user-defined buffer. Both allocated_size and data_size are set to the size of the user-controlled data behind them.

1. Triggering the Overflow: Using the AMD NPU integer overflow vulnerability, the attacker then targets one of the sprayed WNF state data objects. The allocated_size field of this object is overwritten to 0xFDFD (a large value, but within the 0x1000 hardcoded kernel limit for linear out-of-bounds writes), and the data_size is set to 0x7FD. This specific overflow allows for a stable, linear out-of-bounds read and write capability. By observing which WNF state data object has a different size, the attacker can identify the overflowed object.

1. Refining the Out-of-Bounds Primitive: After identifying the overflowed WNF state data, a second WNF state data object is overflowed. This time, the values are set to 0x100 and 0x1000, providing a stable, linear out-of-bounds read and write primitive. The DirectX escape allocation, which was used to trigger the overflow, is designed to clean itself up, leaving the controlled WNF state data object in place.

1. Arbitrary Read (based on Koshel's technique): The remaining WNF state data objects are then replaced with underlying tokens. The second, controlled WNF state data object is used to manipulate a _PNP_ISOLATION_HANDLE_ENTRY structure. By pointing this structure to user space and controlling IsolationPrefixMaximumLength and IsolationPrefixBuffer, an arbitrary read primitive into kernel memory is achieved.

1. Locating EPROCESS: To achieve privilege escalation, the exploit needs to find the EPROCESS structure of the current process. Based on a blog post by Starlabs (which still works on Windows 24H2), the EPROCESS pointer can be reliably located by traversing from the session object, searching for an LFH allocation (at 0xB0), finding an IIO_COMPLETION_OBJECT (at 0x38 offset), which is in the same allocation as an ETWR object, and finally finding EPROCESS at offset 0x30 within the ETWR object.

1. Arbitrary Write (Legacy vs. New Primitive):

• Legacy Method (Patched in 24H2): Historically, setting PreviousMode to zero allowed the use of NtReadVirtualMemory and NtWriteVirtualMemory to read/write directly into kernel space. However, this "PreviousMode trick" was patched in Windows 24H2.
• New Method (Arbitrary Increment for 24H2): The speaker developed a novel arbitrary increment primitive to bypass the 24H2 patch. This technique misuses reference counters within the _PNP_ISOLATION_HANDLE_ENTRY structure. The DuplicateToken API, when called from user space, increments the reference count of the target object if it's not zero.
• The goal is to flip the 0x14 bit within the Present and Enabled fields of the process token to set SeDebugPrivilege.
• By carefully calculating the offset to the target byte within the token (which is an 8-byte field) and repeatedly calling DuplicateToken a calculated number of times, the reference counter at that specific byte can be incremented.
• This increments the target bit to 1, effectively granting SeDebugPrivilege.
• With SeDebugPrivilege, the attacker can debug any unprotected process (e.g., winlogon.exe) and inject code to spawn a system shell, completing the privilege escalation.

The live demonstration showed a user-level process executing the exploit, successfully escalating to a system shell on a Windows 24H2 machine, validating the effectiveness of the arbitrary increment primitive against modern Windows mitigations.

Defensive Implications

▶ Watch: WDDM evolution and the new MCDM for NPUs (8:15)

The research presented on NPU driver vulnerabilities carries significant implications for defenders, necessitating a multi-faceted approach to security:

• Rigorous Input Validation: The most critical takeaway is the need for robust validation of all user-supplied input to kernel-mode drivers, especially for PrivateDriverData and parameters passed to DirectX escape functions. Microsoft and NPU vendors (Qualcomm, AMD, Intel) must ensure dxgkrnl.sys performs comprehensive checks, and vendor-specific drivers must not implicitly trust user-mode input. This includes explicit bounds checks, type verification, and sanitization of all data before kernel processing.
• Enhanced Driver Sandboxing and Isolation: The "all or nothing" nature of the win32k syscall mitigation is impractical. Microsoft should invest in more granular sandboxing mechanisms for NPU access, similar to how Chrome isolates its renderer process. This would allow applications to use NPU capabilities without granting broad access to sensitive kernel interfaces. Stronger process isolation is also crucial for LLM workloads to prevent potential process interference if Copilot+ features gain system-level control.
• Secure Firmware Lifecycle: NPU firmware, like any embedded component, must be subject to a secure development lifecycle. This includes secure boot, signed firmware updates, and robust integrity checks to prevent modification or loading of malicious firmware, addressing concerns raised by AMD's historical XCL bin loading.
• Proactive Security Audits: Given that NPUs represent a new and complex attack surface, continuous security auditing and vulnerability research into NPU drivers are paramount. This should involve both internal security teams from vendors and external researchers, focusing specifically on DDI functions, escape functions, and the interactions between the NPU driver and the core Windows kernel.
• Least Privilege Principle: Applications leveraging NPU capabilities should operate under the principle of least privilege. Developers should ensure their applications request only the necessary permissions and avoid running NPU-accelerated components with elevated privileges unless absolutely essential.
• Timely Patching and Updates: Users and enterprises must prioritize timely patching and updates for NPU drivers and Windows operating systems. The rapid discovery of kernel vulnerabilities, as demonstrated by the arbitrary increment primitive bypassing 24H2 mitigations, means that keeping systems current is vital to mitigate known exploitation techniques.
• Learning from History: The fact that NPU drivers are inheriting vulnerabilities from their GPU predecessors highlights a failure to learn from past mistakes. Future hardware and driver development must integrate security from the ground up, avoiding the reuse of insecure design patterns or interfaces without a thorough security review.

By addressing these defensive implications, the security posture of Copilot+ PCs and the broader ecosystem of AI-powered computing can be significantly strengthened against the emerging threats posed by NPU driver exploitation.

Key Takeaways

• NPUs are here to stay, but their drivers are a new, vulnerable attack surface: The increasing reliance on local AI via NPUs for privacy, cost, and availability benefits means these devices are critical, but their nascent drivers are ripe for exploitation.
• Inherited GPU driver vulnerabilities plague NPUs: Due to the MCDM being a subset of WDDDM, NPU drivers share interfaces with GPUs, inheriting a decade's worth of known DDI and escape function vulnerabilities.
• Unvalidated user input is a critical flaw: PrivateDriverData and PrivateDriverDataSize in DDI functions, often unvalidated by dxgkrnl.sys, allow attackers to smuggle malicious input directly into kernel-mode NPU drivers, leading to severe vulnerabilities like arbitrary read/write.
• Reference counter manipulation is a viable post-arbitrary-read primitive: The "arbitrary increment" technique, using DuplicateToken to manipulate reference counters in kernel objects, demonstrates a powerful and stable method for achieving privilege escalation on modern Windows versions, including 24H2, even after previous arbitrary write primitives are patched.
• Current Windows kernel mitigations are insufficient: Microsoft's win32k syscall disabling is too blunt for practical NPU-reliant applications, highlighting the need for more granular sandboxing and isolation mechanisms.
• Proactive security research and rigorous validation are essential: NPU drivers require intense scrutiny from both vendors and the security community, with a strong emphasis on comprehensive input validation and secure design from the outset, to prevent future exploitation.

About the Speaker(s)

The research presented in this talk was conducted by a graduate student at the UN Surirk University of Applied Science in collaboration with their supervisor, Gangur, who is also affiliated with the same institution. The graduate student's work focuses on exploring the security of emerging technologies, specifically Windows NPU drivers. Beyond academic pursuits, the speaker shared a personal preference for martial arts, a "heresy" according to their Swiss colleagues. The talk concluded with an invitation for further contact, indicating an ongoing commitment to the field of security research, with plans to publish a blog post detailing the arbitrary increment technique.