Efficient Use-After-Free Prevention with Opportunistic Page-Level Sweeping

Efficient Use-After-Free Prevention with Opportunistic Page-Level Sweeping

Speakers: Chanyoung Park, Researcher

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=c0bn3VBqaaY

Overview

In this insightful talk, Chanyoung Park introduced HUSHVAC, a novel and highly efficient approach to preventing use-after-free (UAF) vulnerabilities in software utilizing manual memory management, predominantly C and C++. The UAF vulnerability remains a critical security threat, allowing attackers to exploit dangling pointers to freed memory, leading to severe consequences such as privilege escalation, arbitrary code execution, or information leakage. Despite decades of research and mitigation efforts, a truly universal and efficient solution has remained elusive, often requiring significant performance trade-offs.

HUSHVAC addresses a fundamental challenge observed in existing UAF prevention strategies: the dilemma between security effectiveness and performance efficiency, particularly for allocation-intensive workloads. Traditional mark-sweep garbage collection paradigms, while effective, often introduce substantial overhead due to heap fragmentation and reduced spatial locality. Conversely, "one-time allocation" models, which never reuse freed memory, offer good performance but are unsustainable for long-running applications due to virtual address space exhaustion.

The core innovation of HUSHVAC lies in its ability to reconcile these conflicting requirements. By leveraging an allocator optimized for fresh memory allocation (FFmalloc) and integrating an opportunistic, page-level mark-sweep engine that reuses virtual address space, HUSHVAC delivers robust UAF prevention with significantly lower performance overhead than prior mark-sweep systems. This talk detailed the intricate design choices and extensive evaluation that position HUSHVAC as a compelling solution for securing modern C/C++ applications without crippling their performance.

Background

▶ Watch: Introduction to HUSHVAC and UAF problem (0:00)

The use-after-free (UAF) vulnerability is a class of memory safety error that occurs when a program attempts to access memory after it has been freed. This typically happens when a program frees a heap chunk but retains a dangling pointer to that memory region. If this freed memory is subsequently reallocated to a new object, the dangling pointer can be misused by an attacker to manipulate the new object's data, control program flow, or leak sensitive information. The central challenge in UAF prevention is reliably determining whether any dangling pointers to a freed memory region still exist before that region is reused.

Existing UAF prevention techniques primarily fall into two categories, both focused on delaying the reuse of freed memory:

1. Mark-Sweep Approaches: Systems like MarkUs and MineSweeper adopt a garbage collection-like paradigm. When memory is freed, it is placed onto a quarantine list. Reuse is only permitted after a memory scan (the "mark" phase) confirms that no dangling pointers to the quarantined memory exist. MarkUs utilizes an existing C++ garbage collector, while MineSweeper performs a linear memory scan. While these systems are effective at preventing UAF, they suffer from significant performance degradation, often exceeding 100% overhead for allocation-intensive benchmarks such as xalancbmk. This slowdown is largely attributed to increased heap fragmentation and reduced spatial locality, as the allocator is forced to fetch memory from distant regions when freed chunks are unavailable.

1. One-Time Allocation Approaches: Examples include Oscar and FFmalloc. These systems take a more extreme stance: they avoid reusing freed heap chunks altogether, instead always allocating from fresh virtual pages. The rationale is that if a chunk is never reused, a dangling pointer to it becomes harmless. FFmalloc, in particular, is highly optimized for this model and has demonstrated surprisingly low overheads, even for allocation-intensive workloads. However, a fundamental limitation of this approach is its unsustainability for long-running applications due to the eventual exhaustion of the virtual address space. Although physical pages can be reclaimed by the operating system, the virtual address space remains consumed, leading to eventual resource depletion.

HUSHVAC's work operates under a specific threat model. It assumes that the program running with HUSHVAC has one or more use-after-free vulnerabilities, and these are the only vulnerabilities an attacker can exploit. Other memory safety violations, such as spatial safety errors (e.g., buffer overflows) or logic bugs, are considered outside HUSHVAC's scope. Furthermore, HUSHVAC itself is assumed to be well-written and free from exploitable vulnerabilities. This threat model is consistent with prior research focused purely on UAF mitigation and prevention.

Key Findings

▶ Watch: Limitations of current Mark-Sweep and One-Time allocation (2:40)

The research behind HUSHVAC yielded several critical findings that informed its design and demonstrated its effectiveness:

• Root Cause of Mark-Sweep Overhead: A primary finding was the identification that the significant performance overhead observed in existing mark-sweep UAF prevention approaches (like MarkUs) for allocation-intensive benchmarks is not inherent to scanning, but rather stems from increased heap fragmentation and reduced spatial locality. Delaying the reuse of individual chunks forces allocators to frequently request fresh memory, leading to scattered allocations and poor cache performance.
• Efficiency of One-Time Allocation, with a Catch: The analysis confirmed that delaying reuse until program termination (the one-time allocation model, as seen in FFmalloc) can indeed be remarkably efficient for certain workloads. However, this efficiency comes at the cost of virtual address space exhaustion, making it unsuitable for long-running applications or services.
• HUSHVAC's Hybrid Success: HUSHVAC successfully demonstrates that it is possible to combine the benefits of both approaches. It achieves efficient UAF prevention by leveraging an allocator optimized for fresh chunk allocation (like FFmalloc) and integrating an opportunistic page-level sweeping mechanism that reuses virtual pages rather than individual chunks, thereby mitigating fragmentation issues.
• Cruciality of Comprehensive Scanning: The research highlighted that robust UAF prevention necessitates comprehensive memory scanning. This means not just inspecting the heap, but also the stack and any other memory pages obtained directly by the application via system calls (e.g., mmap), as dangling pointers can reside in unexpected locations.
• Performance Breakthrough: HUSHVAC achieved the lowest performance overhead among mark-sweep UAF prevention approaches. For instance, on the SPEC CPU 2006 benchmark suite, it incurred an average geometric mean overhead of just 4.7%, dramatically outperforming MarkUs' 11.4%. Crucially, for the allocation-intensive xalancbmk benchmark, HUSHVAC's overhead was only 35%, compared to MarkUs' staggering 110%.
• VMA Management Efficiency: By retaining virtual page mappings and detaching only physical pages (using mmap with MAP_FIXED), HUSHVAC effectively bounds the number of Virtual Memory Area (VMA) structures, preventing the "VMA explosion" issue that can plague systems that frequently unmap and remap virtual pages.
• Opportunistic Triggering Reduces Pauses: The design choice to opportunistically trigger the mark-sweep procedure only when the application is not actively allocating significantly reduces stop-the-world pauses, leading to a much smoother application experience.

Technical Deep Dive

▶ Watch: HUSHVAC's core technical approach and design overview (4:00)

HUSHVAC's design is built upon the insight that modern systems can efficiently perform synchronous marking and concurrent sweeping with minimal performance intervention, combined with the observation that heap allocators can be optimized for fresh chunk allocation. The system integrates an opportunistic page-level mark-sweep engine with FFmalloc as its underlying allocator, incorporating five key design choices:

1. FFmalloc as the Underlying Allocator: HUSHVAC starts by leveraging FFmalloc, an allocator specifically optimized for allocating fresh memory chunks. This means that, by default, HUSHVAC prioritizes allocating from new, unused memory rather than immediately reusing freed chunks. This decision is inspired by FFmalloc's demonstrated efficiency, indicating that fresh allocations do not inherently incur high overhead.
2. Page-Level Reuse of Virtual Address Space: Instead of reusing individual freed chunks, HUSHVAC primarily reuses freed virtual pages. A critical enabler is the Linux kernel's capability to detach physical pages from virtual pages using mmap with MAP_FIXED and MAP_NOREPLACE. This removes the backing physical memory without unmapping the virtual page itself. This mechanism prevents the "over-splitting VMA structure" issue often associated with frequent mmap invocations and allows HUSHVAC to reuse virtual pages without the overhead of re-mapping them, contributing significantly to its efficiency.
3. Opportunistic Mark-Sweep Procedure: To minimize interference with application performance, HUSHVAC performs its mark-sweep procedure to reclaim virtual address space only when the application is not actively allocating heap chunks. It continuously monitors the frequency of heap allocations, triggering the mark-sweep only when this frequency drops below an empirically set threshold (e.g., 1.1x of the average). The minimal metadata cost of virtual pages in the quarantine list (approximately 16 bytes per page) allows HUSHVAC to quarantine many pages and delay the sweep until an opportune moment.
4. Careful Sub-Page Reuse: Strict page-level sweeping could lead to memory waste if a few small live chunks prevent an entire page from being reused. To mitigate this, HUSHVAC selectively and carefully reuses some chunks within a partially freed page before the entire page becomes eligible for page-level sweeping. This sub-page reuse mechanism is designed to maintain spatial locality and ensure that individual chunks are efficiently utilized without nullifying the benefits of page-level sweeping.
5. Comprehensive Memory Scanning: Unlike some existing mark-sweep approaches that focus solely on the heap, HUSHVAC performs a more comprehensive memory scan. It examines not only the heap but also the stack and any memory pages obtained directly by the application via system calls (e.g., mmap) without using a heap allocator. This conservative approach is vital for discovering all potential dangling pointers, including those hidden in non-heap regions, which was validated by a Proof-of-Concept demonstrating unsafe reuse when a pointer was stored in an anonymous mmaped page.

Detailed Components

• B. Mark-Sweep for Virtual Pages: HUSHVAC's mark-sweep engine tracks the freed status of each chunk within a virtual page using a bitmap. When the last live chunk in a 4-KiB virtual page is freed, HUSHVAC immediately pushes that virtual page to a quarantine list. Crucially, it detaches the corresponding physical page using mmap with MAP_FIXED and MAP_NOREPLACE, which removes the physical memory without unmapping the virtual address space. This avoids VMA fragmentation and reduces physical memory usage while retaining the virtual page mapping for faster reuse later.
• C. Two-Staged Mark Phase: To minimize the application's stop-the-world time, HUSHVAC employs a two-staged mark phase:

1. Concurrent Mark: This phase runs concurrently with the application. It determines the set of pages to scan from /proc/self/maps, clears dirty bits, and then traverses pages, treating 8-byte values as pointers if they fall within the heap range, setting mark bits for referenced chunks.
2. Synchronous Mark: This phase briefly pauses the application. It then rescans only the dirty pages (those modified since the last clearance of their dirty bit) and any pages whose mark bits were set during the concurrent phase but whose dirty bit was 0. This ensures the mark map is sound, accurately reflecting all reachable pointers at a specific point in time, without prolonged application pauses. The application resumes immediately after this brief synchronous phase.

• D. Page-Level Sweeping: The sweep phase runs concurrently with the application, reclaiming virtual pages from the quarantine list that are safe to reuse. The sweeping thread checks the mark map for each quarantined virtual page. If all mark bits for a page are cleared (meaning no dangling pointers), the page is moved to a reuse batch list. When the heap allocator needs more memory, it first checks this reuse batch list before invoking mmap for fresh pages. This amortizes the cost of decision-making and reduces mmap calls. Again, only physical memory is detached; the virtual page remains mapped.
• E. Opportunistically Triggering Mark-Sweep Procedure: HUSHVAC avoids triggering the synchronous mark phase when the application is actively allocating. It continuously monitors the frequency of heap allocations. The mark-sweep procedure is only triggered when the allocation frequency drops below a certain threshold (empirically set to 1.1x of the average). This opportunistic approach is feasible because the metadata for quarantined virtual pages is very small (approx. 16 bytes per page), allowing many pages to remain in quarantine without significant memory overhead.
• F. Comprehensive Scanning of the Memory Space: HUSHVAC's mark-sweep procedure scans the entire memory space, excluding only allocator metadata. This includes the stack, the heap, and any memory regions obtained directly by the application via mmap system calls. This conservative approach is critical because applications might store heap pointers in unexpected locations. The necessity was demonstrated by a Proof-of-Concept (PoC) using a modified HardsHeap fuzzer, which triggered a UAF issue in an existing scheme by placing a dangling pointer in an mmaped anonymous page (Figure 3 in the original paper).
• G. Sub-Page Reuse: To counter potential memory waste from strict page-level sweeping, HUSHVAC implements sub-page reuse. It does not maintain a separate quarantine list for individual chunks. Instead, it builds a sub-page reuse batch list from pages containing at least one safely reusable chunk. When a new allocation request comes, HUSHVAC prioritizes retrieving chunks from this batch list. This design ensures that chunks allocated via sub-page reuse maintain good spatial locality and that the sub-page reuse mechanism does not prevent pages from eventually becoming fully freed and eligible for page-level sweeping.

Implementation Details and Experimental Setup

HUSHVAC was implemented using FFmalloc as the underlying allocator. The evaluation system ran Ubuntu 18.04 with Linux kernel 5.4.0-150-generic, on a machine equipped with an AMD Ryzen 5 2600 processor and 32 GB of main memory. HUSHVAC employs one reclaimer thread and ten scanner threads per process. For comparison, HUSHVAC was evaluated against MarkUs 11 and FFmalloc 36. The evaluation used a diverse set of benchmarks:

• SPEC CPU 2006: 19 single-threaded C/C++ workloads.
• SPEC CPU 2017: 12 multi-threaded C/C++ workloads.
• BBench 2.0 on Firefox: A real-world browser rendering benchmark.
• Mimalloc-bench: Microbenchmarks and allocation-intensive application workloads.
• PARSEC 3.0: 12 multi-threaded C/C++ workloads.

Measurements included execution time (using time utility) and maximum resident set size (MaxRSS) for memory overhead.

Demo / Proof of Concept

▶ Watch: Design: Opportunistic sweeping and careful sub-page reuse (5:00)

The talk presented compelling evidence of HUSHVAC's effectiveness through several rigorous evaluations, demonstrating its ability to prevent both synthetic and real-world UAF exploits.

Effectiveness Evaluation:

• HardsHeap Fuzzer: HUSHVAC was subjected to the HardsHeap 38 fuzzer, specifically designed to uncover vulnerabilities in heap allocators, for over 20 continuous hours. HUSHVAC successfully prevented all use-after-free examples generated by the fuzzer, and HardsHeap reported no working UAF vulnerabilities against HUSHVAC.
• NIST Juliet Test Suite: HUSHVAC was tested against the NIST Juliet Test Suite 20, a comprehensive collection of C/C++ test cases including various UAF scenarios. HUSHVAC did not abort any test cases, confirming its ability to prevent UAF without crashing the application.
• Real-World CVEs: The system was evaluated against four public CVE-assigned UAF exploits found in three different versions of PHP. As detailed in the original paper's Table I, HUSHVAC successfully prevented all four exploits from achieving their malicious goals, such as arbitrary code execution or memory disclosure. For instance, CVE-2016-5773, a double-free vulnerability, was not only prevented but the double-free condition itself was detected. The fundamental reason for this success is HUSHVAC's guarantee that a vulnerable heap chunk, pointed to by a dangling pointer, is simply not reused, thereby nullifying the exploit's attempt to manipulate program behavior.

Proof of Concept for Comprehensive Scanning:

A critical aspect of HUSHVAC's design is its comprehensive memory scanning, which extends beyond the heap to include the stack and any memory regions obtained directly via mmap system calls. The necessity of this comprehensive approach was validated by generating a specific Proof-of-Concept (PoC). This PoC involved a modified HardsHeap fuzzer that deliberately placed a dangling pointer in an mmaped anonymous page. This scenario successfully triggered a UAF issue in existing schemes that did not perform such comprehensive scanning, demonstrating that solely scanning the heap is insufficient for robust UAF prevention and underscoring the importance of HUSHVAC's design choice.

Defensive Implications

▶ Watch: Design: Comprehensive memory scanning for all pointers (5:35)

The introduction of HUSHVAC has several significant implications for software defenders and developers working with C/C++ applications:

• Practical UAF Prevention for C/C++: HUSHVAC offers a highly practical and efficient solution for preventing UAF vulnerabilities in manually managed memory environments. For organizations struggling with the trade-offs between security and performance in their C/C++ codebase, HUSHVAC provides a viable path to robust UAF protection without the crippling overhead seen in previous mark-sweep approaches.
• Prioritize Adoption for Allocation-Intensive Workloads: Applications with high allocation and deallocation rates, which historically suffered the most from mark-sweep UAF prevention (e.g., xalancbmk), can now benefit from strong UAF guarantees with a much more acceptable performance impact. Defenders should consider integrating HUSHVAC, or similar principles, into their development and deployment pipelines for such critical applications.
• Beyond Heap-Only Scanning: HUSHVAC's comprehensive scanning, including the stack and mmaped anonymous pages, highlights a crucial lesson for defenders: dangling pointers can reside in unexpected memory regions. Security audits and custom UAF prevention mechanisms should not assume that pointers are exclusively stored within the heap.
• Understanding Performance Trade-offs: While HUSHVAC dramatically improves execution time overhead compared to MarkUs, it does incur a higher memory usage overhead (e.g., 59.8% for SPEC CPU 2006 compared to MarkUs' 25.1%). Defenders must evaluate this trade-off based on their application's specific resource constraints and security requirements. In memory-constrained environments, further optimization or alternative strategies might be necessary.
• Architectural Resilience: HUSHVAC's approach of reusing virtual pages while detaching physical memory, and its opportunistic mark-sweep, contributes to a more resilient memory management architecture. This design mitigates issues like VMA explosion, which can destabilize long-running systems. Defenders can draw inspiration from these architectural choices for building more robust custom memory allocators or security mechanisms.
• Continuous Improvement: The identified limitations, such as potential negative impacts of sub-page reuse on locality for specific workloads and higher overhead on PARSEC 3.0 compared to FFmalloc, suggest areas for future research and refinement. Defenders should stay updated on advancements in this area, as UAF prevention remains an active field of study.

Ultimately, HUSHVAC empowers defenders with a powerful tool to significantly enhance the security posture of their C/C++ applications against a persistent and dangerous class of vulnerabilities, making UAF prevention a more practical reality for real-world deployments.

Key Takeaways

• HUSHVAC is a novel UAF prevention system that combines the efficiency of allocators optimized for fresh chunk allocation (like FFmalloc) with an opportunistic, page-level mark-sweep mechanism for virtual address space reuse.
• It successfully addresses the major performance bottleneck of prior mark-sweep UAF prevention approaches on allocation-intensive workloads, specifically mitigating heap fragmentation and reduced spatial locality.
• HUSHVAC achieves remarkably low execution time overheads, averaging 4.7% on SPEC CPU 2006 and 4.1% on SPEC CPU 2017, significantly outperforming MarkUs (e.g., 35% vs. 110% on xalancbmk).
• Comprehensive memory scanning across the heap, stack, and mmaped anonymous pages is crucial for discovering all potential dangling pointers and ensuring robust UAF prevention.
• HUSHVAC effectively prevents real-world UAF exploits, successfully mitigating four public CVE-assigned UAFs in PHP and passing over 20 hours of continuous fuzzing with the HardsHeap fuzzer.
• Its design leverages Linux kernel features to detach physical pages while retaining virtual page mappings, preventing VMA explosion and reducing mmap system call overheads.

About the Speaker(s)

Chanyoung Park is a researcher in the field of memory safety, with a particular focus on addressing challenging vulnerabilities such as use-after-free (UAF) in systems programming languages. Their work, as exemplified by HUSHVAC, demonstrates expertise in designing and evaluating novel memory management techniques that balance strong security guarantees with high performance for real-world applications. The presentation showcases a deep understanding of allocator design, operating system interactions (like virtual memory management), and the practical implications of memory safety errors.