SENSE: Enhancing Microarchitectural Awareness for TEEs via Subscription-Based Notification

SENSE: Enhancing Microarchitectural Awareness for TEEs via Subscription-Based Notification

Speakers: Fan Sang

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=_-SNOD-frmA

Overview

Trusted Execution Environments (TEEs) are critical for securing sensitive computations, but they face a persistent and formidable adversary in side-channel attacks (SCAs). Despite continuous advancements in defense mechanisms, effectively mitigating these attacks remains a significant challenge. Current detection-based approaches often falter against sophisticated, stealthy attacks that leak small portions of secrets over multiple executions, as they primarily rely on observing abnormal victim performance characteristics. The core problem lies in the inherent information asymmetry: attackers can gather a wealth of microarchitectural signals, even from privileged kernel space, while the victim inside the TEE remains largely blind.

This talk introduces SENSE, a novel hardware-software co-design solution that directly addresses this fundamental limitation. SENSE empowers userspace TEEs with unprecedented microarchitectural awareness by enabling them to subscribe to fine-grained microarchitectural events. This subscription-based notification mechanism allows software within TEEs to contextualize ongoing microarchitectural states and proactively defend against SCAs. By providing a trusted and direct source of microarchitectural information, SENSE transforms a traditional side channel into a high-fidelity, trustworthy direct channel accessible to TEEs at runtime.

The presentation demonstrates SENSE's capability by applying it to defeat state-of-the-art cache-based side-channel attacks. A comprehensive security analysis ensures that SENSE does not introduce new attack surfaces or leak more information than a system without it. Prototyped on a gem5-based emulator, SENSE proves to be secure, effectively defeats cache SCAs, and incurs a negligible performance overhead of just 1.2% under benign situations, making it a practical and impactful solution for enhancing TEE security.

Background

▶ Watch: Introduction to SENSE and problem statement (0:00)

Microarchitecture refers to the underlying hardware components that are not directly exposed to software, typically abstracted by the Instruction Set Architecture (ISA). These components, such as CPU caches, Translation Lookaside Buffers (TLBs), and instruction pipelines, operate transparently from the software layer. However, their internal operations generate microarchitectural events related to instruction executions and memory operations, including instruction fetch, dispatch, retirement, and crucial memory events like CPU cache hits and misses, TLB hits and misses, and Page Table Walks. While not directly accessible, these events can be indirectly inferred by carefully crafted software execution, leading to information leakage.

Attackers exploit these shared hardware components to infer the behavior of other programs, forming the theoretical basis of side-channel attacks (SCAs). Among the diverse array of SCAs, cache-based attacks are particularly prominent and pose significant risks to secure computing across various platforms, including Intel SGX, AMD SEV, and ARM TrustZone. These attacks can disclose both fine-grained and coarse-grained private data, such as bypassing ASLR, deducing keystroke patterns, and exposing sensitive cryptographic keys for algorithms like RSA and AES. Common techniques include Prime+Probe, which monitors cache set access patterns, and Flush+Reload, which evicts shared target cachelines to observe reloads.

To counter these attacks, researchers have proposed various detection-based countermeasures. These solutions aim to identify ongoing attacks by monitoring program performance characteristics, such as cache miss rates or interrupt counts, to detect suspicious processes. However, these strategies face several critical challenges. Firstly, TEEs lack a trusted source of microarchitectural data. While Hardware Performance Counters (HPCs) offer direct information, their reliance on an untrusted Operating System (OS) or hypervisor mediation makes them unsuitable for TEEs. Secondly, the available data is often of low quality, being statistical and noisy, leading to delayed and imprecise attack detection. This allows stealthy attacks to leak small amounts of information over multiple executions, making them difficult to differentiate from benign activity. Thirdly, these techniques are inflexible; without detailed microarchitectural information, victims have limited contextual awareness and often resort to coarse-grained decisions like terminating or retrying workloads. Finally, many existing techniques are platform-specific, relying on particular hardware features like Intel TSX, which may be deprecated, limiting their extensibility and long-term applicability. It is ironic that TEEs, designed to shield programs from external inferences, inadvertently block them from proactively using detection-based SCA countermeasures, while attackers, unconstrained, are free to gather all kinds of microarchitectural signals, even from kernel space.

Beyond detection-based defenses, isolation-based defenses for cache side channels exist. These include partitioning-based approaches that propose new cache architectures to allocate resources exclusively to protected domains. While effective against many SCAs, they often lead to cache underutilization and are inflexible due to static assignments or reliance on an untrusted OS for page coloring. Randomization-based defenses, employing randomized mapping tables, aim to diminish the bandwidth of cache attacks rather than eradicate them completely, often incurring significant performance overhead and still allowing eviction operations. SENSE fundamentally eliminates the unreliability and inflexibility of these existing approaches by providing timely, accurate, and flexible notifications directly to userspace TEEs.

Detection-based defenses can be further categorized into signature-based and anomaly-based methods. Signature-based methods, such as NIGHTS-WATCH and HexPADS, use HPCs and machine learning models to detect specific attack patterns like Prime+Probe. Anomaly-based methods, including CacheShield and SpyDetector, monitor LLC cache misses or L3/L1 data cache activity to identify deviations from normal behavior. While these works leverage microarchitectural events as feature vectors, they are susceptible to interference from system loads, leading to high false positives and false negatives. SENSE, in contrast, detects potentially malicious behaviors at their exact locations and promptly notifies userspace TEEs, preventing the delayed awareness inherent in signature and anomaly-based strategies, and empowering victims with flexible responses.

Key Findings

▶ Watch: SENSE: A novel hardware-software co-design solution (0:50)

SENSE represents a significant advancement in TEE security, offering a novel paradigm for combating microarchitectural side-channel attacks. The key findings and contributions of this work are multifaceted:

• New Paradigm for Microarchitectural Awareness: SENSE proposes an innovative approach that transforms traditionally exploitable side channels into a high-fidelity, trustworthy direct channel. By actively exposing microarchitectural states through a subscription-based notification mechanism, it empowers victims within TEEs to access critical hardware information at runtime, directly addressing the long-standing problem of information asymmetry.
• Comprehensive, Platform-Agnostic Design: The work provides a detailed and architecture-agnostic design for the SENSE architecture, specifying the necessary processor extensions and cache components required to effectively mitigate cache-based SCAs in TEEs. This ensures broad applicability across different hardware platforms.
• Rigorous Security Analysis: An in-depth security analysis of the SENSE interface and potential attack scenarios was conducted. This analysis demonstrates that SENSE is designed to not introduce new attack surfaces or inadvertently empower attackers beyond the capabilities they already possess in a system without SENSE.
• Proven Effectiveness Against State-of-the-Art SCAs: Evaluations confirm that SENSE successfully defeats sophisticated cache-based side-channel attacks, including Prime+Probe and Flush+Reload. It achieves this by eliminating useful microarchitectural traces that attackers typically rely on to infer sensitive information.
• Negligible Performance Overhead: A crucial finding is SENSE's minimal performance impact. Under benign operating conditions, it incurs an average overhead of only 1.2%, making it practical and viable for real-world TEE workloads without unduly impacting application performance.
• Versatility and Extensibility: SENSE's subscribe-notify-act mechanism offers substantial versatility. It supports dynamic adaptation of defenses, allows for the implementation of custom event handlers, enables on-demand loading of secure libraries, and facilitates the verification of OS contracts, extending its utility significantly beyond basic SCA detection.
• Open-Source Availability: To foster community contributions and accelerate the adoption of a more transparent microarchitectural paradigm, SENSE has been released as an open-source project.

Technical Deep Dive

▶ Watch: Background: Challenges of current SCA defenses in TEEs (2:00)

SENSE is designed as a generic mechanism to feed microarchitectural events directly to userspace TEEs. Its core is an architecture-agnostic processor extension featuring a new CPU SENSE mode. This mode directly notifies a running thread inside a TEE about the microarchitectural events it has subscribed to. Only threads operating under TEEs can enter SENSE mode and subscribe to relevant events, allowing proactive defense by enforcing security invariants, such as pinning secret-dependent cache entries.

SENSE is structured into three main modules:

1. The Subscription Module (SM): This module enables TEE threads to subscribe to cache-related microarchitectural events, such as cache evictions, and monitors their occurrence.
2. The Notification Module (NM): This module manages the CPU SENSE mode and provides the architectural interface for delivering these microarchitectural events to userspace TEEs for handling.
3. The Action Module (AM): This module allows the occurred events to be handled by a userspace event handler within the TEE.

The benefits of SENSE are derived from this architecture: it provides a trusted native information source by bypassing untrusted privileged software, offers prompt and precise notifications of events at the exact subscribed cache entry level, ensures versatility and extensibility through dynamic adaptation and custom handlers, maintains compatibility and generality with existing defenses and architectures (x86, ARM, RISC-V), and is designed for security without introducing new attack surfaces.

SENSE targets platforms with a TEE component (like Intel SGX, AMD SEV, Intel TDX) and a modern cache architecture, assuming a standard hierarchy with core-exclusive L1/L2 and shared LLC caches. The threat model assumes an untrusted OS kernel and hypervisor, with adversaries capable of executing access-based and conflict-based cache SCAs from all privilege levels (except the highest containing trusted software).

The operational flow of SENSE involves five key steps:

1. Initialization (NM): An enclave thread enters SENSE mode at the start of a security-critical block. The address of the event handler trampoline within the enclave is registered with the CPU.
2. Subscription (SM): Cache microarchitectural events, such as evictions for memory resources within the enclave, are subscribed under SENSE monitoring.
3. Notification (SM to NM): If a subscribed event occurs (e.g., a monitored cache entry is evicted) during instruction execution, SENSE mode pauses. Microarchitectural states related to subscribed events are cleared (e.g., flushing the cache). Control flow is then transferred by the CPU to the registered event handler trampoline inside the enclave, supplying detailed microarchitectural information. This transition is enforced by the CPU without changing privilege levels.
4. Action (NM to AM): The SENSE handler saves the interrupted context, reenters SENSE mode, handles the event, and restores the saved context before returning. Nested handling is supported.
5. Resumption: The enclave thread resumes execution. All subscriptions are canceled when the enclave thread exits SENSE mode.

Implementation Details

The SENSE prototype is implemented on a cycle-accurate gem5-based x86 emulator with an out-of-order CPU model and the default Classical Memory System.

A. Subscription Module (SM)

The SM extends the existing cache architecture for precise event location and prompt notification.

• Precise Subscription: Users can subscribe to events at the exact cache entry level. Each cache entry is extended with two SENSE status bits: an SS bit (monitored) and an SS_FAULT bit (monitored entry evicted).
• Prompt Delivery: An information channel informs the NM immediately. This is achieved by embedding a notification flag in existing memory responses sent back to the CPU Load-Store Unit (LSU).

The gem5 implementation involves:

1. Adding SS and SS_FAULT bits to each cache entry.
2. Initializing cache event subscriptions at the start of the enclave process. For eviction events, relevant cache entries are prefetched into memory, and their SS bits are turned on using a PREFETCH_SS memory request flag.
3. Marking cache entries: When PREFETCH_SS is set, the SS bits of prefetched cachelines are turned on, adding them to the watch set.
4. Signaling subscribed eviction: If a watch set member is evicted while the enclave thread is in SENSE mode, its SS_FAULT bit is set.
5. Promptly informing the NM: If the SS_FAULT bit is set in the memory request response, the cache flushes all entries to eliminate microarchitectural traces, adds an SS_FAULT flag to the response, and the CPU enters NM to trigger a SENSE notification.

B. Notification Module (NM)

The NM manages the CPU SENSE mode and introduces new ISA instructions:

• New ISA Instructions:
• ssbegin addr: Starts SENSE mode, registers addr as the trampoline.
• ssend: Terminates SENSE mode.
• sstramp addr: Pushes RIP and transfers control to addr after preparation.
• sssub addr type: Subscribes to an event of type for the resource at addr.
• ssunsub addr type: Unsubscribes from an event of type for the resource at addr.
• SENSE Block and Event Notification: A sequence of enclave instructions under SENSE mode is a SENSE block. If a subscribed event occurs or an instruction prevents SENSE from tracking events (e.g., SYSCALL), the enclave thread is notified via a SENSE Fault (#SS), a dedicated hardware exception using an unused interrupt/exception vector (e.g., 20 under x86).
• SENSE Fault Control Logic: Upon an #SS, the CPU pauses SENSE mode (unsetting CR.SS_MODE), validates CR_SS_TRAMP, extracts event information, pushes it to the enclave stack, and transfers control to the address in CR_SS_TRAMP.
• Modes of Monitoring: SENSE supports broadcast (default, for multiple enclaves monitoring the same event) and exclusive monitoring (first-come, first-served, aborting subsequent subscriptions) for security-critical applications.
• Implementation: Integrates extended control registers (CR.SS_MODE, CR.SS_TRAMP) and modifies interrupt controller logic. Software wrapper functions like _ssbegin(), _ssend(), _sswatch(), and _ssunwatch() simplify usage.

C. Action Module (AM)

The AM defines how events are handled. The event handler saves context, reenters SENSE mode, handles the event, and restores context. SENSE provides default handlers:

• ABORT: Pessimistically aborts the enclave process on event.
• INVARIANT: Preserves a safety property, e.g., refetches evicted cachelines to pin them in the cache.
• THRESHOLD: Maintains a thread-local counter, terminating the process if event count exceeds a user-defined threshold.

Developers can also design and register custom event handlers.

D. Compatibility and General Hardening

SENSE is backward compatible, only providing notifications when processes run under TEEs. It's synergistically compatible with existing partitioning and randomization-based cache SCA defenses. Architecturally, SENSE aims for ISA abstractions generic enough for x86, ARM, and RISC-V. Beyond SCAs, SENSE enables on-demand loading of secure libraries (switching to a constant-time version upon attack detection) and verifying OS contracts (e.g., enforcing cache coloring).

Demo / Proof of Concept

▶ Watch: SENSE's advantage over existing cache side-channel defenses (4:00)

While the talk did not feature a live demonstration, the authors presented a comprehensive evaluation of their SENSE prototype, implemented on a cycle-accurate gem5-based x86 emulator with an out-of-order (O3) CPU model running at 4 GHz with Linux kernel version 4.8.1. Experiments were conducted using a simulated 4-core Intel i7-6700K CPU and 64 GiB of RAM, with average values reported from 10 runs.

Security Evaluation: Hardening AES T-Tables

SENSE's ability to close timing-based cache SCAs was evaluated using a Prime+Probe attack against a vulnerable AES T-table implementation.

• Without SENSE: The evaluation showed a clear cache hit pattern, revealing information about the T-table entries accessed by the victim.
• With SENSE (INVARIANT handler): When the INVARIANT handler was deployed, it actively refetched evicted cachelines, effectively pinning them in the cache. This demonstrated that no useful microarchitectural traces were left for the attacker to infer secret key bits, effectively defeating the attack. Similar efficacy was observed against Flush+Reload attacks.

Attacker's Exploitation of SENSE (Efficiency Benefits)

The evaluation also assessed whether SENSE could inadvertently provide efficiency benefits to a malicious TEE.

• Detecting Single Cache Access: SENSE notifications for a single cache access were approximately 8 times faster (taking 576,000 emulated CPU cycles compared to 4,649,000 cycles for traditional timing channel probing). This allows a malicious TEE to perform significantly more probing attempts within the same timeframe.
• Reduced False Positive Rate: SENSE notifications exhibited a reduced false positive rate compared to conventional timing channels, indicating improved accuracy due to the immediate hardware callback mechanism.
• Stronger Prime+Probe Signal: Assisted by faster execution speed and lower false positive rates, the signal strength for Prime+Probe attacks was demonstrably stronger when utilizing SENSE from a malicious TEE. However, it's crucial to note that SENSE does not expose new information; rather, it makes existing observable information more efficiently accessible.

Performance Evaluation

The performance overhead of each SENSE module was evaluated using PolyBenchC benchmarks, treating small matrix functions as security-critical sections within a TEE.

• Subscription Module (SM) Overhead: The SM, responsible for prefetching security-critical memory objects and marking cache entries, incurred an average overhead of 1.2% of the overall execution time, which is considered negligible.
• Notification Module (NM) Overhead: This overhead, stemming from extra CPU control logic, state cleaning, and control transfer, was found to be imperceptible for realistic cache sizes (up to 256kB). However, for extremely small cache sizes (e.g., 16kB) designed to induce high contention, the overhead increased due to more frequent evictions and notifications.
• Action Module (AM) Overhead: The AM overhead depends heavily on the complexity of the event handler. Using dummy handlers with busy-waiting loops, it was shown that AM overhead can dominate SENSE's total overhead in extreme cases with small cache sizes and complex handlers. However, for practical use cases with minimal enclave programs and simpler default handlers, the AM overhead was negligible.
• Performance of SENSE under Attack: When AES encryption performance with and without SENSE protection (using the INVARIANT handler) was compared under attack, it was observed that without attack, AES encryption was fast. Under attack, with T-table entries intentionally evicted, performance decreased as the number of evictions increased. This degradation was primarily attributed to SENSE notifications and the INVARIANT handler refetching evicted entries to maintain security.

Comparing with Intel TSX

SENSE was compared with Intel TSX, a hardware transactional memory feature sometimes used for SCA mitigation. TSX requires partitioning programs into small sections that fit within a transaction and restarts the entire transaction upon abort, making it inflexible. SENSE, in contrast, allows execution to resume at the interrupted location. The evaluation showed that SENSE's performance decreased more slowly than TSX under increasing evictions, indicating SENSE's greater scalability and efficiency for mitigating cache-based SCAs due to its ability to handle events in place.

On-demand Loading of Cryptographic Functions

A use case demonstrating SENSE's ability to dynamically switch from an optimized but potentially vulnerable library to a secure version upon attack detection was presented. The function switching overhead was found to be negligible, around 1.46% of encryption operations, and was paid only once. This allows for dynamic security adaptation, where the overhead of a secure version is incurred only when necessary.

Verification of Cache Coloring

SENSE's capability to verify OS contracts, specifically cache coloring, was also evaluated. The event handler could verify if the thread evicting a monitored cacheline matched its own thread identity. The performance overhead for this verification was found to be negligible.

Hardware and Memory Overhead

• Logic Overhead: An RTL implementation of SENSE indicated a minor logic overhead, estimated at 0.1% of an 8-core Xeon Nehalem processor (2.3 billion transistors).
• Memory Overhead: This includes additional register components per logical core and two bits per cache entry for tracking SENSE status. This overhead could potentially be eliminated if unused reserved registers and bits are available in modern processors.

Defensive Implications

▶ Watch: Key technical approach and methodology of SENSE (6:00)

SENSE provides TEE defenders with a potent new tool to proactively address microarchitectural side-channel attacks, moving beyond reactive detection. Here are the key defensive implications:

• Proactive, Fine-Grained SCA Detection: Defenders should leverage SENSE to gain direct, trusted, and precise microarchitectural information. This allows for the immediate detection of suspicious cache eviction patterns or other subscribed events, significantly reducing the attacker's window of opportunity.
• Customizable Security Policies: The flexibility of the Action Module (AM) allows developers to design and register custom event handlers. This means defenses can be tailored to specific application vulnerabilities and security requirements, moving beyond generic termination or retries. For instance, a custom handler could implement application-specific obfuscation or data re-randomization upon detecting an attack.
• Critical Data Pinning: The INVARIANT handler offers a powerful mechanism to ensure the cache residency of security-critical data, such as cryptographic T-tables or keys. By automatically refetching evicted cachelines, it effectively "pins" these secrets in the cache, rendering Prime+Probe and Flush+Reload attacks ineffective against them.
• Dynamic Security Adaptation: SENSE enables dynamic security. Defenders can initially run applications with performance-optimized but potentially vulnerable libraries. Upon detection of malicious microarchitectural activity (e.g., excessive evictions), SENSE can trigger the on-demand loading and execution of a more secure, constant-time version of the library, paying the performance overhead only when security is actively threatened.
• OS/Hypervisor Contract Verification: SENSE empowers TEEs to verify whether the untrusted OS or hypervisor is honoring its security-critical contracts, such as enforcing cache coloring or resource isolation. If a security-critical cacheline is evicted by a thread not assigned to its designated color, the SENSE handler can detect this breach of contract and take appropriate protective actions, enhancing the overall trustworthiness of the system.
• Synergistic Defense Integration: SENSE is compatible with existing isolation-based defenses (like cache partitioning or randomization). Defenders can integrate SENSE to further enhance security, potentially reducing the hardware overhead of isolation by sharing tracking bits, or to detect attacks that might still bypass coarse-grained isolation.
• Awareness of Shared Monitoring Caveats: Defenders must be aware of the security caveat when both an attacker and victim TEE monitor a shared data region and map to the same cacheline, especially concerning the broadcast monitoring mode. In such scenarios, an attacker might gain an "eviction quota" if their handler refetches while the victim's handler aborts. To mitigate this, exclusive monitoring should be employed for highly sensitive shared resources, or future hardware could integrate enclave identity into monitored cache entries, albeit with increased overhead.
• Focus on Stateful SCAs: While SENSE is versatile, its most practical application is for stateful side channels like TLBs and caches, where the event notification rate allows software to react effectively. For very high-frequency attacks (e.g., port contention), the notification rate might overwhelm software handlers, requiring different mitigation strategies.

Key Takeaways

• SENSE is a novel hardware-software co-design that fundamentally addresses the lack of direct, trusted microarchitectural awareness within Trusted Execution Environments (TEEs).
• It introduces a subscription-based notification mechanism, allowing TEEs to receive prompt and precise information about fine-grained microarchitectural events, transforming side channels into a trustworthy direct channel.
• The system effectively mitigates state-of-the-art cache-based side-channel attacks (Prime+Probe, Flush+Reload) by eliminating useful microarchitectural traces for attackers, with a negligible performance overhead of only 1.2% under benign conditions.
• SENSE offers significant versatility and extensibility, enabling dynamic security adaptations like on-demand loading of secure libraries, verification of OS contracts, and customizable response mechanisms via userspace event handlers (ABORT, INVARIANT, THRESHOLD).
• Designed with security in mind, SENSE does not introduce new attack surfaces or inadvertently empower attackers, although specific configurations (e.g., broadcast monitoring of shared data) require careful consideration.
• The architecture is platform-agnostic, compatible with existing TEEs (Intel SGX, AMD SEV, Intel TDX), and aims for general applicability across diverse processor architectures like x86, ARM, and RISC-V.

About the Speaker(s)

Fan Sang is the presenter of this work, SENSE: Enhancing Microarchitectural Awareness for TEEs via Subscription-Based Notification, at the NDSS Symposium. The talk highlights their contributions to the field of Trusted Execution Environments and side-channel attack mitigation through innovative hardware-software co-design.