TrustSketch: Trustworthy Sketch-based Telemetry on Cloud Hosts
Zhuo Cheng
Network and Distributed System Security (NDSS) Symposium 2024 · Day 3 · Network Infrastructure
Overview
In the dynamic landscape of cloud computing, network telemetry forms the backbone of critical management functions such as anomaly detection, precise billing, and efficient traffic engineering. While traditionally performed on specialized network hardware, the shift towards software-based monitoring on end-host hypervisors offers unparalleled flexibility and ease of deployment. At the forefront of this evolution are sketches, approximate data structures renowned for their ability to provide high-fidelity network statistics—like identifying heavy hitters, counting distinct flows, and measuring entropy—with a remarkably low resource footprint. These compact structures, exemplified by tools like NitroSketch in Intel DPDK, are increasingly integrated into virtual network stacks.

Key moments
- 0:00 Introduction: Vulnerability of sketch-based cloud telemetry
- 1:25 Defining essential requirements for a trustworthy sketch framework
- 2:00 Background: Cloud telemetry and Count-Min sketch basics
- 2:55 Understanding the threat model and attacker capabilities
- 3:35 Illustrative examples of integrity attacks on sketches
- 4:30 Formal security and practical system requirements for TrustSketch
TrustSketch: Trustworthy Sketch-based Telemetry on Cloud Hosts
Speakers: Zhuo Cheng
Conference: NDSS Symposium
YouTube: https://www.youtube.com/watch?v=nya1P1gIlVQ
Overview
In the dynamic landscape of cloud computing, network telemetry forms the backbone of critical management functions such as anomaly detection, precise billing, and efficient traffic engineering. While traditionally performed on specialized network hardware, the shift towards software-based monitoring on end-host hypervisors offers unparalleled flexibility and ease of deployment. At the forefront of this evolution are sketches, approximate data structures renowned for their ability to provide high-fidelity network statistics—like identifying heavy hitters, counting distinct flows, and measuring entropy—with a remarkably low resource footprint. These compact structures, exemplified by tools like NitroSketch in Intel DPDK, are increasingly integrated into virtual network stacks.
However, a fundamental security vulnerability undermines the trustworthiness of these promising solutions: they operate under the assumption of a benign environment. The reality of cloud infrastructure dictates a more adversarial threat model, where a malicious tenant, having achieved privilege escalation through vulnerabilities in software like SaltStack, VMware ESXi, or Xen, can compromise the hypervisor. Once an attacker gains control over the host, they can subtly manipulate telemetry results by altering sketch execution logic, modifying memory regions storing sketch counters, or injecting/dropping/modifying network packets. Such corruption can have devastating consequences, disabling critical downstream anomaly detection systems, allowing sophisticated attacks like undetected DDoS campaigns, or even enabling fraudulent manipulation of network billing.
"TrustSketch: Trustworthy Sketch-based Telemetry on Cloud Hosts," presented by Zhuo Cheng at the NDSS Symposium, directly confronts this challenge. The talk introduces TRUSTSKETCH, a novel framework designed to ensure the integrity of software-based, sketch-driven network telemetry in cloud environments. By meticulously defining the essential requirements for trustworthy telemetry and demonstrating the shortcomings of existing security solutions, TRUSTSKETCH proposes a robust architecture that leverages the unique security properties of secure enclaves (like Intel SGX) and SmartNICs. The system guarantees the correctness of telemetry results or immediately raises an alert, effectively providing "tamper evidence" against a powerful adversary controlling the entire host software stack.
Background
▶ Watch: Introduction: Vulnerability of sketch-based cloud telemetry (0:00)
The operational backbone of modern public cloud providers like AWS, Microsoft Azure, and Google Cloud Platform relies heavily on sophisticated network telemetry. This infrastructure is vital for a myriad of management tasks, from detecting Distributed Denial of Service (DDoS) attacks and enabling dynamic traffic engineering to accurately calculating customer billing. Historically, these monitoring functions were embedded within dedicated network hardware such as routers and programmable switches. However, the industry is progressively moving towards software-based telemetry solutions deployed on end-host servers, particularly within the hypervisor. This shift is driven by the immediate deployability, inherent flexibility, and reduced dependency on specialized programmable router hardware, making it a mainstream approach despite the emerging potential of offloading network stacks to SmartNICs.
Within this software-centric paradigm, sketches have emerged as a cornerstone technology. These are compact, probabilistic data structures that efficiently summarize network traffic, providing accurate flow-level statistics while consuming minimal computational and memory resources. A prime example is the Count-Min Sketch, which represents a general model for many sketching algorithms. It operates by maintaining a two-dimensional array of counters. Upon packet arrival, it extracts a flow key (e.g., a 5-tuple of source/destination IP, ports, and protocol), computes multiple independent hash values, and uses these hashes as indices to increment counters in each row. The minimum of these counters then provides an estimate of the total packets for that specific flow. Sketches are commonly integrated into the data plane of virtual switches like Open vSwitch (OVS) or VPP, or fast I/O modules like DPDK, to obtain real-time traffic statistics, which are then reported to a controller for analysis.
The core problem addressed by TRUSTSKETCH stems from a critical gap in the security model of existing sketch-based telemetry solutions. These solutions largely assume a benign operating environment, an assumption that is fundamentally flawed in a multi-tenant cloud setting. The threat model for TRUSTSKETCH posits a powerful attacker who has achieved privilege escalation and gained complete control over the entire software stack of a physical server. This includes the hypervisor, the virtual switch, and all tenant VMs. This is a practical and realistic threat, as evidenced by real-world vulnerabilities exploited in hypervisors such as VMware ESXi or configuration management tools like SaltStack. However, the threat model assumes the attacker cannot compromise secure hardware enclaves (like Intel SGX) or the SmartNIC, relying on their inherent hardware security features. The focus is exclusively on integrity, not confidentiality, thus side-channel attacks are considered out of scope.
To illustrate the severity of this threat, the talk details three distinct attack vectors:
- Runtime Compute Attack: An attacker can modify the sketch's runtime execution logic. For instance, altering just 20% of the counter update logic with random values can degrade the accuracy of crucial metrics like L2 Norm, Entropy, L1 Norm, and Cardinality by up to 100%. This allows an attacker to completely obscure traffic patterns or abnormal behavior.
- Memory Attack: An attacker can directly manipulate the sketch's data structures in memory. By reading the operating system's memory mapping table, an adversary can locate and modify critical data structures like the sketch's heap. Removing only 20% of entries from a heavy-hitter heap can lead to a 40% error in reported statistics, effectively hiding significant flows.
- Input Attack: An attacker can inject packets directly into the virtual switch's packet buffer, bypassing the sketch's monitoring logic. These injected packets are sent by the NIC but are never seen by the sketch. Injecting just 5% of packets can cause a 40% error in telemetry, allowing an attacker to generate malicious traffic undetected or manipulate billing by attributing traffic to incorrect sources.
These examples underscore a critical reality: while sketch-based telemetry offers significant advantages, its current implementations are highly vulnerable to integrity attacks. Without robust protection, adversaries can exploit these weaknesses to achieve their objectives—whether it's launching an undetected DDoS, manipulating resource accounting, or evading security alerts—with profound implications for cloud security and reliability.
To counter these threats, TRUSTSKETCH formally defines three essential security requirements for an ideal trustworthy software sketch framework:
- Sketch-compute-integrity: The sketch's execution logic (e.g., hash computation, counter updates, heap updates) must remain uncorrupted at runtime.
- Sketch-memory-integrity: The sketch's internal data structures (e.g., counters, heap) must not be modified in memory by an external entity.
- Sketch-input-integrity: The sketch must monitor every packet in the correct order, precisely as it is sent or received by the Network Interface Card (NIC).
Beyond security, the framework also demands critical system requirements: Generality (support for diverse sketching algorithms) and Performance (minimal throughput degradation, low latency, and efficient resource utilization). Existing solutions, such as cross-checking telemetry results, static software attestation, or secure memory features like AMD SME, universally fall short. They either incur prohibitive overheads, lack runtime integrity guarantees, or provide insufficient coverage, necessitating a novel approach like TRUSTSKETCH.
Key Findings
▶ Watch: Background: Cloud telemetry and Count-Min sketch basics (2:00)
The research behind TRUSTSKETCH delivers several pivotal findings and contributions that significantly advance the field of trustworthy network telemetry in cloud environments:
- Formalized Requirements for Trustworthy Telemetry: The work rigorously defines the critical correctness properties—sketch-compute-integrity, sketch-memory-integrity, and sketch-input-integrity—that are indispensable for any reliable sketch-based telemetry system. This formalization provides a clear benchmark against which future solutions can be evaluated.
- Critique of Prior Solutions: A comprehensive analysis demonstrates why existing security mechanisms, including cross-checking, software attestation, and hardware secure memory features, are inadequate for protecting sketch-based telemetry. Their limitations stem from the probabilistic nature of sketches, the stringent performance demands of network monitoring, and their inability to provide comprehensive runtime integrity protection.
- Novel Design (TRUSTSKETCH) for Tamper-Evident Telemetry: The paper introduces TRUSTSKETCH, a groundbreaking framework that strategically combines secure enclaves (like Intel SGX) for protecting the sketch's computation and memory, with SmartNICs for robust validation of network input integrity. This hybrid approach addresses the unique challenges of cloud telemetry.
- Robust Input Integrity Mechanism: A core contribution is the development of a sophisticated mechanism for ensuring input integrity. This involves the use of time-based epochs, in-band MAC synchronization, packet tagging with sequence numbers, and reorder buffers to effectively handle multithreaded and potentially out-of-order packet processing on high-speed network interfaces.
- Comprehensive End-to-End Evaluation Against Real Attacks: An extensive experimental evaluation validates TRUSTSKETCH's effectiveness. The prototype successfully detected a wide array of realistic integrity attacks, including packet injection and header modification, in a timely manner (less than 1 second) and without any prior knowledge of the attacker's specific strategy.
- Minimal Performance Overhead: Despite providing strong security guarantees, TRUSTSKETCH demonstrates remarkably low overhead. It incurs only a 7% throughput degradation, adds a mere 6µs additional latency, and consumes a negligible 1.9% of NIC CPU utilization and 0.53% of NIC on-chip memory while achieving 40Gbps throughput.
- Generality and Maintained Accuracy: The framework supports a wide spectrum of 8 state-of-the-art sketching algorithms—covering diverse telemetry tasks like heavy hitter detection, cardinality estimation, and flow size distribution—without any degradation in their inherent accuracy.
- Significant Reduction in Trusted Computing Base (TCB): By carefully partitioning system components and isolating only the essential sketch logic within the secure enclave, TRUSTSKETCH reduces the Trusted Computing Base size by an impressive 99x compared to an approach that would place all components (virtual switch, I/O module, sketch) within the enclave.
Technical Deep Dive
▶ Watch: Understanding the threat model and attacker capabilities (2:55)
TRUSTSKETCH's innovative design hinges on a strategic partitioning of the network telemetry system, leveraging two distinct roots of trust available on modern server hardware: secure enclaves and SmartNICs.
Roots of Trust:
- Secure Enclave (Intel SGX): This CPU feature creates an isolated, encrypted memory region, or enclave, where code and data are protected from any software outside the enclave, including the operating system kernel and hypervisor. TRUSTSKETCH utilizes SGX (specifically SGXv2) for its integrity properties, assuming the attacker cannot control the enclave itself. The design focuses on integrity, thus side-channel attacks are considered out of scope.
- SmartNIC: A programmable Network Interface Card capable of executing custom code for packet processing. SmartNICs are widely deployed and considered trustworthy in this model because their code can be cryptographically signed, they operate as standalone hardware with their own firmware, and their codebase is typically smaller and more stable than a full hypervisor. TRUSTSKETCH aims for minimal resource usage on the SmartNIC to accommodate commodity hardware.
Design Space Exploration and TRUSTSKETCH's Chosen Architecture:
The research explored various placements for the virtual switch, sketch, and I/O module components (illustrated conceptually in Figure 5 of the talk):
- Vanilla (Figure 5a): All components reside on the untrusted host. This fails all integrity requirements.
- Sketch in NIC (Figure 5b): Placing only the sketch on the SmartNIC. While potentially secure for very simple sketches, resource constraints of commodity NICs are prohibitive for more complex algorithms like UnivMon or Hydra. Furthermore, the virtual switch on the host might modify packet headers (e.g., NAT), leading to incorrect input for a sketch residing solely on the NIC, thereby failing the generality requirement.
- All in Enclave (Figure 5c): Placing the entire virtual switch, sketch, and I/O module within the enclave. This approach fundamentally fails sketch-input-integrity because packets still traverse untrusted host memory between the enclave and the SmartNIC. Moreover, virtual switch and I/O modules heavily rely on system calls that are not supported inside enclaves, and existing shielded execution frameworks lack low-level networking support. This also drastically increases the Trusted Computing Base (TCB) by 99x, making it more vulnerable.
TRUSTSKETCH's Optimal Design (Figure 5d):
TRUSTSKETCH carefully partitions the system to overcome these limitations:
- Compute and Memory Integrity: The core insight is to run only the sketch within the secure enclave. The sketch's computation logic and its memory regions (counters, heaps) are critical for telemetry accuracy. Sketches are ideal candidates for enclaves due to their small memory footprint and simple operations (hashing, additions). The virtual switch and I/O module remain in untrusted host memory. To avoid costly data copies, the sketch accesses packet headers in host memory via pointers, minimizing enclave memory usage and transition overhead. This design guarantees sketch-compute-integrity and sketch-memory-integrity while maintaining generality for diverse sketch algorithms.
- Input Integrity: Ensuring that the sketch processes precisely the packets sent or received by the NIC, in the correct order, is crucial. Since the enclave cannot directly access NIC memory securely, TRUSTSKETCH leverages a domain-specific observation: any packet reordering, drops, or injections between the enclave and the NIC must be the work of an attacker. TRUSTSKETCH employs a simple yet highly effective input validation mechanism with two checkers: an Enclave-Checker and a NIC-Checker. Both checkers compute a Message Authentication Code (MAC) over the packet headers they have observed in a given epoch. These MACs are periodically exchanged, and a shared secret key (negotiated at bootstrap) prevents attackers from forging or re-calculating MACs. The validation is time-triggered and directed (separate for incoming and outgoing traffic) to ensure timely detection regardless of traffic rate.
Ensuring Input Integrity at Scale and with Multithreading:
Implementing robust input integrity, especially in high-throughput, multithreaded environments, presents significant practical challenges:
A. Synchronizing MAC Exchanges:
Initial strawman approaches, like using packet-number-based epochs or separate MACs for ingress/egress, were found to be insufficient. For example, if an attacker drops all packets between the enclave and NIC, no MAC packets are exchanged, and the attack goes undetected (Figure 7 in the talk).
TRUSTSKETCH's Approach (Figure 8):
- Time-based Epoch: Both the Enclave-Checker and NIC-Checker run independent timers, signaling the end of an epoch (e.g., every second). This ensures timely validation even during traffic lulls or complete drops.
- In-band Synchronization: When the enclave timer fires for outgoing packets, the Enclave-Checker computes a MAC over all packets processed in that epoch. This MAC is then sent in-band as a special MAC packet immediately after the data packets. The NIC-Checker, upon receiving this MAC packet, computes its own MAC for the packets it processed in that epoch and compares the two. A symmetric procedure applies to incoming packets.
- Nonce-based Cryptographic Hash: To prevent attackers from counterfeiting or replaying MAC packets, a nonce-based cryptographic hash is added to the MAC packet, making each MAC unique per epoch.
- Alerting: Any MAC mismatch or failure to receive a MAC packet within a predefined timeout triggers an immediate alert to the cloud controller.
B. Handling Multithreading:
Modern SmartNICs utilize multiple cores and threads (e.g., Netronome NIC with 54 cores, 8 threads per core) to process packets in parallel for high throughput. This parallelism introduces two major problems for input integrity:
- MAC Mismatch (1 vs. N): Each NIC thread processes a substream and generates its own MAC, while the enclave processes the entire stream and generates a single MAC. This leads to a fundamental mismatch.
- Out-of-Order Processing: Packets can be processed out-of-order by different NIC threads, even with hardware reorder modules for egress, causing MAC input order mismatches.
TRUSTSKETCH's Approach (Figure 9):
- Packet Tagging: To maintain consistent order, when an outgoing packet arrives at the enclave, the Enclave-Checker updates a global packet sequence counter. It uses this sequence number to determine which logical substream the packet belongs to, copies the packet header to a buffer, tags the packet with the sequence number, and forwards it to the NIC-Checker.
- Reorder Buffers: Both the Enclave-Checker and NIC-Checker maintain reorder buffers (e.g., 2M buffers for M substreams per direction). They use the packet's sequence number tag to reconstruct the original logical substreams. This ensures that both checkers process packets in the identical logical order, irrespective of physical out-of-order processing.
- Streaming MAC: To minimize the memory footprint of these buffers, a streaming MAC (specifically, SipHash 18) is used. This algorithm updates an internal state incrementally as packet headers arrive, eliminating the need to buffer all packet headers for an entire epoch.
- MAC Packet Handling: If a MAC packet itself is processed out-of-order, it includes the total number of packets in that epoch, allowing the NIC-Checker to wait until all packets for that epoch are received before computing its MAC, thus maintaining synchronization.
Security Analysis:
TRUSTSKETCH's design was rigorously validated through both qualitative case-by-case analysis and a formal model using the Alloy system modeling language.
- Qualitative Analysis: Confirmed that placing the sketch in the enclave protects compute and memory integrity. The input validation mechanism guarantees input integrity by detecting any injection, drop, modification, or reordering of packets between the enclave and NIC through MAC inconsistencies or timeouts. Attacks on the MAC packets themselves (injection, modification, replay, dropping, reordering) are all designed to be detected. Even a Denial of Service (DoS) attack against the enclave (leading to packet drops at the virtual switch) would result in correct telemetry for the actual processed and tagged traffic, as the NIC only sends packets that successfully passed through the enclave.
- Model-Driven Analysis (Alloy): This formal approach defined system components, expected behaviors, and security constraints. It was instrumental in identifying design flaws, such as the vulnerability of packet-number-based epochs to complete packet drops (leading to the adoption of time-based epochs). The Alloy model confirmed TRUSTSKETCH's robustness, finding no counterexamples within a check scope of 10, and demonstrating that TRUSTSKETCH successfully prevents all six modeled attacks that compromise existing unsecure or partially-secure systems (Table II).
Demo / Proof of Concept
▶ Watch: Illustrative examples of integrity attacks on sketches (3:35)
The efficacy of TRUSTSKETCH was demonstrated through a comprehensive end-to-end prototype implementation and rigorous evaluation, showcasing its ability to detect real-world attacks with minimal performance impact.
Implementation Details:
The prototype incorporated several key optimizations and specific hardware/software choices:
- Optimizations for Enclave Transition Overhead: A naive approach of running all packet processing (virtual switch, sketch, I/O module) in a single enclave thread would incur high overhead. TRUSTSKETCH instead runs these components in separate threads on the host, passing pointers to packet data in a shared packet buffer. This parallelization eliminates costly enclave transitions, although it utilizes two more CPU cores than a single-threaded approach. Crucially, monitored packet headers are copied into the enclave before hashing and sketching operations to prevent adversary tampering of data that the sketch truly processes.
- Optimizations for Memory Usage: To minimize the memory footprint of input validation, which would otherwise require buffering all packet headers in an epoch for MAC calculation, TRUSTSKETCH employs a streaming MAC (specifically, SipHash 18). This algorithm incrementally updates an internal state as packet headers arrive, drastically reducing the required buffer size. The SipHash is computed over the 5-tuple (source IP, destination IP, source port, destination port, protocol) to ensure the integrity of these critical flow identification fields.
- Prototype Components:
- NIC Implementation: A Netronome Agilio CX 1x40GbE SmartNIC was used. The NIC-Checker logic was implemented in P4 34 and Micro-C, leveraging the SmartNIC's 54 cores (8 threads per core) for packet processing. The system was configured with 64 substreams (M=64) per direction, balancing throughput and memory.
- Sketch Implementation (Software): Open vSwitch 77 provided virtual networking, and DPDK 41 served as the fast I/O module. To enable the sketch (within the enclave) to access packets in host memory, the DPDK Mbuf module glue code was patched following the SEC-IDS paradigm, allowing zero-copy packet processing by passing pointers.
- Packet Tagging: The unique sequence number tag is inserted between the Ethernet and IP headers of outgoing packets. A single bit within this tag differentiates MAC packets from regular data packets, allowing the NIC to invoke distinct processing functions.
- Timers: The enclave uses a secure counting thread for its timer. The SmartNIC's timer was simulated for evaluation purposes, acknowledging that a hardware timer would be necessary for real-world deployment. The epoch length was set to 1 second.
- Reactions & Key Management: Upon detecting a MAC mismatch, TRUSTSKETCH can notify the cloud operator via a secure channel and provides hooks for immediate data plane responses, such as isolating the compromised server. For long-term integrity, the shared secret key is envisioned to be periodically refreshed using a strong cryptographic key generation algorithm like HOTP 87.
- TCB Size: A significant achievement is the drastically reduced Trusted Computing Base (TCB). The TRUSTSKETCH enclave component consists of only 5K Lines of Code (LoC). In stark contrast, Open vSwitch is 364K LoC, and DPDK is 131K LoC. By keeping these large, complex components in untrusted host memory, TRUSTSKETCH achieves a 99x reduction in TCB size.
Security Evaluation (End-to-End Validation with Real Attacks):
The evaluation used a testbed with two commodity servers (Intel Xeon Gold 5317 CPU, 512GB RAM, Netronome SmartNICs, SGXv2 enabled), generating realistic packet traces with MoonGen. TRUSTSKETCH was compared against a Vanilla (unsecure) implementation and Safebricks (sketches running in an enclave, but without input integrity).
- Inject-Packet-Attack: Malicious traffic was injected to bypass the sketch in the enclave.
- Vanilla: Failed to report the attack traffic volume, showing no change in monitored traffic despite high sent traffic (Figure 12a).
- TRUSTSKETCH: Successfully detected the unmonitored traffic and raised an alert within 1 second, subsequently shutting off all traffic (Figure 12b).
- Modify-Packet-Header-Attack: Outgoing packet headers were modified in the virtual switch buffer.
- Vanilla: Telemetry accuracy (L2 Norm, Entropy, Cardinality) degraded significantly after the attack launch (Figure 13a).
- TRUSTSKETCH: Immediately detected the attack within 1 second, indicated by a rapid increase in the Cfail counter on the NIC, confirming correct detection (Figure 13b).
Performance Evaluation:
- Throughput (Figure 14): Compared to Vanilla, TRUSTSKETCH incurred only a 7% overhead in throughput across diverse workloads (CAIDA, Data Center, Min-sized, Max-sized packets).
- Latency (Figure 15): TRUSTSKETCH added only 6µs to the overall latency compared to Vanilla. Safebricks, which also uses enclaves, added about 4µs, primarily due to context switches.
- Accuracy (Figure 16): TRUSTSKETCH achieved the same accuracy as Vanilla for all evaluated telemetry metrics (e.g., UnivMon Sketch shown), demonstrating no degradation in the quality of the approximate statistics.
- Optimization Gains (Figure 17): The optimizations, particularly using multiple NIC threads and reducing enclave transition overhead, significantly improved throughput, making the solution practical.
- NIC Resource Usage: Achieving 40Gbps throughput, TRUSTSKETCH's input validation mechanism consumed only 1.9% of NIC CPU utilization (10 cores with validation vs. 9 without). It used a mere 104KB of memory, representing only 0.53% of the total NIC on-chip memory (19.2MB). This minimal footprint confirms its feasibility for deployment on commodity SmartNICs.
Defensive Implications
▶ Watch: Formal security and practical system requirements for TrustSketch (4:30)
The findings and solution presented by TRUSTSKETCH carry profound implications for cloud providers and defenders responsible for securing large-scale network infrastructures. The fundamental takeaway is that the traditional assumption of a benign hypervisor environment for software-based telemetry is dangerously flawed.
Firstly, cloud providers must operate under the assumption that the hypervisor can be compromised. Given the prevalence of privilege escalation vulnerabilities (e.g., in VMware ESXi, Xen, or SaltStack), relying on untrusted software for critical telemetry directly exposes core operational functions to manipulation. This necessitates a paradigm shift towards solutions that can guarantee the integrity of monitoring data even when the host operating system is fully controlled by an adversary.
Secondly, integrity protection is paramount for network telemetry. As demonstrated by the compute, memory, and input attacks, corrupted telemetry can effectively blind security systems, allow undetected malicious activities (like DDoS attacks or data exfiltration), and enable financial fraud through manipulated billing. Defenders must prioritize solutions that provide comprehensive sketch-compute-integrity, sketch-memory-integrity, and especially sketch-input-integrity.
Thirdly, leveraging hardware roots of trust is critical. TRUSTSKETCH successfully demonstrates that a combination of secure enclaves (like Intel SGX) and SmartNICs can provide the necessary hardware-backed security guarantees. Cloud architects should integrate these technologies into their monitoring infrastructure, specifically isolating sensitive telemetry logic within enclaves and utilizing SmartNICs for robust, offloaded input validation. The minimal resource footprint and performance overhead of TRUSTSKETCH (7% throughput degradation, 6µs latency, 1.9% NIC CPU, 0.53% NIC memory) prove that such robust security is achievable without sacrificing essential cloud performance or resource efficiency.
Finally, timely and actionable alerts are essential. TRUSTSKETCH's ability to detect integrity violations within 1 second provides cloud operators with a crucial window to react. Defenders should implement not only robust detection mechanisms but also predefined, automated data plane responses, such as isolating a compromised server or rerouting traffic, to mitigate ongoing attacks swiftly. Furthermore, for cloud customers renting bare metal servers, the principles of TRUSTSKETCH can be applied to secure their own private cloud telemetry, ensuring trustworthiness within their managed infrastructure even if the specific threat models differ. The significant reduction in the Trusted Computing Base (TCB) by 99x also highlights the importance of careful system partitioning to minimize the attack surface of trusted components.
Key Takeaways
- Software-based network telemetry in cloud environments, particularly those relying on sketches on hypervisors, is fundamentally vulnerable to integrity attacks from privilege-escalated adversaries.
- TRUSTSKETCH introduces a novel, tamper-evident framework that guarantees the integrity of sketch-based telemetry by strategically leveraging secure enclaves (Intel SGX) for compute and memory protection, and SmartNICs for robust input validation.
- A sophisticated input integrity mechanism, employing time-based epochs, in-band MAC synchronization, packet tagging, and reorder buffers, is crucial for handling the complexities of high-throughput, multithreaded network traffic.
- TRUSTSKETCH demonstrates impressive practical effectiveness, detecting realistic integrity attacks (like packet injection and header modification) within 1 second, with minimal performance overheads (7% throughput degradation, 6µs additional latency).
- The framework is highly generalizable, supporting a wide array of 8 state-of-the-art sketching algorithms without compromising their accuracy, and significantly reduces the Trusted Computing Base (TCB) by 99x through careful component partitioning.
- Cloud providers and defenders must adopt solutions that ensure comprehensive integrity for network telemetry, utilizing hardware roots of trust to safeguard against hypervisor compromise and enable timely, automated responses to detected attacks.
About the Speaker(s)
Zhuo Cheng is the speaker who presented "TrustSketch: Trustworthy Sketch-based Telemetry on Cloud Hosts" at the NDSS Symposium. The talk highlights their expertise in network security, cloud computing, and the development of robust, hardware-backed solutions for critical infrastructure monitoring.
All talks from Network and Distributed System Security (NDSS) Symposium 2024