Navigating the Virtualization Battlefield: A Deep Dive into Hypervisor Attack Vectors

Navigating the Virtualization Battlefield: A Deep Dive into Hypervisor Attack Vectors

Speakers: Austin Gadient, CTO and co-founder, Valley Cyber

Conference: BSides NYC

YouTube: https://www.youtube.com/watch?v=ayw-l-8mULI

Overview

This talk, presented by Austin Gadient, CTO and co-founder of Valley Cyber, provides a critical examination of the escalating threat landscape surrounding hypervisors, with a particular focus on VMware ESXi. Gadient highlights how these foundational technologies, which underpin modern cloud infrastructure and large enterprise environments, have become increasingly attractive targets for both ransomware groups and sophisticated nation-state actors. The presentation delves into various attack vectors, from exploiting decades-old vulnerabilities to leveraging advanced lateral movement techniques and bypassing native hardening mechanisms.

The importance of this topic cannot be overstated. Despite misconceptions about its declining relevance, VMware ESXi continues to dominate the enterprise virtualization market, making it a high-value target for attackers seeking maximum impact. The financial repercussions of successful hypervisor attacks, exemplified by the MGM Resorts breach, can be staggering, extending far beyond operational disruption to include massive legal liabilities from data theft. Furthermore, the 2024 MITRE breach underscores the strategic value of hypervisors for persistent, undetectable network infiltration.

Gadient's expertise, stemming from a master's thesis on automatic exploit generation and experience securing national satellite infrastructure, provides a robust foundation for understanding these complex threats. The talk serves as an urgent call to action for organizations to re-evaluate their hypervisor security posture, moving beyond traditional perimeter defenses to implement more advanced runtime protections and comprehensive hardening strategies against an increasingly sophisticated adversary.

Background

▶ Watch: Introduction to hypervisors and ESXi focus (0:00)

Hypervisors, the fundamental technology enabling virtual machines, come in two primary types. Type 2 hypervisors, such as VirtualBox or VMware Workstation, run on top of a host operating system. In contrast, Type 1 hypervisors, the focus of this discussion, are bare-metal systems designed to run directly on server hardware, solely for orchestrating virtual machine workloads. Among Type 1 hypervisors, VMware ESXi holds a dominant market position, with Broadcom reporting that approximately 87% of its top 10,000 customers remain with VMware despite recent pricing changes. This prevalence makes ESXi an irresistible target for threat actors, offering "more bang for their buck" as malware developed for ESXi can impact a vast number of organizations.

The motivation for attacking hypervisors has surged dramatically in the last three to four years. The most common driver is ransomware, where attackers infiltrate the hypervisor to encrypt all of an organization's virtual machines simultaneously. This results in devastating operational downtime and immense pressure to pay ransoms. Notable examples include the MGM Resorts breach, which incurred an estimated $100 million in damages from the attack itself, plus an additional $45 million in class-action lawsuit settlements, not including legal fees. Many ransomware attacks also involve data exfiltration, leaking VMDK files (virtual hard disks) before encryption, leading to further legal and reputational damage.

Beyond ransomware, nation-state actors are increasingly targeting hypervisors for persistence and lateral movement. The May 2024 breach of MITRE's VMware environment exemplifies this, where the goal was to establish a long-term presence on the hypervisor—a highly privileged position often devoid of traditional Endpoint Detection and Response (EDR) solutions. From a hypervisor, attackers can launch rogue VMs, steal VMDKs, and conduct further attacks into the network undetected for extended periods. The gravity of this threat led MITRE to release ATT&CK v17 in April 2024, adding ESXi as a new platform with specific Tactics, Techniques, and Procedures (TTPs) due to the significant increase in observed threat activity.

Attackers typically gain initial access to ESXi environments through several common vectors: credential compromise (e.g., phishing an administrator), misconfigurations (exposing ESXi assets on public networks), and exploiting vulnerabilities and CVEs. These entry points pave the way for a range of sophisticated attacks that exploit the unique architecture and management paradigms of hypervisor systems.

Key Findings

▶ Watch: VMware's market dominance post-Broadcom acquisition (4:00)

The talk reveals several critical findings regarding the current state of hypervisor security and attacker methodologies:

• ESXi is a High-Value, Under-Secured Target: Despite its foundational role in enterprise infrastructure and its continued market dominance, ESXi environments are frequently overlooked in terms of dedicated security tooling, leaving a significant blind spot for defenders. The rapid increase in attacks, leading to ESXi's inclusion in MITRE ATT&CK, underscores this critical gap.
• Perimeter Defenses are Insufficient: Relying solely on firewalls and network segmentation is a dated and ineffective strategy against sophisticated hypervisor attacks. Attackers routinely bypass these defenses through credential compromise, lateral movement within internal networks, and "living off the land" using legitimate administrative tools and native ESXi binaries.
• Native Hardening Mechanisms Can Be Bypassed: Even seemingly robust ESXi security features like "Exec Installed Only" and "Secure Boot" can be circumvented. Attackers can exploit an understanding of ESXi's internal configuration mechanisms (e.g., the config store CLI) to modify runtime settings and install unsigned, malicious software, demonstrating a need for deeper, real-time integrity monitoring.
• VM Escape Attacks Pose an Existential Threat: Highly sophisticated VM escape vulnerabilities, which allow an attacker to break out of a guest VM to gain control of the underlying hypervisor, are extremely difficult to detect with traditional VM-level EDR and require specialized detection capabilities at the hypervisor level. The significant patch delays for these critical vulnerabilities further exacerbate the risk.
• Active Directory Integration is a Major Security Risk: Joining ESXi hosts to Active Directory creates a dangerous privilege escalation path. A compromised AD environment can grant attackers immediate administrative access to all joined ESXi hosts without requiring separate credential theft, fundamentally undermining the principle of least privilege and control plane isolation.
• Lack of Dedicated Runtime Protection: The absence of purpose-built EDR or runtime protection on ESXi hosts themselves is a significant vulnerability. Traditional logging (Syslog) is often too noisy, easily tampered with, and too slow to be effective against rapid, in-memory attacks, highlighting the need for proactive, preventative security controls.
• Publicly Exposed ESXi Hosts Remain Prevalent: Despite highly publicized ransomware campaigns targeting exposed ESXi systems, a substantial number of these critical assets remain directly accessible on the internet, discoverable via tools like Shodan, presenting an ongoing and easily exploitable attack surface for opportunistic threat actors.

Technical Deep Dive

▶ Watch: Common attack vectors for hypervisor systems (5:30)

The talk meticulously dissects various hypervisor attack vectors, providing granular technical details and real-world examples.

ESXir Ransomware and CVE-2021-21974

The ESXir ransomware campaign, which gained notoriety in early 2023, leveraged CVE-2021-21974, a two-year-old remote code execution (RCE) vulnerability in the Service Location Protocol (SLP) service of VMware ESXi. This attack capitalized on poor firewall hygiene, targeting ESXi hosts publicly accessible on the internet.

The exploit for CVE-2021-21974 is a sophisticated, two-stage memory corruption attack involving a buffer over-read followed by a buffer overflow. The initial stage focuses on heap grooming to arrange objects in memory, allowing the attacker to leak critical information from the vulnerable SLP process's address space. This includes identifying the memory addresses of functions like system, which are essential for executing arbitrary commands. Once these addresses are obtained, the second stage triggers a buffer overflow to achieve arbitrary code execution.

Post-exploitation, the attacker's primary objective is virtual machine discovery (MITRE ATT&CK T1673). This involves enumerating active VMs and then forcibly shutting them down using commands like kill VMX. The necessity to kill VMs arises because their VMDK files (virtual hard disks) are locked while in use, preventing encryption or data exfiltration. After the VMs are offline, the ransomware payload, often downloaded via wget, encrypts the VMDKs, rendering the virtual machines unbootable and appending a unique extension (e.g., .arg) to the encrypted files.

Rapid7's analysis revealed approximately 19,000 publicly accessible ESXi systems vulnerable to this RCE at the peak of the ESXir campaign, leading to hundreds of impacted organizations. Defensive measures emphasize strict network segmentation, firewalling off ESXi management interfaces, and specifically disabling access to the SLP service if not absolutely required. Patching is crucial, but virtual patching—using firewalls or security tools to detect and block exploit behavior—offers an interim solution for critical systems that cannot tolerate downtime for immediate reboots.

Lateral Compromise and Living Off the Land

The MGM Resorts breach serves as a prime example of lateral compromise and "living off the land" techniques. Attributed to the Scattered Spider threat group, this attack began with sophisticated phishing campaigns, including SIM swapping and AI-powered voice spoofing, targeting help desk personnel. Once initial credentials were compromised, attackers moved laterally from the cloud environment into the VMware infrastructure.

Instead of exploiting new CVEs, the attackers leveraged legitimate administrative tools and protocols. Their typical strategy involves targeting vCenter, the central management console for ESXi hosts. From vCenter, they can enable services like SSH on individual ESXi hosts. With SSH enabled and stolen administrator credentials, attackers can log directly into the ESXi hosts and execute commands using pre-installed utilities.

ESXi hosts, being based on a customized Linux kernel, come with a suite of familiar tools, notably BusyBox (providing commands like grep, ls, rm) and OpenSSL. Attackers use these native binaries to perform ransomware operations. For instance, they can enumerate VMDK files, use OpenSSL for encryption, and then rm the original unencrypted files. To cover their tracks, attackers can easily delete ESXi logs using simple rm commands, obscuring evidence of password usage or malicious activity. This "living off the land" approach makes detection challenging, as the activities appear to be legitimate administrative actions.

Active Directory Integration Vulnerability

A specific vulnerability, highlighted in 2024, arises when ESXi systems are joined to Active Directory (AD). This configuration creates a special AD group called ESXi Admins. Any user added to this group automatically gains administrative privileges on all joined ESXi hosts and can log in using their AD credentials.

If an attacker compromises the AD environment, they can create or add a user to the ESXi Admins group. With this elevated privilege, they can then log into ESXi hosts, modify default user passwords (e.g., the root user), and proceed with ransomware or other malicious activities. Tools like PowerCLI, VMware's command-line interface based on PowerShell, can be used to automate these actions across multiple hosts, changing passwords or creating new user accounts programmatically. This demonstrates a critical breakdown in privilege separation, making AD compromise a direct pathway to hypervisor control.

Persistence and Hardening Bypass

Nation-state actors, as seen in the MITRE breach, prioritize persistence on ESXi. Achieving persistence is challenging on ESXi because its file system is largely ephemeral; many files disappear after a reboot. Attackers overcome this by creating and installing malicious vSphere Installation Bundles (VIBs). VIBs are essentially .targz files with XML metadata that define how software is installed and persisted on ESXi.

However, modern ESXi versions include hardening features:

• Exec Installed Only: Enabled by default on newer ESXi versions, this protection prevents the execution of any binary not installed via a VIB. While it blocks standalone executables, it does not prevent "living off the land" attacks using native shell scripts, Python, or BusyBox commands.
• Secure Boot: If enabled, Secure Boot prevents the installation of unsigned VIBs, leading to a "purple screen of death" upon reboot if an unsigned VIB is detected. While crucial, Secure Boot is often disabled (estimated 75% of environments).

Attackers can bypass these protections. While directly forcing an unsigned VIB install (esxcli software vib install -f) might fail due to acceptance level checks, and changing the host's acceptance level (esxcli system settings advanced set) might be blocked by Secure Boot, a more subtle bypass exists. The esxcli commands are Python scripts, and by analyzing their underlying logic, attackers discover that acceptance levels are stored and manipulated via the config store CLI. The config store is a runtime memory component, structured as JSON, that defines system settings. Attackers can directly modify the runtime acceptance level using configstorecli set, bypassing Secure Boot's checks on static configuration. This allows the installation of an unsigned VIB, achieving persistence and enabling malware execution even on hardened systems.

VM Escape Attacks

VM escape attacks represent the pinnacle of hypervisor exploitation, allowing an attacker with administrative privileges within a guest VM to break out and gain control of the underlying ESXi host. These are generally highly sophisticated memory corruption exploits targeting virtual device drivers or virtual interfaces presented by the hypervisor to the guest.

Recent examples include CVE chains disclosed in March 2024 and those demonstrated at Pwn2Own in July 2024. A significant challenge with these vulnerabilities is the patching delay (often two months or more) due to VMware's validation partnerships with hardware vendors like HPE and Dell. This delay leaves a window of vulnerability for organizations. Detecting VM escapes from within the guest VM is exceptionally difficult, as the malicious activity occurs at the hypervisor level, outside the scope of traditional EDR operating within the VM. The esoteric and highly specialized nature of these exploits further complicates detection.

PowerCLI Abuse and Rogue VMs

Two other significant areas of research and attack are PowerCLI abuse and rogue VMs.

PowerCLI is a powerful API for remote administration of VMware environments. Attackers who gain access to vCenter often leverage PowerCLI to automate their activities, such as creating new user accounts, changing passwords, or enabling services like SSH across multiple ESXi hosts simultaneously. This enables rapid, scripted deployment of ransomware or other malicious payloads.

Rogue VMs, famously used in the MITRE breach, are virtual machines launched directly from the ESXi command line using the vmx application and a specially crafted .vmx file. Crucially, these VMs are not visible in the vCenter or ESXi web GUI, making them a form of "shadow IT" for attackers. Rogue VMs serve several purposes: they provide a persistent platform for command and control (C2) or other malware, and they can be used to steal data. By attaching a rogue VM to the VMDK files of other critical VMs (e.g., a domain controller), attackers can exfiltrate sensitive data like password hashes, facilitating further lateral movement within the network.

Demo / Proof of Concept

▶ Watch: MITRE ATT&CK Framework v17 adds ESXi platform (6:00)

The talk features several compelling demonstrations illustrating the discussed attack vectors:

1. ESXir Ransomware (CVE-2021-21974) on VMware 6.7:

• The demo begins with a vulnerable ESXi 6.7 system hosting VMs named Gandalf, Frodo, and Gimli.
• On the attacker's machine, a Netcat listener is set up. The exploit, a two-stage buffer over-read and buffer overflow, is launched.
• The first stage performs heap grooming to leak memory addresses, specifically the system function, to enable arbitrary code execution.
• Once the system address is obtained, the exploit sends further packets to pop a shell back to the Netcat listener.
• Upon successful shell access, the attacker executes uname -a to confirm control.
• The next step involves wget to download the ransomware payload.
• Before encryption, the ransomware script outputs kill VMX, demonstrating the VM enumeration and shutdown phase necessary to unlock VMDKs.
• After encryption, an ls command on the datastore reveals the Gimli.vmdk file now has an .arg extension.
• Attempting to power on the Gimli VM from the ESXi web console results in a "VMDK file cannot be found" error, signifying successful encryption and operational disruption.

1. Lateral Movement Ransomware (MGM Style) via SSH:

• This demo assumes an attacker has compromised vCenter credentials.
• The attacker navigates the vCenter GUI to the configuration tab of a target ESXi host and enables the SSH service.
• Then, via SSH, the attacker logs into the ESXi host.
• They cd into the directory of a target VM (Walter White) and execute a "living off the land" ransomware command. This command uses grep to list files, pipes them to OpenSSL for encryption (using a password passed on the command line), and then uses rm to delete the original files.
• The encrypted files are shown with a "nuclear sign" extension.
• Attempting to start the Walter White VM results in an "Operation failed" error, confirming the VM is unbootable. The speaker notes that logs containing the OpenSSL password could be deleted with a simple rm command.

1. Active Directory Integration Privilege Escalation:

• The demo starts with access to a Windows host with domain controller privileges.
• An ESXi Admins group is created in Active Directory, and a user named "Billy Jean" is added to it.
• The attacker then logs into the ESXi web console using Billy Jean's AD credentials, immediately gaining administrative access.
• From the ESXi console, the attacker navigates to "Users," selects the root user, and modifies its password.
• Finally, the attacker logs into the ESXi host via SSH using the newly set root password and proceeds to run the OpenSSL-based ransomware, leading to another "bad day" for the system administrator.

1. Persistence Bypass (Exec Installed Only/Secure Boot):

• This demo targets a hardened ESXi environment with Exec Installed Only and Secure Boot enabled.
• First, an attempt to run a standalone ransomware executable is blocked, with the VM kernel log displaying "execution of non-installed file prevented."
• Next, an attempt to naively install an unsigned VIB using esxcli software vib install fails due to an acceptance level error.
• An attempt to change the acceptance level using esxcli system settings advanced set also fails, explicitly stating that Secure Boot is enabled.
• The bypass is then demonstrated: The attacker uses the configstorecli to directly modify the runtime acceptance level to communitySupported. This manipulation bypasses the Secure Boot check that would normally prevent the acceptance level change via esxcli.
• With the runtime acceptance level successfully modified, the esxcli software vib install command for the unsigned VIB now succeeds.
• The malicious VIB is installed, allowing the ransomware to execute and encrypt files, proving that even advanced native hardening can be circumvented.

Defensive Implications

▶ Watch: Case Study: ESXiArgs ransomware attack (early 2023) (6:40)

Securing hypervisor environments, particularly VMware ESXi, requires a multi-layered and proactive approach that extends beyond traditional perimeter defenses.

• Prompt Patching and Virtual Patching: Organizations must prioritize applying security patches for ESXi diligently. Acknowledge the potential delays in patch availability for critical vulnerabilities (e.g., VM escape CVEs) due to vendor validation cycles. For critical infrastructure or systems that cannot tolerate downtime, virtual patching is a vital interim strategy. This involves using firewalls or dedicated security tools to detect and block the behavior of known exploits, rather than directly patching the underlying code.
• Robust Network Segmentation: Isolate ESXi hosts and their management interfaces (vCenter, SSH, web console) on highly restricted network segments. Implement strict firewall rules to limit access to these interfaces to only necessary administrative sources. Crucially, disable unnecessary services like the SLP service (which was exploited by ESXir) if not explicitly required by your environment. A proactive step is to search Shodan for publicly exposed VMware assets belonging to your organization and immediately take them offline.
• Dedicated Runtime Protection (EDR for ESXi): Traditional EDR deployed within guest VMs is insufficient for hypervisor-level attacks. Organizations need specialized runtime protection or EDR solutions specifically designed for ESXi hosts. These tools can detect "living off the land" attacks (e.g., malicious use of OpenSSL, BusyBox, PowerCLI), unauthorized VIB installations, and other suspicious activities directly on the hypervisor before they can cause widespread damage. Such solutions should also incorporate anti-tampering capabilities to prevent attackers from disabling them.
• Multi-Factor Authentication (MFA) Everywhere: Implement strong MFA for all access to vCenter, ESXi hosts, and any jump hosts used to manage the environment. This is a fundamental defense against credential compromise, which is often the initial step in lateral movement attacks. Even if credentials are stolen, MFA acts as a critical barrier to entry.
• Avoid Active Directory Integration for ESXi: A significant recommendation is to not join ESXi systems to Active Directory. While seemingly convenient for management, this creates a dangerous privilege escalation path where a compromise of your AD environment can immediately grant administrative access to all joined ESXi hosts. Keep the VMware control plane as isolated and segregated as possible.
• Enhanced Logging and Monitoring (with caveats): While Syslog can gather logs from ESXi hosts and forward them to SIEMs for analysis, it has limitations. Syslog generates a high volume of data, much of which is not security-relevant, making it challenging to identify true threats. Furthermore, attackers who gain access can easily tamper with or delete Syslog data, undermining its forensic value. Relying solely on Syslog for detection of sophisticated VM escapes or rapid ransomware deployment is often too slow and unreliable. However, for less sophisticated attacks, monitoring specific commands (e.g., DCUI process logs related to VM execution) can provide some visibility.
• Harden ESXi Configurations: Follow VMware's official hardening guides. Enable Secure Boot where feasible, understanding its implications for unsigned VIBs. Be aware of Exec Installed Only and other native protections, and understand their bypass mechanisms to implement compensating controls.
• Assume Breach Mentality: Recognize that the "strong perimeter" defense model is obsolete for hypervisors. Assume that attackers may find a way inside and focus on detecting and responding to lateral movement and post-exploitation activities within the VMware environment.

Key Takeaways

• Hypervisors are Prime Targets: VMware ESXi remains a dominant, high-value target for both ransomware and nation-state actors, with attacks rapidly increasing in frequency and sophistication.
• Beyond Perimeter Defense: Traditional firewalls and network segmentation are insufficient. Attackers frequently bypass these by exploiting credential compromise, lateral movement, and "living off the land" using native ESXi tools.
• Native Hardening is Not Absolute: ESXi's built-in protections like Exec Installed Only and Secure Boot can be bypassed by understanding and manipulating runtime configuration mechanisms, highlighting the need for deeper integrity checks.
• Specialized Security is Critical: Standard EDR within guest VMs is blind to hypervisor-level threats and VM escapes. Dedicated runtime protection or EDR for ESXi hosts is essential for detecting and preventing attacks on the hypervisor itself.
• Isolate and Authenticate: Strong MFA for all management access and avoiding ESXi integration with Active Directory are crucial for preventing credential-based attacks and privilege escalation.
• Proactive Visibility is Key: Regularly audit publicly exposed ESXi assets using tools like Shodan, and implement robust monitoring that goes beyond basic Syslog to detect malicious activity on the hypervisor.

About the Speaker(s)

Austin Gadient is the CTO and co-founder of Valley Cyber, a company dedicated to hypervisor security. His background includes extensive experience in offensive and defensive security. Prior to co-founding Valley Cyber, Gadient completed a master's thesis focused on automatic exploit generation, developing a system called "Rage" that could automatically create memory corruption exploits to bypass modern Linux defenses like Data Execution Prevention (DEP) and Address Space Layout Randomization (ASLR). He also served in the Air Force, where he was responsible for software security within the nation's satellite infrastructure, including both orbiting satellites and ground systems, which predominantly ran on Linux operating systems. His experiences with the challenges of existing Linux security tools motivated him to start Valley Cyber, ultimately pivoting the company's focus to the critical and underserved area of hypervisor security.