What Lies Beneath the Surface? Evaluating LLMs for Offensive Cyber Capabilities

What Lies Beneath the Surface? Evaluating LLMs for Offensive Cyber Capabilities

Speakers: Michael Peretti, Murza Daughter, Alex Burn, MITRE

Conference: Black Hat USA

YouTube: https://www.youtube.com/watch?v=p9T4gWds54o

Overview

The rapid proliferation and increasing sophistication of large language models (LLMs) have sparked critical questions within the cybersecurity community: To what extent do these models possess offensive cyber capabilities, and should defenders be concerned? This talk, delivered by a multidisciplinary team from MITRE, addresses this complex challenge head-on by presenting novel methodologies and tools for scientifically evaluating LLMs' potential as cyber threats. The speakers, Michael Peretti, Murza Daughter, and Alex Burn, alongside their broader team, highlight the current lack of comprehensive, metric-driven assessments and introduce three distinct testing frameworks designed to provide a clearer, more quantifiable understanding of LLM capabilities across various offensive cyber domains.

The research stems from a pressing need to move beyond speculative "maybe" answers when assessing LLM cyber risk. Existing evaluation approaches are often fragmented, lacking integration with real-world cyber tools, and fail to provide actionable insights for defenders or policymakers. MITRE's work aims to fill this gap by developing tests that span traditional question-answering benchmarks to full-scale emulation of complex enterprise networks, all while pairing LLMs with the same cyber tools human operators use. This scientific approach seeks to not only measure current capabilities but also to track the evolution of LLM proficiency in offensive cyber operations, providing a crucial early warning system for the cybersecurity landscape.

Background

▶ Watch: Introduction and novel LLM cyber capability tests (0:00)

The emergence of powerful LLMs has naturally led to concerns about their potential misuse in offensive cyber operations. Approximately a year prior to this talk, the question of whether LLMs represent a serious cyber threat began to be posed in earnest. Initial investigations by the MITRE team revealed a critical void: while there was no immediate evidence of LLMs conducting sophisticated, multi-tactic offensive campaigns akin to advanced persistent threats (APTs), there was also a distinct lack of robust testing methodologies to definitively prove or disprove such capabilities. The models were growing larger, becoming more prolific, and increasingly applied to cyber-specific problems, yet a scientific, metric-driven framework for assessing their offensive potential remained elusive.

Existing research and benchmarks, often developed by the very companies creating these LLMs, were found to be a "hodge podge" of disparate approaches. Some focused on simple query-and-prompt interactions, others on code generation (malicious or vulnerable), and a few on basic emulations. However, these efforts frequently overlooked a crucial aspect of real-world offensive operations: the integration of LLMs with established cyber tools. Offensive operators do not work in a vacuum; they leverage a vast ecosystem of specialized tools. Current LLM evaluations often tested models in isolation, failing to assess how an LLM, when augmented with these tools, might perform. This left defenders and policymakers with an "unclear" and often "incomprehensible" understanding of the actual risks. For instance, while an LLM might be shown to identify a SQL injection vulnerability when prompted with specific details, it was unclear what this implied for its general capability in injection attacks, initial access, or broader attack path planning. This fragmented and often irrelevant testing landscape underscored the urgent need for a more comprehensive, scientific, and operationally relevant evaluation framework.

Key Findings

▶ Watch: The core question: Are LLMs actual cyber threats? (1:08)

The MITRE team’s research establishes a novel framework for evaluating LLMs, introducing three distinct and progressively complex tests designed to measure offensive cyber capabilities. These tests move beyond simple query-and-answer scenarios to full-scale emulation, aiming to provide a scientific and metric-driven understanding of LLM performance. The core finding is that while current LLMs do not yet exhibit comprehensive offensive operator capabilities, their potential is significant, especially when integrated with existing cyber tools.

The three key testing methodologies developed are:

1. Query and Prompt: A foundational, traditional LLM benchmark focusing on question-answering related to cyber tasks. This test assesses a model's ability to recall and apply cyber knowledge in response to specific prompts.
2. Tactical-Level Emulation (TACL): A more advanced scenario involving a series of 30 questions designed to emulate the decision-making process of an offensive cyber operator. These questions span the entire MITRE ATT&CK framework, covering 46 specific Techniques, Tactics, and Procedures (TTPs) across all 14 ATT&CK tactics, from Initial Access to Command and Control.
3. Full-Scale Emulation and Simulation (Cyber Layer): The most sophisticated test, leveraging an in-house simulation tool called Cyber Layer. This environment is designed to be a one-to-one behavioral model of an operator's experience, allowing LLMs to interact with simulated enterprise networks and tools in a highly realistic manner.

Initial results from the Query and Prompt tests, specifically an evaluation against Bloodhound-equivalent queries for Active Directory attacks, showed limited but present capabilities. For instance, a Mix Roll model and a larger parameter model achieved only 5 correct answers out of 12 queries, indicating that while some knowledge exists, it is not yet robust enough for complex, multi-step reasoning in this domain. The development of TACL and Cyber Layer represents a significant step towards a more comprehensive and actionable evaluation of LLM offensive potential.

Technical Deep Dive

▶ Watch: Critiquing existing LLM offensive cyber evaluation methods (2:38)

The MITRE team's approach to evaluating LLM offensive cyber capabilities is structured around three progressively sophisticated testing methodologies, each designed to probe different facets of an LLM's understanding and operational prowess.

1. Query and Prompt

This initial testing phase serves as a baseline, utilizing a traditional question-and-answer (Q&A) format commonly found in LLM benchmarks. The objective is to assess an LLM's ability to process natural language queries related to cyber security concepts and respond with accurate and relevant information. A specific example cited was the evaluation of an LLM's capacity to perform tasks equivalent to those facilitated by Bloodhound, a popular open-source tool for mapping Active Directory attack paths.

In this test, LLMs (including a Mix Roll model and a larger parameter model) were presented with 12 queries related to Active Directory reconnaissance and privilege escalation. The results indicated nascent capabilities, with the best-performing model answering only 5 out of 12 questions correctly. This suggests that while LLMs can recall some cyber-related information, their ability to conduct complex, multi-faceted reasoning or to effectively substitute for specialized tools like Bloodhound in a purely prompt-based interaction is currently limited. This highlights the gap between general knowledge and the specific, contextual reasoning required for effective offensive operations.

2. Tactical-Level Emulation (TACL)

The TACL framework represents a significant leap in complexity, moving beyond simple Q&A to emulate the tactical decision-making process of an offensive cyber operator. This test is designed to measure an LLM's ability to navigate a simulated attack scenario, making choices that align with established offensive methodologies. The TACL evaluation consists of 30 distinct questions, meticulously crafted to span 46 specific Techniques, Tactics, and Procedures (TTPs), encompassing all 14 tactics of the MITRE ATT&CK framework. This comprehensive coverage ensures that the LLM's capabilities are assessed across the entire attack lifecycle, from initial access and execution to persistence, privilege escalation, defense evasion, credential access, discovery, lateral movement, collection, exfiltration, and command and control.

The questions in TACL are not merely knowledge recall exercises; they are scenario-based prompts that require the LLM to understand context, identify potential vulnerabilities, suggest appropriate tools or commands, and plan subsequent steps in an attack chain. For example, a question might present a simulated network state and ask the LLM how it would achieve initial access or move laterally. The goal is to determine if the LLM can generate responses that reflect the strategic thinking and operational steps an actual red teamer would undertake to achieve their objectives. The talk specifically mentioned testing the Llama 3 70B model within this framework, indicating a focus on state-of-the-art open-source LLMs.

3. Full-Scale Emulation and Simulation (Cyber Layer)

The most advanced and technically intricate component of MITRE's evaluation framework is the Cyber Layer tool. This in-house simulation platform is designed to provide a high-fidelity, one-to-one behavioral model of an operator's experience within a complex enterprise environment. The core innovation of Cyber Layer lies in its ability to behaviorally model existing cyber tools and network components, making the simulated environment indistinguishable from a real one for the agents (human or LLM) interacting with it. The developers have invested significant effort to ensure this realism, often making it challenging even for their own development team to discern between simulated and real tool outputs.

Cyber Layer offers unparalleled capabilities for data generation, allowing researchers to create highly customizable and scalable enterprise networks. This includes:

• Brand new enterprise network topologies: Defining the layout and interconnections of various network segments.
• Diverse user groups and social networks: Simulating human interactions and their impact on security.
• Complex firewall rules: Enforcing network segmentation and access controls.
• Multi-enterprise scenarios: Modeling supply chain relationships and cross-organizational attacks.
• Granular host-level details: Specifying operating systems (Windows, Mac, etc.), running services, file systems, and software versions on individual machines.

This rich, dynamic environment allows for the testing of LLMs not just on theoretical knowledge, but on their ability to interact with a realistic, tool-driven operational landscape. By integrating LLMs with simulated versions of common offensive cyber tools, Cyber Layer can assess their capacity for reconnaissance, vulnerability exploitation, post-exploitation activities, and attack path planning in a way that closely mirrors real-world red teaming engagements. The aim is to understand how LLMs perform when given access to the same operational context and tools as human attackers, addressing a critical gap in previous evaluation methodologies.

Demo / Proof of Concept

▶ Watch: Miter's proposed methodology for LLM cyber evaluation (4:00)

The talk featured a demonstration of the Cyber Layer simulation tool, showcasing its capabilities through a scenario dubbed "Darthnut." This scenario represented a fairly basic penetration testing exercise designed to illustrate how an LLM agent could navigate a simulated enterprise environment and achieve offensive objectives.

The Darthnut scenario began with the LLM agent positioned on the internet, with the objective of gaining control within the internal user and server networks. The initial target was a set of web servers located in the DMZ (Demilitarized Zone). A key challenge was that direct access from the DMZ to the internal networks was restricted by firewall rules. The attack path hinged on identifying and exploiting a command injection vulnerability in the web servers.

The demonstration depicted the LLM's simulated actions:

1. Initial Access: The LLM, operating within Cyber Layer, would first identify the web servers in the DMZ.
2. Vulnerability Exploitation: It would then leverage knowledge of the command injection vulnerability to send malicious commands to these web servers.
3. Relay and Lateral Movement: Crucially, these injected commands would be designed to be relayed by the compromised web servers, allowing the LLM to gain initial visibility into the user and server networks behind the DMZ.
4. Gaining Control: The ultimate goal was to transition from mere visibility to full control within these internal networks, potentially through further reconnaissance and exploitation facilitated by the initial beachhead.

While the specific LLM's real-time decision-making process during the demo wasn't fully detailed, the demonstration effectively highlighted Cyber Layer's ability to model complex attack chains. It showed how the platform could simulate network topologies, host configurations, vulnerabilities, and the interaction of offensive tools, providing a realistic sandbox for evaluating LLM agents. This proof of concept underscored Cyber Layer's potential to assess an LLM's capacity for strategic planning, tactical execution, and adaptation within a dynamic, high-fidelity cyber environment, moving beyond theoretical capabilities to demonstrate practical application.

Defensive Implications

▶ Watch: Overview of three distinct LLM offensive cyber tests (4:50)

The research presented by the MITRE team offers crucial insights for cybersecurity defenders and policymakers grappling with the evolving threat landscape posed by large language models. The primary defensive implication is the establishment of a scientific, metric-driven framework to objectively assess LLM offensive capabilities. This moves the discussion beyond anecdotal evidence or speculative fears, providing concrete data that can inform risk assessments and defensive strategies.

Key defensive implications include:

• Informed Risk Assessment: By quantifying LLM capabilities across various attack tactics and TTPs (as demonstrated by the TACL framework spanning all 14 MITRE ATT&CK tactics), defenders can better understand specific areas where LLMs might pose a threat. This allows for targeted hardening of systems and processes against LLM-assisted attacks.
• Tracking Capability Evolution: The methodology, designed to show an "evolution of capability," enables defenders to monitor the progression of LLM proficiency over time. This acts as an early warning system, allowing defensive measures to adapt proactively as models improve.
• Understanding Tool Integration: The emphasis on testing LLMs "paired with all these cyber tools that we know exist" is critical. Defenders must understand that LLMs are unlikely to operate in isolation. Their true offensive potential emerges when integrated with existing red team tools and frameworks. This informs the need to focus defenses not just on LLM-generated outputs, but on the combined threat of LLM intelligence driving established attack tools.
• Guiding Policy and Regulation: For policymakers, the scientific metrics provide a foundation for developing informed regulations and guidelines regarding LLM development and deployment, particularly concerning their safety and potential for misuse. It helps distinguish between "interesting" but irrelevant capabilities and those that are genuinely "relevant to a red team or offensive operator."
• Developing Counter-LLM Strategies: By understanding how LLMs think and operate in offensive scenarios, defenders can begin to develop counter-LLM strategies, such as detecting LLM-generated attack patterns, identifying anomalies in LLM-driven reconnaissance, or even leveraging defensive LLMs to counter offensive ones.
• Bridging the Knowledge Gap: The research directly addresses the "unclear" and "incomprehensible" nature of LLM capabilities that previously plagued defenders. By providing clear data and a common language (e.g., MITRE ATT&CK), it helps defenders articulate and understand the risks more effectively.

Ultimately, this research empowers defenders to make data-driven decisions, anticipate future threats, and build more resilient cyber defenses in an era increasingly influenced by advanced AI.

Key Takeaways

• Current LLM Offensive Capabilities are Nascent but Evolving: While no LLM currently demonstrates the full, multi-campaign capabilities of a human offensive operator, the research indicates a clear potential for growth, especially when LLMs are integrated with existing cyber tools.
• Comprehensive, Scientific Evaluation is Crucial: The cybersecurity community lacked a robust, metric-driven framework to assess LLM offensive potential. MITRE's work provides this by introducing three distinct testing methodologies: Query & Prompt, Tactical-Level Emulation (TACL), and Full-Scale Emulation (Cyber Layer).
• Tool Integration is Key to Real-World Risk: A critical gap in prior LLM evaluations was the failure to pair models with common cyber tools. This research emphasizes that LLMs' true offensive impact will likely come from their ability to leverage and direct existing tools, mirroring human operators.
• MITRE ATT&CK Provides a Standardized Measurement: The TACL framework, covering 46 TTPs across all 14 MITRE ATT&CK tactics, offers a standardized and comprehensive way to measure an LLM's tactical proficiency in offensive operations.
• High-Fidelity Simulation is Essential for Advanced Testing: The in-house Cyber Layer tool, with its one-to-one behavioral modeling of operator experience and powerful data generation capabilities, allows for realistic, full-scale emulation of complex attack scenarios, providing deeper insights into LLM operational performance.
• Data-Driven Insights for Defenders and Policy Makers: This research aims to provide concrete metrics to help defenders and policymakers understand LLM risks, track capability evolution, and develop proactive defensive strategies against LLM-assisted cyber threats.

About the Speaker(s)

The research presented was a collaborative effort by an interdisciplinary team from MITRE, bringing together expertise in cyber security, artificial intelligence, and software development. The primary speakers for this talk were Michael Peretti, Murza Daughter, and Alex Burn.

Michael Peretti, Murza Daughter, and Alex Burn are part of a larger MITRE team that also includes Michael, Guido, Dan, Ethan, and Jampolo. This diverse team works across different departments and divisions within MITRE, united by the interdisciplinary nature of this research project that intersects cyber security, AI, and software engineering. Their collective background provides the unique blend of skills necessary to design, implement, and evaluate the complex testing frameworks discussed, aiming to provide a scientific and comprehensive understanding of LLM capabilities in offensive cyber operations.