Preparing for the Future: A Discussion of our Rapidly Evolving Threat Landscape

Preparing for the Future: A Discussion of our Rapidly Evolving Threat Landscape

Speakers: Panel

Conference: DEF CON 32

YouTube: https://www.youtube.com/watch?v=dGSPysbdvJo

Overview

This DEF CON 32 panel, organized by the IoT Village and moderated by Rachel, brought together industry experts to dissect the rapidly evolving cybersecurity landscape, with a particular focus on the intersection of Artificial Intelligence (AI), Machine Learning (ML), and Internet of Things (IoT) security. The discussion featured Steve McGregory, Director of Cyber Security R&D at Keysight; Jamie Hardy, Product Manager for Adversary Management at Intuit; and Ted Harrington, Partner at Independence Security Evaluators and co-founder of IoT Village. The panelists explored how these transformative technologies are reshaping both offensive and defensive security strategies, presenting unprecedented challenges and opportunities.

The core of the talk revolved around the dual-edged sword of AI: its potential to empower adversaries to launch more sophisticated and automated attacks, contrasting with its promise to augment defenders and address the chronic shortage of cybersecurity talent. The discussion underscored the inherent vulnerabilities of IoT devices, exemplified by past events like the Mirai botnet, and highlighted how AI could amplify such threats if foundational security principles are neglected. This conversation is crucial for anyone involved in developing, deploying, or defending connected systems, offering insights into preparing for a future where intelligent automation plays a pivotal role in the ongoing cyber arms race.

The panel's insights are particularly relevant for organizations struggling with limited security resources, such as small to medium-sized businesses (SMBs), who stand to gain significantly from AI-driven automation in an increasingly complex threat environment. By examining how AI can accelerate discovery, predict adversary movements, and streamline defensive operations, the speakers provided a candid assessment of the current state and future trajectory of cybersecurity, emphasizing the need for proactive integration of security into product design and strategic adoption of emerging defensive technologies.

Background

▶ Watch: Panel introduction and speaker backgrounds (0:00)

The discussion began by grounding itself in the historical context of IoT security, acknowledging the well-documented vulnerabilities that have plagued these devices since their widespread adoption. Steve McGregory immediately invoked the Mirai botnet as a stark reminder of the potential for mass compromise when IoT devices lack robust security measures. Mirai, which emerged in 2016, famously leveraged default or easily guessed credentials on hundreds of thousands of internet-connected devices, primarily CCTV cameras and DVRs, to launch devastating Distributed Denial of Service (DDoS) attacks. This event served as a critical wake-up call, illustrating how seemingly innocuous devices could be weaponized on a massive scale due to fundamental security oversights.

Jamie Hardy further emphasized the critical importance of foundational security, specifically highlighting the hardware root of trust as the bedrock for secure IoT systems. This concept refers to a set of immutable functions, implemented in hardware, that are always trusted by the system. It forms the basis for secure boot, secure storage, and cryptographic operations, ensuring the integrity and authenticity of the device from its very first power-on. The absence or weakness of such a root of trust leaves devices susceptible to a myriad of attacks, from firmware manipulation to credential exfiltration.

The panel also touched upon the persistent challenge of under-resourced security teams, particularly within small and mid-sized businesses. Steve McGregory recalled statistics from a decade prior, indicating a severe imbalance of security personnel to employee count—often one IT generalist handling security for hundreds of employees. This historical deficiency has created a fertile ground for adversaries, as many organizations lack the dedicated expertise and tooling to effectively defend against evolving threats. This context sets the stage for understanding why the advent of AI and ML is viewed with both trepidation and hope: it represents a potential force multiplier for attackers, but also a lifeline for defenders struggling to keep pace. The problem exists because security has often been an afterthought, particularly in rapidly developing IoT ecosystems, and the human element of security expertise remains scarce and expensive.

Key Findings

▶ Watch: Steve's view: AI amplifying IoT botnet threats (3:40)

The panel's discussion yielded several crucial insights into the evolving threat landscape, primarily centered on the transformative impact of AI and ML:

Firstly, a consensus emerged that AI and ML technologies are accelerating both offensive and defensive capabilities in cybersecurity. Jamie Hardy noted that while defenders are leveraging AI for smarter, faster responses, adversaries are equally adept at using tools like ChatGPT to generate malicious code, craft sophisticated phishing campaigns, and automate attack processes. This creates a relentless "arms race" where the pace of innovation dictates the advantage. Steve McGregory specifically warned that AI-powered automated attack tools could make past incidents like the Mirai botnet "look minuscule potentially," indicating a significant escalation in potential scale and impact.

Secondly, the panelists stressed that while the technological landscape is changing rapidly, fundamental security principles remain constant. Ted Harrington pointed out that "some new tech comes around and we're like, everything's different now. And it's like, well, no, not really, the fundamental principles of how we secure systems, how systems get exploited, those are the same." This highlights that even with AI, the core vulnerabilities (e.g., weak credentials, unpatched software, insecure configurations) persist, and the focus should remain on building security into the product lifecycle, especially for IoT devices.

Thirdly, the panel underscored the inherent weakness of many IoT devices. They are often deployed with minimal security considerations, making them prime targets for automated attacks. Jamie Hardy's emphasis on the hardware root of trust highlighted that foundational security is often missing or poorly implemented in these devices, leaving them vulnerable to compromise from the ground up.

Fourthly, the discussion revealed a professionalization of cybercrime, with adversaries operating with business-like efficiency. Jamie Hardy colorfully remarked that "there are adversaries out there literally running their books in QuickBooks," illustrating that cybercriminal enterprises are organized, financially motivated, and increasingly leveraging advanced tools to streamline their operations. This shift demands an equally sophisticated and organized defensive posture.

Finally, a significant finding was the potential for AI to serve as a force multiplier for under-resourced security teams, particularly in small to mid-sized businesses. Steve McGregory expressed hope that "AI ML and AI are meant to process large amounts of data and do things autonomously, which should help that space." Ted Harrington provided a concrete example: AI's ability to significantly reduce the "person power" required for handling security questionnaires—both in drafting responses and analyzing incoming queries—by automating the process and identifying red flags. This suggests AI could bridge the talent gap and provide essential security capabilities to organizations that cannot afford dedicated security teams.

Technical Deep Dive

▶ Watch: Jamie's view: Hardware root of trust and AI-driven adversary speed (4:20)

The panel discussion, while primarily strategic, touched upon several key technical concepts and implications, particularly concerning the interaction between AI, ML, and IoT security.

At the heart of the IoT vulnerability discussion was the Mirai botnet. Technically, Mirai operates by scanning the internet for IoT devices accessible via Telnet or SSH. It then attempts to log in using a list of common factory default usernames and passwords. Once authenticated, it installs malware that turns the device into a bot, part of a larger network controlled by the attacker. These bots then await commands to launch DDoS attacks against specified targets. The technical implication highlighted by Steve McGregory is that AI could significantly enhance this process, enabling adversaries to:

1. Automate Exploit Development: AI/ML can be used to analyze device firmware, identify zero-day vulnerabilities, or even generate exploit code much faster than human researchers.
2. Sophisticated Scanning: Instead of simple password guessing, AI could learn patterns in IoT device configurations, predict common vulnerabilities based on device type or manufacturer, and prioritize targets for maximum impact, making the "minuscule" Mirai threat seem quaint.
3. Adaptive Attack Campaigns: AI-driven botnets could dynamically adapt their attack vectors, evade detection, and self-heal, making them far more resilient and difficult to dismantle.

Jamie Hardy's insistence on the hardware root of trust is a fundamental technical security concept. A hardware root of trust typically involves a tamper-resistant module (e.g., a Trusted Platform Module (TPM) or a Hardware Security Module (HSM)) that stores cryptographic keys and performs secure boot operations. During boot-up, the root of trust cryptographically verifies the integrity of subsequent boot stages and the operating system. Without this, an attacker could inject malicious code at an early stage of the boot process, compromising the entire device before any software-based security measures can even activate. This is especially critical for IoT devices, which often run lightweight operating systems and may not have robust software-based protection.

The panel also explored the technical capabilities of AI/ML in attack surface management and defense. Jamie Hardy articulated how AI tools can enhance discovery: "Every port we discover, we can tell like 10 more." This points to advanced machine learning algorithms that can:

1. Predictive Analysis: By analyzing network traffic, known vulnerabilities, and historical attack data, AI can predict which new ports or services might be exposed, or where adversaries are likely to probe next. This moves beyond simple port scanning to intelligent, context-aware discovery.
2. Automated Asset Discovery: AI can continuously scan and map an organization's digital footprint, identifying previously unknown or rogue devices, cloud instances, and shadow IT, which are often overlooked attack vectors.
3. Anomaly Detection: ML algorithms can establish baselines of normal network behavior and instantly flag deviations, such as unusual port activity, unexpected data flows, or unauthorized access attempts, allowing for faster incident response.

Ted Harrington's example of automating security questionnaires illustrates AI's potential in Natural Language Processing (NLP) and document automation. AI models can be trained on past security policies, compliance documents, and questionnaire responses. When presented with a new questionnaire, the AI can:

1. Parse and Understand: Analyze the questions, identify key themes, and map them to internal knowledge bases.
2. Generate Draft Responses: Formulate accurate and consistent answers based on approved templates and existing documentation, significantly reducing manual effort.
3. Identify Gaps/Red Flags: For incoming questionnaires from vendors, AI can quickly scan for non-compliance, high-risk answers, or inconsistencies that human analysts might miss, thereby streamlining vendor risk management.

In essence, the technical deep dive reveals AI as a tool for both sophisticated, automated exploitation and intelligent, large-scale defense, with the foundational security of hardware remaining paramount, especially for the vast and vulnerable IoT landscape.

Demo / Proof of Concept

▶ Watch: Ted's view: Enduring security principles despite emerging AI tech (6:00)

This panel discussion was a conversational exploration of ideas and perspectives on the evolving threat landscape, rather than a technical presentation involving specific demonstrations or proofs of concept. Therefore, no live demo or technical PoC was conducted during the session.

Defensive Implications

▶ Watch: Addressing security challenges for small to mid-sized businesses (8:50)

The panel's insights offer critical guidance for defenders navigating the AI-driven threat landscape, emphasizing both foundational security practices and the strategic adoption of new technologies.

Firstly, the paramount implication for defenders, particularly those involved in IoT development and deployment, is the absolute necessity of building security in from the start. Steve McGregory's cautionary note about Mirai and Jamie Hardy's strong advocacy for the hardware root of trust underscore that retrofitting security is often ineffective and prohibitively expensive. Defenders must demand and implement secure design principles, including:

• Secure Hardware: Utilizing tamper-resistant components and ensuring a robust hardware root of trust for secure boot and cryptographic operations.
• Secure Software Development Lifecycle (SSDLC): Integrating security checks, code reviews, and vulnerability testing throughout the entire development process.
• Strong Defaults: Eliminating default or easily guessable credentials, requiring unique, strong passwords, and ensuring secure configurations out-of-the-box.
• Regular Updates: Implementing a robust mechanism for secure, over-the-air (OTA) updates to patch vulnerabilities and maintain device integrity throughout its lifecycle.

Secondly, organizations must actively leverage AI and ML to augment their defensive capabilities. Given the accelerated pace of attacks and the persistent shortage of human security talent, AI becomes a critical force multiplier. Defenders should explore using AI for:

• Enhanced Threat Detection and Response: Deploying AI-powered Security Information and Event Management (SIEM) and Extended Detection and Response (XDR) solutions that can process vast amounts of data, identify subtle anomalies, and automate initial response actions faster than human analysts.
• Proactive Attack Surface Management: Utilizing AI for continuous asset discovery, vulnerability scanning, and predictive analysis to identify and prioritize potential attack vectors before adversaries exploit them. Jamie Hardy's point about AI's ability to "tell like 10 more" ports from one discovery highlights this predictive power.
• Automation of Mundane Tasks: As Ted Harrington illustrated with security questionnaires, AI can automate repetitive, time-consuming tasks, freeing up valuable human resources for more complex analytical and strategic work. This extends to compliance auditing, incident triage, and report generation.
• Security Orchestration, Automation, and Response (SOAR): Integrating AI into SOAR platforms to automate playbooks for common incidents, reducing mean time to detect (MTTD) and mean time to respond (MTTR).

Thirdly, the panel implicitly stressed the importance of focusing on fundamental security hygiene. While AI changes the implementation, the core attack vectors often remain the same. This means defenders must not neglect basics such as:

• Patch Management: Promptly applying security updates to all systems and devices.
• Access Control: Implementing least privilege principles and robust Multi-Factor Authentication (MFA).
• Network Segmentation: Isolating critical systems and IoT devices from the broader network to contain breaches.
• Employee Training: Educating users about phishing, social engineering, and secure computing practices.

Finally, there's a clear implication for the cybersecurity industry to develop accessible, AI-driven solutions for small to mid-sized businesses (SMBs). These organizations, often lacking dedicated security teams, are highly vulnerable. The hope is that AI can democratize advanced security, providing sophisticated protection that was once only available to large enterprises. This could involve cloud-based security services with embedded AI, simplified security tools, or managed security service providers (MSSPs) leveraging AI to offer cost-effective protection.

In summary, defenders must embrace a dual strategy: fortify the foundational security of their systems, especially IoT, and strategically integrate AI/ML technologies to enhance their detection, response, and automation capabilities against an increasingly intelligent and automated adversary.

Key Takeaways

• AI/ML Accelerates Both Attackers and Defenders: The rapid evolution of AI empowers adversaries with faster, more automated attack tools, making threats like Mirai-scale compromises potentially much larger. Simultaneously, AI provides defenders with enhanced capabilities for detection, analysis, and response.
• Foundational Security Remains Critical for IoT: Despite AI advancements, the inherent weaknesses of IoT devices necessitate a strong emphasis on hardware root of trust and "security by design" from the initial development phase to prevent widespread vulnerabilities.
• AI as a Force Multiplier for Under-Resourced Teams: AI/ML offers a promising solution for organizations with limited security personnel, such as SMBs, by automating labor-intensive tasks like security questionnaires and enhancing attack surface management through predictive discovery.
• Professionalization of Cybercrime: Adversaries are operating with increasing sophistication and business-like efficiency, leveraging advanced tools and organized structures, demanding a similarly professional and adaptive defensive posture.
• Fundamental Principles Endure: While technology evolves, the core principles of securing systems and understanding exploitation methods remain constant. Defenders must not neglect basic security hygiene while adopting advanced tools.

About the Speaker(s)

The panel featured a diverse group of cybersecurity experts:

• Rachel (Moderator): The organizer of the IoT Village, Rachel moderated the discussion, bringing together industry leaders to address critical topics in the evolving threat landscape.
• Steve McGregory: As the Director of Cyber Security R&D at Keysight, Steve McGregory focuses on developing products that assist organizations in testing network security and assessing device vulnerabilities. With a career spanning since 1991, he spent the 90s running an internet service provider, which led him to dedicate his efforts to cybersecurity in 2000. He previously built an intrusion prevention system at TippingPoint before transitioning into security testing.
• Jamie Hardy: A Product Manager for Adversary Management at Intuit, Jamie Hardy brings a wealth of experience from various sectors, including Fintech, semiconductors, and government. She transitioned from engineering to product management when she took over IoT security at Qualcomm, giving her deep insights into the challenges and opportunities in the IoT and cloud security space.
• Ted Harrington: A Partner at Independence Security Evaluators and a co-founder of IoT Village, Ted Harrington is also the author of the book "Hackable, how to do application security right." His work focuses on applying ethical hacking insights from the front lines to help organizations build more secure applications.