RSAC Tech Talk - Blade Runners for AI & Critical Infrastructure by Divjot Bawa

Blade Runners for AI & Critical Infrastructure

Speakers: Divjot Bawa, Cyber-AI Policy Fellow, CISA

Conference: RSAC 2024

YouTube: https://www.youtube.com/watch?v=tPKOXF3BqOs

Overview

In this compelling RSAC 2024 talk, Divjot Bawa, a Cyber-AI Policy Fellow at CISA (Cybersecurity and Infrastructure Security Agency), posited CISA as the "Blade Runner" for the US digital ecosystem, particularly concerning the integration of Artificial Intelligence (AI) into critical infrastructure. Drawing parallels to the 1982 sci-fi classic Blade Runner, where enforcement officials mitigate risks from bio-engineered replicants in a technologically advanced dystopia, Bawa articulated CISA's mission: to safeguard American critical infrastructure from the unpredictable risks posed by AI systems while harnessing their transformative benefits.

The presentation underscored the profound importance of critical infrastructure—the systems and services essential for daily life, ranging from clean drinking water and energy supply to healthcare and transportation. Bawa emphasized that CISA's role is to ensure the AI assurance of these systems, meaning they perform as intended without external influence, and maintain robustness even when probed by malicious or non-malicious actors. This talk offered a comprehensive look into CISA's unique positioning, the observed patterns of AI adoption across critical sectors, the associated risks, and the proactive mitigation strategies being developed and implemented.

This topic holds significant weight as AI rapidly permeates every facet of modern society, including the foundational services upon which nations depend. Understanding how AI is being deployed, the vulnerabilities it introduces, and the frameworks being established to manage these risks is paramount for cybersecurity professionals, policymakers, and critical infrastructure operators alike. Bawa's insights provide a crucial roadmap for navigating the complex landscape of AI security in an era of accelerating technological change, highlighting the delicate balance between innovation and resilience.

Background

▶ Watch: Introducing CISA as 'Blade Runner' for digital ecosystem (0:00)

The analogy of "Blade Runners" serves as a potent framework for understanding CISA's multifaceted mission in the realm of AI and critical infrastructure. Just as the Blade Runners in the film were tasked with mitigating the unpredictability and risks posed by advanced, yet potentially dangerous, replicants to safeguard a futuristic Los Angeles, CISA aims to mitigate the inherent unpredictability and risks associated with AI systems to protect the essential services that underpin American society. This includes everything from the availability of clean water and electricity to the functionality of financial systems and emergency services.

Bawa defined unpredictability in two key dimensions: first, AI assurance, which refers to the ability to guarantee that AI systems consistently perform their intended functions as designed, free from unintended behaviors or external manipulation. Second, unpredictability encompasses the system's resilience when subjected to scrutiny or attack, whether by a malicious adversary or an accidental probing. Ensuring that AI systems remain secure and functional under these conditions is central to CISA's mandate.

CISA's unique positioning to address this challenge stems from three core pillars. Firstly, CISA is America's Cyber Defense Agency, serving as the technical and operational lead for cyber defense. Its responsibilities include identifying threats and vulnerabilities for federal civilian executive branch (FCEB) agencies and building cyber capacity across critical infrastructure data and networks nationwide. Secondly, CISA is a key implementer of President Biden's affirmative vision for trustworthy AI, as outlined in the Executive Order on Safe and Secure AI released in October 2023. This landmark executive order tasked CISA with several critical initiatives, including an operational vulnerability pilot to leverage AI for strengthening cyber defense, contributing to the development of AI red teaming methodologies, and actively protecting critical infrastructure owners and operators through the safe adoption of AI. Trustworthy AI, in this context, encompasses systems that are safe, secure, resilient, and fair.

Finally, CISA holds the crucial role of national coordinator for US critical infrastructure security and resilience, a responsibility recently reaffirmed by National Security Memorandum 22. This broad mandate covers 16 critical infrastructure sectors, which are incredibly diverse in their operational functions, security maturity, and risk profiles. These sectors include:

• Chemical
• Commercial Facilities
• Communications
• Critical Manufacturing
• Dams
• Defense Industrial Base
• Emergency Services
• Energy
• Financial Services
• Food and Agriculture
• Government Facilities
• Healthcare and Public Health
• Information Technology
• Nuclear Reactors, Materials, and Waste
• Transportation Systems
• Water and Wastewater Systems

These sectors are responsible for ensuring physical security, cybersecurity, emergency communications, and comprehensive risk management across countless organizations. To assist with this monumental task, Sector Risk Management Agencies (SRMs), such as the Department of the Treasury for the financial sector, collaborate with CISA to enhance the physical and cybersecurity posture of their respective critical infrastructure components. This extensive purview and strategic positioning underscore why CISA is uniquely equipped to navigate the complex challenges and opportunities presented by AI integration into the nation's most vital systems.

Key Findings

▶ Watch: Three reasons CISA is uniquely positioned for this mission (4:00)

CISA's examination into the adoption of AI across critical infrastructure sectors revealed both widespread integration and a nascent understanding of the associated risks. The Biden administration's Executive Order on Safe and Secure AI tasked CISA, in conjunction with Sector Risk Management Agencies (SRMs), to conduct a comprehensive inventory and catalog of AI adoption use cases and the risks they present. This effort, completed in January 2024, yielded significant insights.

From 150 distinct AI use cases reported across various sectors, CISA distilled these into 10 overarching categories of adoption. While the full list of 10 was presented visually during the talk, Bawa highlighted several of the most frequent applications, presented in descending order:

1. Operational Awareness: Utilizing AI to gain deeper insights into system status, performance, and environmental conditions.
2. Performance Optimization: Employing AI to enhance the efficiency and effectiveness of existing processes and systems.
3. Automation of Operations: Integrating AI to automate routine or complex operational tasks.
4. R&D Systems: Leveraging AI for research and development activities to foster innovation.
5. Planning: Using AI for strategic forecasting, resource allocation, and future scenario planning.
6. Physical Security: Deploying AI to enhance surveillance, access control, and threat detection in physical environments.

Examples of these applications include water utilities using AI to detect pipeline leaks and predict maintenance needs, organizations collaborating to forecast energy demand and identify potential system failures, and the US Postal Service leveraging AI to optimize mail delivery efficiency.

Several key trends emerged from this data. Firstly, the adoption clearly demonstrated AI's dual nature, presenting both significant benefits in efficiency, resilience, and predictive capabilities, alongside inherent risks. Secondly, AI is predominantly being used to support systems that were already partially automated, suggesting an incremental rather than revolutionary integration in many areas. This indicates that organizations are often extending existing automation capabilities with AI rather than building entirely new AI-driven systems from scratch in critical contexts. Finally, critical infrastructure sectors identified AI as a powerful tool to address many long-standing, persistent challenges that have historically plagued them, such as improving supply chain security and boosting cyber defense capabilities.

However, alongside these benefits, CISA identified three primary categories of risks associated with AI adoption:

1. Attacks Using AI: This category encompasses malicious actors leveraging AI technologies to enhance their offensive capabilities. A prominent example cited was the use of Large Language Models (LLMs) to craft more sophisticated, targeted, and believable social engineering campaigns, particularly phishing emails. LLMs can overcome language barriers, allowing adversaries to generate highly localized and contextually relevant malicious content, making detection significantly more challenging.
2. Attacks Targeting AI Systems: This involves direct cyber operations aimed at compromising the AI systems themselves. Targets can include sensitive training data, model weights (the learned parameters that define an AI model's intelligence), and core algorithms. Such attacks could lead to data exfiltration, model manipulation (e.g., adversarial attacks), or intellectual property theft.
3. Failures in Design and Implementation: This category covers risks arising from flaws within the AI system's lifecycle. On the design front, this includes models that exhibit bias due to skewed training data or inherent algorithmic limitations, leading to unfair or inaccurate outcomes. On the implementation side, risks stem from a lack of Subject Matter Experts (SMEs) required to properly deploy, configure, and maintain AI models as their operational context evolves, potentially leading to misconfigurations, performance degradation, or security vulnerabilities.

These findings highlight the urgent need for robust frameworks and guidelines to manage AI risks effectively across critical infrastructure, a challenge CISA is actively addressing.

Technical Deep Dive

▶ Watch: Overview of the 16 critical infrastructure sectors CISA oversees (5:20)

The three categories of risks identified by CISA—attacks using AI, attacks targeting AI systems, and failures in design and implementation—warrant a deeper technical examination to understand their implications for critical infrastructure.

Attacks Using AI primarily leverage AI's capabilities to augment existing cyberattack vectors, making them more potent and scalable. The most frequently cited example is the enhancement of social engineering attacks, particularly phishing campaigns. Traditionally, phishing relies on human ingenuity to craft deceptive messages, which can be limited by language proficiency, cultural understanding, and the sheer volume of personalized messages required for a broad attack. Large Language Models (LLMs) fundamentally transform this landscape. Adversaries can now use LLMs to:

• Generate highly convincing text: LLMs can produce grammatically correct, contextually relevant, and emotionally persuasive emails, overcoming previous linguistic barriers for non-native English speakers or those targeting specific demographics.
• Personalize at scale: By inputting victim data (e.g., from data breaches), LLMs can quickly generate thousands of unique, personalized phishing emails that appear to come from trusted sources, making them far more difficult to detect with traditional static email filters or by human recipients.
• Adapt to defenses: Sophisticated LLM-driven tools could potentially learn from failed attempts, adjusting their language and tactics to bypass evolving security measures.
• Automate reconnaissance: AI can also be used to rapidly gather open-source intelligence (OSINT) on targets, identifying key personnel, organizational structures, and potential vulnerabilities to inform more effective social engineering.

This shift means that the "human element" of cybersecurity, often the weakest link, is under unprecedented pressure, requiring more advanced training and AI-assisted defense mechanisms.

Attacks Targeting AI Systems represent a direct assault on the integrity and confidentiality of the AI models and their operational environment. These are distinct from attacks using AI, as they aim to subvert the AI itself rather than merely employ it as a tool. Key targets and attack vectors include:

• Sensitive Data: This refers to the training data used to build AI models. Attacks here can involve:
• Data Poisoning: Injecting malicious or biased data into the training set, causing the AI model to learn incorrect or harmful behaviors. In critical infrastructure, poisoned models could lead to incorrect decisions in control systems or faulty predictions in operational awareness.
• Model Inversion Attacks: Extracting sensitive information about the training data from the deployed model, potentially revealing personal identifiable information (PII) or proprietary data used during training.
• Model Weights: The "weights" are the numerical parameters that an AI model learns during its training phase, essentially encapsulating its knowledge and decision-making logic. Compromising model weights can lead to:
• Model Theft/Intellectual Property Theft: Exfiltrating the model weights allows adversaries to replicate the proprietary AI model without having to invest in their own research and development.
• Adversarial Attacks: Crafting subtly perturbed inputs that cause the model to misclassify or make incorrect predictions, often imperceptible to human observers. For example, slight modifications to sensor data could cause an AI-driven system to misinterpret a critical operational state.
• Backdooring Models: Introducing hidden vulnerabilities or specific triggers that cause the model to behave maliciously under certain conditions, which could be exploited later.
• Algorithms: Attacks targeting the underlying algorithms can exploit vulnerabilities in their design or implementation. This might involve reverse-engineering the algorithmic logic to find weaknesses, or exploiting flaws in the software libraries and frameworks used to develop the AI. Integrity of algorithms is crucial for ensuring the reliability and trustworthiness of AI-driven decisions in critical systems.

Failures in Design and Implementation highlight systemic weaknesses that can undermine AI's trustworthiness from within. These are often not malicious attacks but rather inherent flaws or operational shortcomings.

• Design-Front Failures (Bias): Algorithmic bias arises when AI models produce systematically unfair or inaccurate results for certain groups or conditions. This can stem from:
• Biased Training Data: If the data used to train the AI does not adequately represent the diversity of real-world conditions or contains historical prejudices, the AI will learn and perpetuate these biases. For example, an AI designed to predict equipment failure might perform poorly on older infrastructure types if its training data was predominantly from newer systems.
• Flawed Algorithmic Design: Even with unbiased data, the choice of algorithm or its objective function can inadvertently introduce bias if not carefully considered.
• In critical infrastructure, biased AI could lead to inequitable resource distribution, discriminatory security protocols, or unreliable operational decisions affecting specific populations or regions.
• Implementation-Front Failures (Lack of SME): The effective deployment and maintenance of AI systems require specialized expertise that is often scarce. A lack of Subject Matter Experts (SMEs) can lead to:
• Improper Deployment: AI models may be deployed in environments for which they were not designed, without adequate calibration or integration testing, leading to unpredictable behavior.
• Inadequate Monitoring and Maintenance: AI models are not static; their performance can degrade over time due to data drift (changes in input data characteristics) or concept drift (changes in the relationship between inputs and outputs). Without continuous monitoring by SMEs who understand both the AI and the operational context, these degradations can go unnoticed, leading to critical failures.
• Security Vulnerabilities: Improper configuration, lack of patching, or inadequate access controls during implementation can expose AI systems to the very attacks mentioned above.

Addressing these technical challenges requires a holistic approach, encompassing secure development lifecycles, robust validation and verification processes, and continuous operational oversight by interdisciplinary teams.

Demo / Proof of Concept

▶ Watch: Examples of AI adoption by critical infrastructure operators (6:20)

The talk focused on CISA's strategic initiatives, policy frameworks, and risk assessments concerning AI in critical infrastructure, rather than demonstrating specific technical tools or proof-of-concept exploits. As such, no live demo or detailed proof of concept was presented during Divjot Bawa's address. The emphasis was on the overarching governmental response, guidelines, and future directions for managing AI-related risks.

Defensive Implications

▶ Watch: CISA's 10 categories of AI adoption use cases (7:20)

CISA's proactive approach to mitigating AI risks in critical infrastructure culminates in the DHS/CISA Safety and Security Guidelines for AI, a pivotal document released just weeks prior to the RSA conference. This guidance is designed to equip critical infrastructure owners and operators with actionable strategies to safely and securely integrate AI into their operations. The guidelines are notable for two key characteristics: they are "addictive" and "iterative."

The "addictive" nature refers to their foundation in existing federal government guidance and resources. Critically, these guidelines operationalize the NIST AI Risk Management Framework (AI RMF), which serves as the foundational document for the US government's efforts to create trustworthy AI systems across the entire AI development lifecycle—from design and development to implementation and maintenance. By building upon the NIST AI RMF, the DHS/CISA guidelines provide a consistent and recognized structure for risk management.

The mitigation strategies within the guidelines are structured into four core categories, directly derived from the NIST AI RMF:

1. Govern: This category focuses on establishing a robust governance structure for AI use. It involves defining clear roles, responsibilities, and accountability for AI systems; developing organizational policies, standards, and procedures for AI deployment; and fostering an organizational culture that prioritizes AI safety, security, and ethical considerations. For critical infrastructure, this means embedding AI risk management into existing enterprise risk frameworks.
2. Map: This involves identifying and characterizing the AI risks and their potential impacts. Organizations must comprehensively understand their AI systems, including their purpose, data sources, design choices, operational environment, and potential failure modes. This mapping exercise helps to identify specific vulnerabilities, potential biases, and the criticality of the AI system to overall operations.
3. Measure: This category emphasizes developing metrics and methods to assess, analyze, and monitor AI risks. It calls for continuous evaluation of AI system performance, security posture, and adherence to ethical principles. For critical infrastructure, this could involve developing specific KPIs for AI system reliability, accuracy, and resilience against adversarial attacks, as well as establishing monitoring frameworks to detect anomalies.
4. Manage: This final category focuses on implementing risk mitigation strategies and controls. Based on the mapping and measurement of risks, organizations must deploy appropriate technical, operational, and administrative controls to reduce identified risks to an acceptable level. This includes implementing secure development practices, robust testing and validation, incident response plans for AI failures, and ongoing maintenance and updates.

The "iterative" aspect of the guidelines highlights their dynamic and collaborative development. They benefited from extensive inter-agency input, including contributions from other Sector Risk Management Agencies (SRMs), NIST, the Department of Commerce, and various regulatory commissions. This broad collaboration ensures the guidelines are comprehensive, practical, and reflect a consensus across diverse stakeholders.

Looking forward, CISA's work in this domain is continuous. The Executive Order on Safe and Secure AI mandates annual cross-sector risk assessments to track the evolving threat environment and adapt the guidelines accordingly. These guidelines will also serve as a crucial launchpad for future actions, notably for the newly established DHS AI Safety and Security Board, which has adopted critical infrastructure as its primary use case. As critical infrastructure owners and operators begin to implement these guidelines, CISA anticipates identifying implementation gaps, which will inform the need for additional guidance, analysis, and targeted actions.

Ultimately, the defensive implications underscore a proactive, collaborative, and adaptable strategy. Defenders in critical infrastructure must embrace these guidelines, integrate them into their operational security practices, and actively participate in the ongoing dialogue to ensure that AI's transformative benefits are realized without compromising the safety and resilience of essential services. The overarching message is one of healthy respect for AI's potential, coupled with a vigilant focus on mitigating its risks to prevent the dystopian outcomes envisioned in science fiction.

Key Takeaways

• CISA as the "Blade Runner" for AI in Critical Infrastructure: CISA is uniquely positioned and actively working to mitigate the unpredictable risks posed by AI systems to safeguard essential American critical infrastructure, drawing a compelling parallel to the sci-fi classic.
• Widespread AI Adoption with Identified Use Cases: An inventory conducted in January 2024 revealed 150 distinct AI use cases across critical infrastructure, categorized into 10 areas, with Operational Awareness, Performance Optimization, and Automation of Operations being the most frequent.
• Three Primary Categories of AI Risk: CISA identified risks stemming from attacks using AI (e.g., LLM-enhanced social engineering), attacks targeting AI systems (e.g., compromising model weights or sensitive data), and failures in design and implementation (e.g., model bias or lack of SME).
• DHS/CISA Safety and Security Guidelines as a Mitigation Framework: The recently released guidelines, built upon the NIST AI Risk Management Framework (AI RMF), provide a structured approach to managing AI risks across four key categories: Govern, Map, Measure, and Manage.
• Iterative and Collaborative Approach: The guidelines are the product of extensive inter-agency input and are designed to be dynamic, with annual risk assessments and continuous refinement planned to adapt to the evolving AI threat landscape.
• Balanced Perspective on AI: CISA advocates for a healthy respect for AI's transformative benefits while maintaining a vigilant focus on proactive risk mitigation, ensuring that AI integration enhances rather than endangers critical infrastructure.

About the Speaker(s)

Divjot Bawa is a Cyber-AI Policy Fellow at CISA (Cybersecurity and Infrastructure Security Agency). In this role, Bawa is at the forefront of shaping policy and strategic initiatives related to the intersection of artificial intelligence and cybersecurity, specifically within the context of critical infrastructure protection. His work involves understanding the evolving landscape of AI adoption, identifying associated risks, and developing comprehensive frameworks and guidance to ensure the safe, secure, and resilient integration of AI technologies across vital national systems.