Modus OperandAI: Practical Security for Artificial Intelligence

Modus OperandAI: Practical Security for Artificial Intelligence

Speakers: Rory Boyce Warner, Deputy CISO, Southern New Hampshire University

Conference: RSAC 2024

YouTube: https://www.youtube.com/watch?v=X-XG755LbDU

Overview

This talk, "Modus OperandAI: Practical Security for Artificial Intelligence," delivered at RSAC 2024, delves into the real-world challenges and proactive strategies adopted by Southern New Hampshire University (SNHU) in securing their rapidly expanding use of artificial intelligence. Presented by Rory Boyce Warner, Deputy CISO of SNHU, the session outlines the university's journey to integrate AI responsibly following the widespread emergence of technologies like ChatGPT in late 2022. Unlike many institutions focused solely on preventing AI misuse, SNHU embraced AI as a tool to enhance its mission of making higher education accessible and affordable, leading to a unique set of security imperatives.

The presentation provides an "honest look" at the experience of a security team navigating an organization that is "moving fast and breaking things" in the realm of AI adoption. It highlights the scramble faced by SNHU's Information Security Office (ISO) to define and implement practical security measures in the absence of established industry standards from bodies like NIST or OWASP specific to AI at the time. The talk aims to share the practical insights and lessons learned from their journey, from initial discussions in September 2023 to the present, focusing on the development of new capabilities and the adaptation of existing security processes to meet the demands of an AI-driven environment.

The insights shared are particularly relevant for security professionals grappling with the rapid integration of AI within their own organizations. SNHU's approach underscores the necessity for agility, innovation, and a pragmatic mindset in addressing emerging technological risks. The discussion frames AI security not merely as a technical challenge but as an organizational imperative, requiring a deep understanding of institutional mission, operational context, and a willingness to challenge traditional security paradigms.

Background

▶ Watch: Introduction to Modus Operandi: Practical AI Security (0:00)

Southern New Hampshire University (SNHU) operates with a distinct mission: to transform lives at scale by making higher education accessible and affordable to everyone. This mission is underpinned by two core principles: being learner-centric, where every decision supports student success and experience, and being willing to challenge the status quo of higher education through innovation and adaptability. While SNHU maintains a traditional on-campus experience for approximately 3,000 students, its primary focus is on non-traditional learners—working adults, military personnel and veterans, military spouses, and working parents. This demographic necessitates flexible, often asynchronous online learning solutions.

SNHU’s pioneering spirit in online education dates back to the early 2000s, a period when only about 40% of American households had internet access. Rory Boyce Warner herself is a testament to SNHU's model, having leveraged their online programs as a single mother working full-time to earn her bachelor's degree in 2003. Under the leadership of a forward-thinking president, SNHU recognized asynchronous online courses and degree programs as the future of higher education and a critical means to fulfill its mission. Today, this vision has materialized into a vast online learning ecosystem serving over 200,000 learners globally.

The landscape shifted dramatically in late 2022 with the public release of ChatGPT, igniting what many consider the AI revolution. While numerous universities grappled with how to prevent students from using AI for academic misconduct, SNHU adopted a fundamentally different approach. Consistent with its mission to innovate and improve learner experience, SNHU actively sought to explore how this emerging technology could be leveraged to streamline operations, enhance student support, and ultimately better achieve its core mission. This proactive stance, however, presented an immediate and significant challenge for the university's Information Security Office (ISO). The ISO found itself "scrambling" to understand and address the security implications of AI adoption, a task made more complex by the nascent state of AI security standards. At this critical juncture, established frameworks from organizations like NIST and OWASP had not yet fully matured to provide comprehensive guidance specifically for AI security, leaving SNHU's security team to forge their own path in a rapidly evolving technological environment.

Key Findings

▶ Watch: SNHU's historical embrace of innovative education methods (2:18)

Given the rapid adoption of AI technologies like ChatGPT across Southern New Hampshire University (SNHU), the Information Security Office (ISO) quickly identified several critical areas that required immediate attention and strategic development. While the transcript largely outlines the intent and journey of SNHU's security team rather than specific technical findings, it implicitly reveals key realizations and priorities.

Firstly, a primary finding was the necessity for proactive engagement rather than reactive containment. Instead of simply restricting AI use, SNHU recognized the strategic imperative to integrate AI securely into its operations. This meant the security team had to pivot from a traditional gatekeeping role to one of enablement, working closely with business units to understand their AI initiatives and embed security from the outset. This finding highlighted the importance of being an organizational partner rather than a blocker, particularly in an environment "moving fast and breaking things."

Secondly, the ISO quickly understood the absence of mature, standardized AI security frameworks. In late 2022 and early 2023, there were no comprehensive NIST or OWASP standards specifically tailored for AI security. This forced SNHU to develop its own foundational capabilities and processes. This finding underscored the need for internal expertise development and the creation of bespoke security controls, acknowledging that generic cybersecurity practices might not adequately cover the unique risks posed by AI, such as data poisoning, model inversion, or prompt injection.

Thirdly, the experience revealed the criticality of developing new security capabilities and adapting existing ones. The security implications of AI extend beyond traditional data security or network perimeter defense. They touch upon data governance for training data, ethical AI use, model integrity, and the security of AI supply chains. The "scramble" described by Rory Boyce Warner indicates a rapid learning curve and an urgent need to build expertise in areas previously less emphasized within a university's security program. This includes understanding machine learning lifecycles, data science methodologies, and the specific attack surfaces introduced by AI models and their integration points.

Finally, a significant realization was the dynamic nature of AI security. The journey from September 2023 to the present, as described, is one of continuous adaptation. The security landscape for AI is not static; new threats, vulnerabilities, and defensive techniques emerge constantly. This implies that any "findings" are not endpoints but rather milestones in an ongoing process of assessment, development, and refinement. SNHU's experience suggests that flexibility and a commitment to iterative improvement are paramount for effective AI security.

Technical Deep Dive

▶ Watch: SNHU's unique approach to ChatGPT adoption (2:45)

The provided transcript, serving as an introduction to the panel discussion, does not delve into specific technical details, code examples, or architectural diagrams related to SNHU's AI security implementation. Instead, it sets the stage for a broader discussion on the approach to practical AI security within a fast-paced organizational context. The talk's title, "Modus OperandAI: Practical Security for Artificial Intelligence," strongly implies that the subsequent discussion would cover concrete technical and procedural strategies.

Based on the stated challenges—"scrambling to figure out what the security implications of this new technology" were, "before any NIST standards, any OWASP standards"—it can be inferred that a technical deep dive would likely cover areas such as:

1. AI Model Security: This would involve discussions around securing the integrity and confidentiality of AI models themselves. This could include strategies to prevent model poisoning (where malicious data is introduced into training datasets to compromise model behavior), model inversion attacks (where an attacker attempts to reconstruct sensitive training data from model outputs), and adversarial attacks (subtly crafted inputs designed to cause a model to misclassify or behave unexpectedly).
2. Data Security for AI: Given SNHU's learner-centric mission and handling of sensitive student data, securing the data used for training, inference, and fine-tuning AI models would be paramount. This would likely involve robust data governance frameworks, data anonymization or pseudonymization techniques, access controls for data lakes and model repositories, and secure data pipelines.
3. Prompt Engineering and Input Validation: With the rise of large language models (LLMs) like ChatGPT, prompt injection attacks have become a significant concern. A technical deep dive would likely explore strategies for robust input validation, sanitization, and potentially the use of guardrails or content filters to mitigate the risks associated with malicious or manipulative prompts. This could also extend to securing the integrity of prompts themselves, ensuring they are not tampered with.
4. Secure AI Deployment and Integration: This would cover the secure deployment of AI applications, whether hosted in cloud environments or on-premises. It would likely involve discussions on API security for AI services, secure containerization strategies, identity and access management (IAM) for AI components, and secure integration with existing enterprise systems.
5. Monitoring and Observability for AI: Detecting anomalies and malicious activities within AI systems requires specialized monitoring. This could involve tracking model performance deviations, unusual data access patterns, and suspicious model outputs. Techniques like explainable AI (XAI) might be leveraged to understand model decisions and identify potential biases or security issues.

While the provided segment does not elaborate on these technical aspects, the speaker's emphasis on developing "capabilities" and changing "processes" strongly suggests that the full talk would have detailed the practical implementation of controls across these technical domains. The security team's proactive stance in the absence of external standards implies an internal development of best practices, potentially leveraging existing security principles adapted to the unique characteristics of AI systems.

Demo / Proof of Concept

▶ Watch: Overview of the panel's practical AI security journey (3:30)

The provided transcript serves as the introductory segment to the panel discussion, outlining the context and objectives of the talk. Within this initial two-minute portion, there is no mention or presentation of a live demonstration or a proof of concept (PoC). The speaker, Rory Boyce Warner, focuses on introducing Southern New Hampshire University, its mission, the organizational context for AI adoption, and the challenges faced by its Information Security Office in securing these emerging technologies. Therefore, based on the available information, no demo or PoC was presented during this segment of the talk.

Defensive Implications

▶ Watch: Introducing Elizabeth Hubbard, first panelist (3:55)

Southern New Hampshire University's (SNHU) proactive journey into securing artificial intelligence offers several critical defensive implications for organizations grappling with similar challenges. The experience shared by Rory Boyce Warner underscores the necessity for security teams to adopt an agile, anticipatory, and deeply integrated approach to AI security, rather than a reactive or prohibitive one.

Firstly, the most significant implication is the call for proactive security engagement in the face of rapid technological adoption. SNHU's decision to embrace AI to enhance its mission, rather than merely prevent its misuse, meant its Information Security Office (ISO) had to become a strategic enabler. Defenders should embed themselves early in AI development lifecycles, collaborating with data scientists, developers, and business units. This ensures that security considerations are designed into AI systems from the ground up, addressing potential risks like data poisoning, model integrity issues, and prompt injection vulnerabilities before they become systemic problems.

Secondly, the talk highlights the imperative to develop bespoke AI security capabilities and processes in the absence of mature, standardized frameworks. When SNHU began its journey, comprehensive NIST or OWASP standards for AI security were not yet fully established. This means organizations cannot solely rely on traditional cybersecurity playbooks. Defenders must invest in upskilling their teams in machine learning fundamentals, AI-specific attack vectors, and responsible AI principles. They should start creating their own internal guidelines, risk assessment methodologies, and control frameworks tailored to the unique risks of their AI deployments, covering aspects such as secure data pipelines, model validation, and ethical considerations.

Thirdly, SNHU's experience reveals the importance of organizational agility and adaptability. The "scramble" described by the Deputy CISO reflects the reality of securing innovation within a "fast and breaking things" culture. Defenders must cultivate an environment where security practices can evolve rapidly to match the pace of technological change. This involves implementing iterative security reviews, fostering continuous feedback loops with AI development teams, and being prepared to rapidly adjust defensive strategies as new AI threats and vulnerabilities emerge.

Fourth, the emphasis on SNHU's mission to be "learner-centric" suggests a defensive implication around responsible AI and ethical considerations. For organizations handling sensitive data or making impactful decisions with AI, defenders must consider not just technical vulnerabilities but also the ethical implications of AI use, including bias, fairness, transparency, and privacy. Security frameworks should extend to include mechanisms for auditing AI models for bias, ensuring data privacy in training datasets, and maintaining transparency in AI decision-making processes, thereby building trust and mitigating reputational risks.

Finally, the talk implicitly advocates for internal advocacy and education. Rory Boyce Warner's personal story and SNHU's mission-driven approach suggest that effective AI security is not just about technical controls but also about fostering a security-aware culture. Defenders should educate stakeholders across the organization—from leadership to end-users—about the risks and responsible use of AI, empowering everyone to contribute to a secure AI ecosystem. This includes training on secure prompt engineering, identifying AI-related phishing attempts, and understanding data handling best practices for AI applications.

Key Takeaways

• Proactive AI Security is Essential: Organizations should embrace AI as an enabler for their mission and proactively integrate security from the outset, rather than reactively trying to prevent its use or mitigate risks after deployment.
• Adaptation in the Absence of Standards: Be prepared to develop internal AI security capabilities and processes when external industry standards (e.g., NIST, OWASP for AI) are still nascent or insufficient for specific organizational contexts.
• Security as an Enabler, Not a Blocker: Security teams must shift from a gatekeeping role to one of strategic partnership, collaborating closely with business units that are "moving fast and breaking things" to ensure secure innovation.
• Invest in New Capabilities: AI introduces unique attack surfaces and risks, requiring organizations to invest in upskilling security personnel and developing new technical and procedural controls beyond traditional cybersecurity frameworks.
• Organizational Agility is Paramount: The rapid evolution of AI technology demands that security strategies and practices be agile, adaptable, and capable of continuous iteration and improvement.
• Mission-Driven Security: Aligning AI security efforts with the organization's core mission and values (e.g., SNHU's learner-centric approach) can provide a strong foundation for responsible and effective AI adoption.

About the Speaker(s)

Rory Boyce Warner is the Deputy CISO of Southern New Hampshire University (SNHU). Her professional background in IT administration and her personal journey as a single mother who leveraged SNHU's online programs to earn her degree provide her with a unique perspective on the university's mission and the practical application of technology. She has firsthand experience with SNHU's commitment to making higher education accessible and affordable, having benefited from their early asynchronous online course offerings. As Deputy CISO, she leads SNHU's Information Security Office, navigating the complex security landscape, particularly concerning emerging technologies like artificial intelligence, to support the university's innovative and learner-centric approach.

Elizabeth Hubbard was introduced by Rory Boyce Warner as the first panelist for the session. However, the provided transcript does not include further biographical details or her specific role within SNHU or another organization.