Beyond the Buzz: SBOMs, AI, and DataOps for Organizational Resilience in a Post-Log4j World

Beyond the Buzz: SBOMs, AI, and DataOps for Organizational Resilience in a Post-Log4j World

Speakers: Jessie Jamieson

Conference: BSides NYC

YouTube: https://www.youtube.com/watch?v=yogmzTXsphk

Overview

In an era defined by escalating supply chain attacks and the rapid proliferation of artificial intelligence, Jessie Jamieson's talk at BSides NYC, "Beyond the Buzz: SBOMs, AI, and DataOps for Organizational Resilience in a Post-Log4j World," delivers a critical message: the foundational principles of data operations are paramount to achieving true organizational resilience. The presentation argues that while technologies like Software Bill of Materials (SBOMs) and AI tools are gaining significant traction, their effectiveness is severely hampered by inadequate underlying data practices. Jamieson posits that without a robust approach to data collection, logging, modeling, and contextualization, organizations will fail to extract meaningful intelligence from these tools, leaving them vulnerable to sophisticated threats.

Jamieson, a mathematician and systems thinker, challenges the industry to move beyond superficial adoption of new security trends and instead focus on the "unsexy" but indispensable work of data management. Her thesis is clear: the ability to effectively counter modern cyber threats, from supply chain compromises like Log4j to the emerging risks posed by AI-driven malware, hinges on an organization's maturity in data operations (DataOps). The talk underscores that vulnerabilities are increasingly introduced "far left of effect" and at scale, necessitating a proactive and data-centric approach to understanding complex interdependencies within an organization.

This article delves into Jamieson's compelling argument, exploring how organizations can leverage data science fundamentals to transform raw data, such as that found in SBOMs, into actionable intelligence. It also examines the inherent risks of deploying AI and machine learning models without stringent data quality and transparency measures, drawing parallels to real-world failures in medical diagnostics and autonomous vehicles. Ultimately, the talk provides a roadmap for defenders to build resilient systems by prioritizing diligent data practices, rigorous process red teaming, and a deep understanding of their organizational ecosystem.

Background

▶ Watch: Introduction to supply chain vulnerabilities and their scale (0:00)

The landscape of cybersecurity has fundamentally shifted, characterized by the introduction of vulnerabilities "far left of effect" and at an unprecedented scale. Supply chain compromises, as defined by CISA, occur when adversaries "insert malicious code into legitimate software, compromise software build processes, or deliver tainted updates." High-profile incidents like SolarWinds, NotPetya, and the widespread Log4j vulnerability have starkly illustrated how a single point of failure deep within the software supply chain can cascade into global disruptions, affecting countless organizations that consume the compromised software. These attacks exploit the complex web of dependencies inherent in modern software development and deployment.

Concurrently, the rise of automation and artificial intelligence (AI) has introduced a new dimension of risk. While AI promises enhanced defensive capabilities, it also enables adversaries to develop and deploy malware, including ransomware, at an accelerated pace. The plenary talk at BSides NYC highlighted the continuous rise of ransomware, underscoring the urgency for organizations to adapt their defenses. However, Jamieson argues that merely adopting new technologies without addressing underlying data deficiencies is akin to building a house on sand.

The core problem, as articulated by Jamieson, is a systemic failure in data operations (DataOps). Organizations are often ill-equipped to collect, process, contextualize, and derive intelligence from their data effectively. This deficiency impacts the utility of critical security tools. For instance, Software Bill of Materials (SBOMs), designed to provide a formal, machine-readable inventory of software components and their hierarchical relationships, are widely recognized but rarely fully utilized. A poll conducted by Jamieson at the beginning of her talk revealed that while many were familiar with SBOMs, very few actually used them in practice. This disconnect highlights the gap between recognizing the value of a security mechanism and possessing the operational maturity to implement it effectively. Jamieson's talk serves as a call to action, urging the security community to re-evaluate its approach to data as the bedrock of all advanced security capabilities.

Key Findings

▶ Watch: Speaker's core thesis: DataOps crucial for new cyber tech (2:20)

Jessie Jamieson's presentation unearths several critical findings that challenge conventional wisdom in cybersecurity:

1. SBOMs are necessary but insufficient on their own: While SBOMs provide a foundational inventory of software components, they are "pretty Bare Bones" and "meant to be machine readable not necessarily human readable." Jamieson asserts that "no set of s-bombs... is going to tell us anything about our organizational Risk by themselves." Extracting meaningful intelligence requires significant additional work in data operations and contextualization.
2. Context is paramount for data utility: The true value of data, whether from SBOMs or other sources, lies in its context. Understanding how software components underpin critical services and products, and how vulnerabilities might impact these, is essential. This necessitates mapping dependencies across an entire organizational ecosystem.
3. Organizational dependencies are measurable network structures: Jamieson, a mathematician, emphasizes that the relationships between services, applications, software components, and even people within an organization form network structures (or dependency trees). These structures can be "measured, studied, and modeled" using principles from graph theory, providing a powerful source of data and insight for incident response and risk analysis.
4. Organizations struggle with counterfactual analysis: People are "extremely bad at counterfactual analysis," or asking "what-if" questions and accurately predicting outcomes, particularly in abstract policy or security scenarios. Data-driven models can put individuals in simulated scenarios, forcing them to confront and defend their decisions, thereby improving learning and preparedness.
5. Poor data quality is a critical failure point for AI/ML: Jamieson highlights numerous real-world examples where AI and machine learning models, even those intended for critical applications, failed catastrophically due to flaws in their training data. These include COVID-19 diagnostic tools, self-driving cars, and large language models (LLMs). The models often learn spurious correlations from poor or biased data, leading to incorrect or dangerous conclusions.
6. Lack of transparency and policy for AI/ML poses significant risks: The "black box" nature of many advanced AI models, coupled with a lack of policy mandating minimal effectiveness, transparency, or accountability, creates a "scary scenario." Jamieson points to recent revelations about LLM training data (e.g., Google C4 and Facebook's Llama using U.S. voter registration data) as deeply concerning, raising questions about unintended uses and privacy implications.
7. Investing in "unsexy" data problems is foundational: To effectively leverage cutting-edge tools like SBOMs and AI, organizations must first address the "unsexy problems" of diligent data collection, logging, cleaning, and modeling. These fundamental data operations are the prerequisite for achieving true organizational resilience and deriving actionable intelligence.

Technical Deep Dive

▶ Watch: The hard truth: SBOMs alone don't reveal organizational risk (4:30)

The core of Jamieson's technical argument revolves around transforming the raw, machine-readable data from Software Bill of Materials (SBOMs) into actionable intelligence through sophisticated data operations (DataOps) and dependency modeling.

An SBOM is formally defined as a "machine readable inventory of software components or dependencies" that provides information about these components and their hierarchical relationships. Standard formats like SPDX provide a general structure, but as Jamieson notes, they are "pretty Bare Bones." By themselves, SBOMs do not convey organizational risk. To bridge this gap, organizations must actively build dependency trees or network structures.

Jamieson illustrates this with a simple example: an organization providing a "service monitoring capability." This capability might depend on a Python application developed internally and a logging service. Each of these applications, in turn, depends on underlying software packages (e.g., pip install packages for Python) and logging software written in different languages. This forms a clear hierarchical dependency.

Expanding on this, Jamieson, drawing on her mathematical background, explains that these are inherently network structures that can be "measured, studied, and modeled." She references the JARM 10-layer model as an example of a more complex framework for modeling network and cyber dependencies, suggesting that organizations can customize these models to their specific needs, extending beyond just software to include hardware components and intermixed layers.

The mathematical representation of these dependencies is a directed acyclic graph (DAG). In such a graph, nodes represent entities like services, capabilities, applications, software components, people, or even approval processes. Edges represent the relationships or flows between these nodes – for instance, a service depends on an application, or information is passed from one person to another for approval. Jamieson describes how such a model could become a "social network" when incorporating people, with edges representing "information, the approvals, the stamps, the requirements that have to be satisfied in order to proceed from one step of the process to the next."

Building these models for large organizations is resource-intensive, requiring extensive interviews to understand who talks to whom and what software they use. Jamieson mentions her prior work at the Applied Physics Lab (APL) where a proprietary system called Dagger (where DAG stands for Directed Acyclic Graph) was developed for this purpose. She notes that open-source papers exist on Dagger, and mathematically, there's nothing preventing organizations from implementing dynamic, time-dependent DAGs using tools like Python, where nodes and edges change based on actions or events. This dynamism allows for simulating changes, such as a critical team member's absence, to understand organizational resilience.

The second major technical area addressed is the critical importance of data quality for Artificial Intelligence (AI) and Machine Learning (ML) systems, and the inherent vulnerabilities when this is overlooked. Jamieson provides several stark examples:

1. COVID-19 Diagnostic Algorithms: A study summarized in the MIT Tech Review revealed that 232 AI algorithms developed to diagnose COVID-19 or predict patient outcomes were "none... fit for clinical use." A key reason was poor data quality. One algorithm, designed to classify X-ray or CT scan images, learned to associate patients sitting upright with better outcomes, and those lying down with severe illness. This was not because sitting up indicated health, but because the training data tags implicitly linked patient posture during the scan to their condition (severely ill patients couldn't sit up). The AI learned a spurious correlation from uncleansed data.
2. Self-Driving Uber Accident: The fatal incident involving a self-driving Uber was partly attributed to the car's inability to classify an object as a pedestrian unless it was near a crosswalk. The training data evidently lacked examples of jaywalking pedestrians, a common occurrence in urban environments like New York City, leading to a critical gap in the model's understanding of real-world scenarios.
3. Google Bard's Factual Error: Google's public demonstration of its Bard LLM made a factual error about the James Webb Space Telescope (JWST), specifically about the first exoplanet images. This simple error, which could have been easily verified via a basic Google search, cost the company an estimated $100 billion in market value, highlighting a lack of robust verification in the data pipeline or model output.
4. Adversarial Attacks on Image Classification: Jamieson presents an example from a math paper where an image of a Bernese mountain dog, correctly classified by an ML algorithm with 73% confidence, was perturbed by a "strategically chosen noise pattern." This imperceptible noise caused the algorithm to reclassify the dog as a "golf cart" with 98.9% confidence. This demonstrates adversarial attacks, where subtle input modifications can trick ML models, a significant concern for cybersecurity applications.
5. LLM Training Data Transparency: Jamieson highlights a recent Washington Post article (published two days before her talk) revealing that Google's C4 and Facebook's Llama LLM training data sets included the "entire voter registration of the United States." This raises profound questions about privacy, ethical data sourcing, and the potential for misuse, emphasizing the critical need for transparency and policy around AI training data.

These technical examples collectively underscore Jamieson's core message: without meticulous attention to data quality, contextual modeling, and transparency, even the most advanced technologies will fall short, potentially introducing new and unpredictable risks.

Demo / Proof of Concept

▶ Watch: Leveraging SBOMs to understand organizational vulnerability and impact (5:00)

While the talk did not feature a live software demonstration or a specific proof-of-concept tool in action, Jessie Jamieson extensively discussed the conceptual framework for building these dependency models and the type of output they generate. Rather than a visual, interactive demo, the "proof of concept" lies in the methodology and the tangible, albeit "boring," results it produces.

Jamieson explains that the end output of this rigorous modeling process is typically a "pretty boring list of recommendations." However, the crucial distinction is that these recommendations are data-driven and informed by a model that is based on their actual data. This analytical rigor leads to a critical outcome: because stakeholders have agreed on the baseline assumptions and the underlying data of the model, they reach a point where they can confidently say, "I get this, you're right, I believe you." This consensus and trust in the derived intelligence are the true "proof of concept" for the methodology.

A key aspect of this framework is its utility in counterfactual analysis. Jamieson references her work on an IRPA FOCUS project, which studied how well people perform "what-if" questions. She notes that humans are "extremely bad at counterfactual analysis," especially concerning abstract concepts like policy. By having a data-driven model of people and processes, organizations can "actually show them and put them in that scenario and say now let's assume you made that decision or you didn't make that decision how would the outcomes have been." This forces individuals to make decisions within the simulated environment and defend them, leading to deeper learning and more robust planning. The recommendations derived can be "counterfactual things where we can go back and actually make changes and then say here's what you should have done or here's what you could have done or here's you know the lessons learned from this entire procedure."

The process itself, involving months of interviews and data collection, serves as a discovery mechanism. As one audience member pointed out, this process can reveal "Shadow IT" or gaps in observability. Jamieson acknowledges that some risks, like Shadow IT, might remain "Known Unknowns" (to use Donald Rumsfeld's terminology). In such cases, the model can incorporate probability distributions to estimate the range or likelihood of impact, providing upper and lower bounds for risk quantification, even in the face of incomplete information. This iterative discovery and refinement, driven by the modeling effort, is a continuous "proof" of its value.

Defensive Implications

▶ Watch: Practical example: Building a dependency tree from services to software (6:00)

Jamieson's talk provides critical defensive implications across two main areas: leveraging SBOMs for supply chain resilience and safely adopting AI/ML technologies.

For SBOMs and Supply Chain Resilience:

1. Invest in Red Teaming Processes, Not Just Components: A fundamental shift is needed from solely red teaming software components to rigorously red teaming organizational processes. This involves examining how people and processes respond to simulated attacks, identifying weaknesses in communication, decision-making, and procedural execution. As Jamieson states, "these processes are the steps by which your people your organization take to make the important decisions for how they respond to these things."
2. Diligently Collect and Log Process Metadata: Organizations must be "diligent and deliberate with respect to the data we collect and log about our internal processes." This goes beyond technical logs to include metadata about who talks to whom, who is critical to specific procedures, and how information flows. This rich contextual data is essential for building accurate dependency models.
3. Integrate Dependency Modeling into Exercises and Playbooks: SBOMs and the resulting dependency models should serve as a "great source of Truth and data" to inform and validate incident response exercises and playbooks. By agreeing on baseline assumptions derived from these models, organizations can ensure that the impacts observed during red team exercises or simulations are considered real and measurable, leading to more effective adjustments.
4. Address Organizational Silos: The process of building dependency models often exposes dysfunctional organizational silos. While some siloing might be justified, the modeling process can force teams to collaborate and highlight where lack of communication or insight poses a significant risk. This can lead to process improvements that inherently foster better cross-functional understanding.
5. Improve Observability through Discovery: Embarking on the dependency modeling journey can itself lead to improvements in observability. As organizations attempt to map their systems, they often discover critical gaps in their monitoring capabilities, prompting investments in better tools and practices.
6. Account for "Known Unknowns": For risks like Shadow IT, which by definition lack full observability, defensive strategies must acknowledge them as "Known Unknowns." Modeling can incorporate probabilistic elements to estimate potential impacts, providing ranges or probability distributions for risk, even if precise quantification is impossible. This allows for informed decision-making despite uncertainty.

For AI and Machine Learning Adoption:

1. Proactive Risk Assessment of AI Tools: Before implementing any AI or ML tool, organizations must conduct a thorough risk assessment. This includes understanding the potential vulnerabilities of the model, the provenance and quality of its training data, and its potential for unintended consequences. This applies not just to cutting-edge LLMs but to any new technology.
2. Demand Transparency and Traceability: Defenders should advocate for and demand greater transparency from AI vendors regarding their models' inner workings, training data, and decision-making processes. The "black box" nature of many AI systems, coupled with a lack of traceability, makes them inherently risky for security applications.
3. Advocate for Policy and Standards: Jamieson emphasizes the urgent need for policies mandating minimal effectiveness, transparency, and accountability for AI algorithms, similar to clinical trial structures for pharmaceuticals. Without such regulatory frameworks, the widespread deployment of unvetted AI poses significant societal and organizational risks.
4. Prioritize "Unsexy" Data Foundations: The most impactful defensive measure for AI is to focus on the fundamental, often overlooked, challenges of data. This means investing in robust data collection, cleaning, pre-processing, and governance. "Don't let your doggos become golf carts" is Jamieson's memorable reminder that flawed data leads to flawed AI.
5. Differentiate Situational Awareness from Understanding: Organizations must strive for "situational understanding" (understanding what you know) rather than just "situational awareness" (knowing what you know). This higher level of comprehension comes from deeply analyzing data, modeling relationships, and testing hypotheses, which is crucial for making informed defensive decisions in a complex threat landscape.

Key Takeaways

• Data Operations are Foundational: True organizational resilience in the face of supply chain attacks and AI-driven threats hinges on robust, diligent data operations. Without a strong data foundation, advanced tools like SBOMs and AI will fail to deliver their full potential.
• SBOMs Require Deep Contextualization: Raw SBOMs are merely an inventory; they must be integrated into comprehensive dependency models (network structures/DAGs) that map critical services, applications, and even people to derive actionable intelligence about organizational risk.
• AI/ML is Only as Good as Its Data: Flawed, biased, or incomplete training data leads to critical failures in AI/ML systems, as evidenced by real-world incidents in medical diagnostics, autonomous vehicles, and large language models. Organizations must prioritize data quality.
• Red Team Processes, Not Just Components: Organizations should invest in red teaming their internal processes and data flows, not just individual software components. This reveals weaknesses in human decision-making, communication, and procedural responses to incidents.
• Demand Transparency and Accountability for AI: The "black box" nature of many AI models and the lack of policy regarding their training data and effectiveness pose significant risks. Defenders must advocate for greater transparency, traceability, and regulatory oversight for AI adoption.
• Invest in the "Unsexy" Work: The most critical investments for future resilience lie in the often-overlooked areas of diligent data collection, logging, cleaning, and modeling, rather than solely chasing the latest buzzwords in security technology.

About the Speaker(s)

Jessie Jamieson is a distinguished mathematician and systems thinker with a keen focus on the intersection of data science and cybersecurity. Her professional background includes significant work at the Applied Physics Lab (APL), where she was involved in developing proprietary systems for dependency modeling, such as Dagger (a system for directed acyclic graphs). During her time at APL, she also contributed to projects like IRPA FOCUS, which studied human capabilities in counterfactual analysis, highlighting her interest in how data can inform decision-making under uncertainty.

More recently, Jamieson has transitioned to the private sector, joining Tenable (as of May, as mentioned in the talk). Her expertise spans various domains, including developing machine learning solutions for Security Operations Centers (SOCs) to improve ticket prioritization. Jamieson positions herself as a "people process person" and a "systems thinker," emphasizing the importance of organizational dynamics and data flows over mere technical components. Her insights are also informed by her personal life, as she humorously mentions her husband's work on the data pipeline for the James Webb Space Telescope, demonstrating a deep appreciation for rigorous data practices across scientific and technical fields.