Barbarians at the Gate: Visualizing and Blocking SDLC Infrastructure Threats with SITF

Barbarians at the Gate: Visualizing and Blocking SDLC Infrastructure Threats with SITF

Speakers: S Berkovich (Google)

Conference: fwd:cloudsec

YouTube: https://www.youtube.com/watch?v=cBq-DRNNiA

Overview

In this compelling talk at fwd:cloudsec, Shay Berkovich from Google (formerly of the WH Threat Research Group) introduced the SDLC Infrastructure Threat Framework (SITF), a novel approach to understanding, visualizing, and defending against the escalating wave of attacks targeting the Software Development Life Cycle (SDLC). Berkovich highlighted a stark reality: more SDLC infrastructure has been compromised in the past 18 months than in the preceding decade combined. This alarming trend, fueled by the ease of AI-assisted payload development and the increasing sophistication of threat actors, demands a fundamental shift in defensive strategies.

The SITF framework addresses critical shortcomings in traditional security models, particularly the prevailing "consumer bias" in supply chain security, which often overlooks the producer's perspective and the initial compromise vectors. By offering a multi-architectural, practical, and real-world anchored framework, SITF aims to provide security teams with a common language and actionable insights to combat these complex threats. The talk included live demonstrations of SITF's visualizer and its cutting-edge AI integration, showcasing how the framework can rapidly map attack flows and identify relevant controls.

Berkovich's presentation underscores the urgent need for a holistic understanding of SDLC attack surfaces, recognizing that "the barbarians are not just at the gate" but can emerge from any point within the development ecosystem. SITF positions itself as an indispensable tool for threat modeling, tabletop exercises, and continuous improvement of SDLC security posture, empowering defenders to move beyond abstract concepts to concrete, implementable protections against an ever-evolving threat landscape.

Background

▶ Watch: Introduction and the surge in SDLC infrastructure attacks (0:00)

The landscape of software supply chain security has undergone a dramatic transformation, marked by an unprecedented surge in sophisticated attacks. Shay Berkovich emphasized that the past 18 months alone have seen more compromises of SDLC infrastructure than the entire preceding decade. Major incidents like SolarWinds, CircleCI in 2023, XZ Utils, and more recent campaigns such as Ultralytics (December 2023), DG Action, Singularity, Classform, various strains of ShyLoot, and TeamTNT underscore this escalating threat. Attackers, now leveraging AI-assisted development tools, can rapidly generate new payloads and strains, putting immense pressure on defenders.

A critical insight from analyzing these attacks is their multi-architectural nature. They rarely target a single component but rather span across five key architectural pillars: endpoint, Version Control System (VCS), CI/CD pipeline, registry, and production. Furthermore, the attack progression is often bi-directional and non-linear. An attack might start with a script injection in a GitHub workflow (CI/CD), lead to publishing a malicious package to a registry, and then infect developer endpoints. Conversely, a compromised endpoint or leaked credential could grant access to the VCS or CI/CD, enabling broader compromise. This fluidity necessitates a multi-paradigm approach to security, acknowledging that attackers can "pop up in the treasury" or "dig a tunnel underneath the walls."

Berkovich highlighted a significant finding from the analysis of ShyLoot v2, a public and successful campaign that compromised over 27,000 public GitHub repositories. Statistical analysis revealed that 78% of the compromised machines were CI runners, not developer workstations. This crucial statistic positions the CI/CD pipeline as the "new endpoint," demanding that any robust defense strategy must be centered around and explicitly include CI/CD security.

A major "bone to pick" with current supply chain security frameworks, according to Berkovich, is their inherent consumer bias. Traditional models focus almost exclusively on defending the consumer – the entity pulling malicious dependencies. However, this perspective often ignores how those initial malicious packages or components were compromised in the first place. For example, ShyLoot v2 didn't just appear; it originated from pull requests (PRs) into "patient zero" organizations like PostHog, Postman, and Sentry, which then created the initial group of compromised packages. Defending the producer – the organization shipping software – requires a different threat model, one that prioritizes the security of the SDLC pipeline itself, beyond just Software Bill of Materials (SBOMs) or vulnerability scanning of consumed components.

Finally, Berkovich emphasized the need for practicality and relevance. Abstract labels like "supply chain attack" are insufficient. A useful framework must pinpoint specific technical details, such as a "misconfigured GitHub action with an overprivileged npm token," because only then can concrete defensive measures be formulated. This forms the foundational requirements for the SITF framework: a multi-architectural approach, a strong focus on CI/CD as the new endpoint, a correction of the consumer-producer bias, and actionable, real-world applicability.

Key Findings

▶ Watch: CI/CD is the new endpoint: 78% of compromises on runners (4:15)

The talk presented several critical findings that collectively advocate for a fundamental re-evaluation of SDLC security and the introduction of the SITF framework:

• Explosive Growth in SDLC Compromises: The sheer volume and frequency of SDLC infrastructure attacks have surged dramatically, surpassing all previous records within a short 18-month period. This indicates a systemic shift in attacker focus and capabilities, partly driven by AI-assisted development.
• Multi-Architectural and Bi-Directional Attacks: Modern SDLC attacks are not isolated to a single component but traverse and pivot across five architectural pillars: endpoint, VCS, CI/CD, registry, and production. The attack flow is often non-linear, making traditional, siloed security approaches ineffective.
• CI/CD as the "New Endpoint": Analysis of major incidents like ShyLoot v2 reveals that CI/CD runners are disproportionately targeted and compromised (78% in ShyLoot v2). This highlights CI/CD pipelines as critical, high-privilege attack surfaces that require endpoint-level defensive scrutiny.
• The "Consumer Bias" in Supply Chain Security: Existing security models predominantly focus on protecting consumers from malicious dependencies, neglecting the "producer" side – how those malicious components are initially injected or compromised within the SDLC pipeline itself. A producer-centric threat model is essential for comprehensive protection.
• Demand for Practical and Actionable Security Insights: Abstract attack classifications are unhelpful. Security frameworks must provide granular, technical details about attack techniques (e.g., "misconfigured GitHub action with overprivileged npm token") to enable the implementation of specific, effective defensive controls.
• Introduction of SITF: To address these findings, the SDLC Infrastructure Threat Framework (SITF) was introduced. Its core contributions are:
• Five Architectural Components: A structured way to categorize and visualize attacks across Endpoint, VCS, CI/CD, Registry, and Production.
• Causal Relationships: Establishing direct links between attack techniques, enabling risks, and mitigating controls, providing immediate context and actionable defense strategies.
• Real-World Anchoring: SITF is continuously updated with techniques observed in recent, live attacks, ensuring its relevance and practicality.

Technical Deep Dive

▶ Watch: The critical need to defend the producer in the supply chain (6:10)

The SDLC Infrastructure Threat Framework (SITF) is built upon two foundational concepts: the five architectural components of the SDLC and the causal relationship between risks, techniques, and controls. This structure provides a practical and actionable approach to SDLC security.

The framework categorizes all elements – risks, techniques, and controls – by their association with one of the five architectural components: Endpoint, VCS (Version Control System), CI/CD (Continuous Integration/Continuous Delivery), Registry, and Production. This clear segmentation helps in understanding where an attack originates, where it progresses, and where defenses need to be applied. For instance, a "script injection in workflows" is clearly associated with CI/CD, while an "npm token" relates to the Registry.

SITF currently comprises a library of 81 techniques, each meticulously documented. These techniques are also mapped to one of three simplified attack stages, offering a streamlined alternative to more complex frameworks like MITRE ATT&CK. For each technique, SITF identifies:

• Associated Risks: These are the underlying conditions or vulnerabilities that enable a particular attack technique. For example, the risk "no commit signing required" directly enables an "impostor commits" technique.
• Protective Controls: These are the specific measures that can prevent the technique from being successfully executed. Continuing the example, "commit signing" serves as a protective control against impostor commits.
• Detective Controls: These are measures designed to detect when a technique has been attempted or executed.

SITF also integrates with the OWASP SPVS (Software Project Vulnerability Standard) framework, providing an additional layer of context and cross-referencing for techniques and controls.

A cornerstone of SITF's utility is its visualizer, a web-based tool that allows users to graphically construct and analyze attack flows. Users can drag and drop techniques from the library onto a canvas, connecting them to illustrate the progression of an attack across different architectural components. As techniques are added, the visualizer automatically generates a control matrix, listing all associated protective and detective controls required to mitigate the sketched attack. This matrix can be exported as a CSV, making it immediately useful for compliance checks, coverage analysis, and informing defensive strategies. The visualizer also offers a "dark mode" for user preference and comes with a pre-existing library of documented attack flows, such as the TeamTNT attack, demonstrating how real-world incidents can be mapped within the framework.

One of SITF's most innovative features is its integration with AI-powered "cloud techniques," designed to act as a proxy for user interaction with the framework. Recognizing the potential of AI, SITF ships with three such skills:

1. Technique Proposal: If a new technique is missing from the library, users can provide a URL, web search description, or verbal description. The AI skill will research, propose, and even prepare a pull request to add the new technique to the SITF library.
2. Attack Flow: Given a URL, a red team engagement report, or a general web search query describing an attack, this skill automatically draws the attack flow within the visualizer. It identifies the relevant techniques from the library, sketches their connections, and generates the corresponding control matrix.
3. Red Team Flow: Similar to Attack Flow, but specifically tailored for generating attack scenarios based on red team reports.

During the demo, Berkovich showcased the "Attack Flow" skill by asking it to sketch the recent Miasma npm packages compromise. Within approximately five minutes and "10,000 tokens later," the AI successfully generated a detailed attack flow, including a human-language description of the initial access, the specific SITF techniques used, the controls needed to prevent similar attacks, and the sources it consulted. The generated flow can be opened directly in the SITF visualizer as a JSON file.

Crucially, SITF is designed to be open-source and self-contained. The visualizer components are React-based HTML files that can be downloaded directly from GitHub and run locally. There is no telemetry, callbacks, or data sent to Google or any other backend, addressing privacy and corporate data concerns. This "day one tool" approach ensures that security teams can immediately leverage SITF for new attacks, tabletop exercises, threat modeling, or as a reference library for professionals new to SDLC security.

Demo / Proof of Concept

▶ Watch: Framework must be practical, anchored in real-world attack specifics (7:00)

Shay Berkovich presented a two-part live demonstration of the SITF framework, showcasing both its manual visualization capabilities and its innovative AI integration.

The first part of the demo began by navigating to the SITF project on GitHub and accessing its web-based visual components: the Technique Library and the Launch Visualizer.

1. Technique Library Walkthrough: Berkovich first demonstrated the Technique Library, which currently houses 81 techniques spanning the five architectural domains (Endpoint, VCS, CI/CD, Registry, Production). He showed how users can browse through the techniques or use an open search function. For example, searching for "impostor commits" immediately brought up detailed information about this technique. The entry clearly listed the associated risk ("no commit signing required") and the corresponding protective control ("commit signing"). It also highlighted its integration with the OWASP SPVS framework. This illustrated how SITF provides granular, actionable context for each attack technique.

1. Manual Attack Flow Visualization: Next, Berkovich launched the main Visualizer. He simulated an attack from scratch, starting with an entry point on the "Endpoint" pillar and progressing to the "VCS" pillar.

• He dragged the "Endpoint Fishing" technique onto the canvas, connecting it to the entry point.
• This was followed by "Harvest Local Secrets," connected from the fishing technique.
• Finally, "Abuse Access to VCS" was added, representing the attacker gaining access to GitHub after obtaining local secrets.
• He demonstrated the ability to label components, such as "James laptop" for the initial endpoint compromise.
• The key takeaway here was that as each technique was added, the visualizer automatically populated a control matrix in parallel. This matrix instantly showed the protective and detective controls relevant to the chosen techniques, which could then be exported as a CSV for compliance or coverage checks.

1. Pre-existing Attack Flow Library: To demonstrate the framework's utility for analyzing known incidents, Berkovich loaded a pre-existing attack flow for the TeamTNT (merge team PCP) attack. This visually depicted the complex chain of events, including the compromise of Aqua Security and its downstream victims like Checkmarx and Barrieri, clearly illustrating the specific techniques attackers employed. He also mentioned the recent addition of a technique related to the Vercel incident (cross-fork object reference abuse), emphasizing SITF's commitment to staying current with real-world threats.

The second part of the demo showcased the AI-powered "cloud techniques." Berkovich initiated this in parallel with the first demo, as it takes a few minutes to process. He used the "Attack Flow" skill, providing it with a description of the Miasma npm packages compromise (an incident that occurred just days before the talk).

• After approximately five minutes, the AI skill returned a detailed output. This included a human-language summary of the attack, a list of the specific SITF techniques it identified and used, the crucial controls needed to prevent similar attacks, and the external sources it consulted.
• He then loaded the generated JSON file into the visualizer, which rendered the Miasma attack flow. Although slightly misaligned due to rendering complexities, all major components of the attack were clearly depicted, demonstrating the AI's ability to rapidly transform a textual description into a structured, visual attack map complete with actionable controls.

This comprehensive demo powerfully illustrated SITF's dual strengths: providing a detailed, visual, and actionable framework for human analysts, while simultaneously leveraging AI to automate and accelerate threat modeling and control identification for recent and emerging threats.

Defensive Implications

▶ Watch: Starting the live demo of the SITF framework (8:00)

The SITF framework and the insights shared by Shay Berkovich provide critical guidance for organizations seeking to fortify their SDLC security posture against the rapidly evolving threat landscape. The defensive implications are multi-faceted and demand a strategic shift in how security teams approach their development pipelines.

Firstly, organizations must adopt a multi-architectural perspective on SDLC security. No longer sufficient are siloed defenses focusing solely on endpoints or production. Defenders must understand that attacks can originate or pivot across the endpoint, VCS, CI/CD, registry, and production environments. This necessitates a holistic security strategy that integrates controls and monitoring across all these components.

Secondly, the talk unequivocally establishes the CI/CD pipeline as the "new endpoint." Given that 78% of ShyLoot v2 compromises occurred on CI runners, these environments must be treated with the same, if not greater, scrutiny as traditional endpoints. This means implementing robust authentication (e.g., OIDC), stringent access controls, least privilege principles for workflows, and continuous monitoring within CI/CD systems. Organizations should re-evaluate their CI/CD configurations to minimize the blast radius of a potential compromise.

Thirdly, a crucial mindset shift is required from a "consumer bias" to a "producer-centric" security model. While protecting consumers from malicious packages remains important, defenders must equally prioritize securing their own SDLC pipelines where software is built and published. This involves scrutinizing pull requests, ensuring proper code review, securing build environments, managing npm tokens and other credentials with extreme care, and implementing controls that prevent the initial injection of malicious code or components into the supply chain.

SITF itself serves as a powerful defensive tool in several ways:

• Common Language and Expressibility: It provides a standardized vocabulary for discussing and documenting SDLC attacks, fostering clearer communication among security teams, developers, and leadership.
• Enhanced Threat Modeling: Instead of relying on abstract threat models, SITF offers a practical canvas for visualizing specific attack paths. By mapping out potential attack flows, organizations can identify critical choke points and prioritize control implementation. The automatic generation of a control matrix based on identified techniques makes threat modeling immediately actionable.
• Tabletop Exercise Catalyst: SITF can be used as a dynamic tool for tabletop exercises, simulating real-world attack scenarios and evaluating the effectiveness of existing controls and incident response plans.
• Reference Library: For security professionals new to the nuances of SDLC security, SITF acts as an invaluable educational resource, demystifying complex techniques like "pull request script injection" or "cross fork object reference abuse."

The discussion around defensive measures also touched upon specific strategies:

• Package Cooldowns: Berkovich acknowledged that implementing a global package cooldown (e.g., a few days to a week) could mitigate a significant portion of attacks, estimating around 50%. However, he cautioned that this is not a panacea, as authorized malicious package publishing (via leaked credentials or compromised workflows) and non-package-related attacks (like TG actions compromises) would still bypass such controls.
• Pinning Dependencies: While pinning dependencies (e.g., action checkout v6 to action checkout@<hash>) is often recommended, Berkovich highlighted a potential blind spot. While hashes offer immutability, they can obscure the actual version or source, and complex dependencies can still be bypassed (as seen in the Trivia attack). He emphasized that pinning should be part of a broader defense-in-depth strategy.
• Depth in Security: The ultimate recommendation is a multi-layered, defense-in-depth approach. This includes not just pinning dependencies but also limiting workflow privileges, implementing robust OIDC for authentication, enforcing commit signing, and continuously monitoring for anomalous activity. No single control is sufficient; a combination of strong preventative, detective, and responsive measures across the entire SDLC is paramount.

Finally, the talk implicitly encourages CI/CD providers and tool vendors to improve their security posture. While a "wall of shame" might not be necessary, the continuous stream of real-world compromises (like those affecting Aqua Security, Checkmarx, and LightM) serves as a potent motivator for the industry to address fundamental architectural weaknesses and implement more secure defaults.

Key Takeaways

• SDLC Attacks are Surging and Complex: The frequency and sophistication of attacks targeting SDLC infrastructure have dramatically increased, often spanning multiple architectural components (endpoint, VCS, CI/CD, registry, production) in bi-directional flows.
• CI/CD is the New Critical Endpoint: CI/CD runners are disproportionately targeted and compromised, functioning as high-privilege "new endpoints" that demand focused defensive strategies and robust security controls.
• Shift from Consumer to Producer Security: Traditional supply chain security's "consumer bias" is insufficient. Organizations must prioritize securing their own SDLC pipelines as producers, addressing how malicious components are initially introduced, not just how they are consumed.
• SITF Provides Actionable, Real-World Context: The SDLC Infrastructure Threat Framework offers a practical, multi-architectural model to visualize attacks, identify enabling risks, and derive concrete protective and detective controls, moving beyond abstract "supply chain attack" labels.
• AI Accelerates Threat Modeling and Defense: SITF's AI-powered "cloud techniques" can rapidly analyze attack descriptions, generate visual attack flows, and propose relevant controls, significantly enhancing the speed and efficiency of threat intelligence and defensive planning.
• Defense-in-Depth is Paramount: No single control (e.g., package cooldowns, dependency pinning) is a silver bullet. A comprehensive, multi-layered defense-in-depth approach, incorporating measures like OIDC, least privilege, commit signing, and continuous monitoring, is essential to mitigate the complex web of trust and attack vectors in modern SDLCs.

About the Speaker(s)

Shay Berkovich is a security expert currently working at Google, where he is a proud member of their threat research group. Prior to his tenure at Google, he was associated with the WH Threat Research Group. On a personal note, Shay is a self-proclaimed owner of two cats and father of two kids, "in that order." He is an enthusiastic and experienced speaker at security conferences, notably presenting at fwd:cloudsec for the second time, a fact he expresses pride in. Shay also extended special thanks to Ramy McCarthy for his invaluable contributions as a bouncing board for ideas and a source of constant feedback on the SITF project.