Hackers vs. Devs - Attacking Dev Tools and Infrastructure

Hackers vs. Devs - Attacking Dev Tools and Infrastructure

Speakers: Ofer Maor, Co-Founder & CTO, Mitiga

Conference: RSAC 2024

YouTube: https://www.youtube.com/watch?v=z4DZ9DjwEyg

Overview

In an insightful and timely talk at RSAC 2024, Ofer Maor, Co-Founder & CTO of incident response firm Mitiga, shed light on a critical shift in the threat landscape: attackers are increasingly targeting developer tools and infrastructure rather than custom application code. Maor, leveraging decades of experience spanning traditional AppSec and modern cloud incident response, presented a compelling case that the prevailing focus on preventing vulnerabilities in bespoke code often misses where real-world breaches originate. His firm’s data, revealing that only three out of hundreds of incidents over four and a half years stemmed from custom code vulnerabilities, underscored this stark reality.

This presentation served as a wake-up call for security practitioners, urging them to re-evaluate their defensive strategies and direct their resources towards the most exploited vectors. Maor highlighted that the allure for attackers lies in the scalability and high return on investment (ROI) of compromising widely used developer platforms, open-source libraries, and cloud-based infrastructure. By compromising a single vendor or a common component, attackers can gain access to thousands of organizations, making these targets far more attractive than labor-intensive custom code exploits.

The talk provided a granular look at the prevalent initial access vectors—identity compromise and leaked secrets—and delved into sophisticated attack techniques, including MFA bypasses and supply chain attacks like the notorious XZ Utils backdoor. Maor's emphasis was on practical, real-world observations derived from handling numerous breaches, offering security teams actionable intelligence to better protect their organizations against the threats that truly materialize in the wild.

Background

▶ Watch: Introduction: GitLab vulnerability highlights dev tool risks (0:00)

Ofer Maor’s journey from a long career in application security (AppSec), encompassing roles as a builder, breaker, and defender, to co-founding an incident response (IR) company, Mitiga, forms the foundational perspective of this talk. This transition provided him with a unique vantage point: the ability to compare the theoretical risks often discussed in AppSec with the actual attack vectors observed during real-world breaches. Maor candidly admitted that while AppSec practitioners, including himself, frequently focus on perceived risks or vulnerabilities that are technically achievable for a penetration tester, the reality seen in IR is often different.

His key revelation, drawn from Mitiga's extensive IR experience, is that attackers are "lazy" in the sense that they operate as a business. Like any business, they seek maximum outcome for minimum effort, prioritizing ROI. This fundamental principle drives them away from niche custom code vulnerabilities—which require significant effort for a single, often non-replicable gain—and towards scalable targets. Developer infrastructure, including cloud-based tools like GitHub, GitLab, CircleCI, and open-source libraries, presents exactly this kind of scalable target. A single compromise can yield access to numerous organizations or critical systems, leading to substantial ransom demands or data exfiltration opportunities.

The talk underscored a critical shift in the security perimeter. For decades, traditional IT infrastructure was protected by firewalls, VPNs, and on-premise identity solutions. However, with the pervasive move to cloud-based services and Software-as-a-Service (SaaS) developer tools, the perimeter has dissolved. Identity providers like Okta and Azure AD have become the new "single point of failure," and developer platforms, often connected to production environments, are now prime targets. Maor highlighted a significant gap: while IT infrastructure typically benefits from robust detection and response technologies, monitoring of developer-centric systems, such as ingesting GitHub logs into a Security Information and Event Management (SIEM) system, remains woefully inadequate in many organizations. This lack of visibility means that even when breaches occur through these vectors, they often go undetected by the victim company, with two-thirds of major incidents discovered by external parties like law enforcement or customers.

Key Findings

▶ Watch: Attackers target dev infrastructure, not custom code (2:00)

The core message of Maor’s talk is a fundamental re-evaluation of where security efforts should be concentrated. His firm's incident response data unequivocally demonstrates that attackers prioritize developer infrastructure over custom application code. In four and a half years, Mitiga has handled hundreds of incidents, with only three attributed to vulnerabilities in custom code. The vast majority stemmed from compromised developer tools, accounts, misconfigurations, or open-source libraries. This preference is driven by the attackers' business mindset, seeking scalable attack vectors with high ROI.

A primary initial access vector in cloud environments is identity compromise, accounting for approximately 50% of all incidents Mitiga observes. This often starts with phishing campaigns targeting user credentials. Crucially, Maor emphasized that MFA is no longer a sufficient deterrent on its own. Attackers have developed sophisticated techniques to bypass multi-factor authentication, rendering it less effective against determined adversaries.

The persistence of secrets in code was highlighted as a ubiquitous and critical vulnerability. Despite developers often claiming otherwise, IR investigations consistently uncover hardcoded credentials, API keys, and other sensitive information within repositories. When developer infrastructure is compromised, these leaked secrets provide immediate and direct access to critical systems and data stores.

The talk also detailed the alarming speed at which accidentally exposed public code or secrets are discovered and exploited. Automated scanning tools continuously scour public repositories, meaning that secrets pushed to a public GitHub repo can be identified and leveraged within single-digit seconds to 20-30 seconds.

Finally, Maor brought attention to the sophisticated and stealthy nature of supply chain attacks, particularly through open-source components. The XZ Utils backdoor served as a prime example of a nation-state level attack that nearly achieved widespread compromise, demonstrating that the traditional "open source is more secure" adage needs critical re-evaluation in the face of targeted, highly obfuscated threats.

Technical Deep Dive

▶ Watch: Real-world examples of major dev tool attacks (3:30)

The technical core of Maor's presentation focused on the prevalent attack vectors and the mechanisms by which adversaries exploit developer infrastructure.

Identity Compromise and MFA Bypass

Maor stressed that Single Sign-On (SSO), while designed for convenience and centralized identity management, has become a single point of failure. Platforms like Okta and Azure AD, when compromised, grant attackers widespread access to all connected applications, including critical developer tools. The traditional reliance on Multi-Factor Authentication (MFA) is increasingly insufficient due to two dominant bypass techniques:

1. MFA Push Fatigue: This technique exploits human psychology and the convenience of push-based MFA. Attackers, having obtained a user's primary credentials (e.g., via phishing), repeatedly attempt logins, triggering a barrage of MFA push notifications to the user's device. While a user might initially deny these requests, sustained harassment often leads to an eventual, accidental approval. Maor noted incidents where "three, four times is what you need for a user to push for yes," sometimes even more for the attacker to register a new MFA device.
2. Adversary-in-the-Middle (AiTM) Attacks: More sophisticated than push fatigue, AiTM attacks involve the attacker setting up a proxy server between the user and the legitimate login portal. The user unknowingly authenticates through the attacker's proxy, which intercepts credentials and any MFA challenges (including one-time passwords or push approvals). By acting as an intermediary, the attacker captures a valid, live session token, effectively bypassing MFA and gaining authenticated access without directly possessing the user's MFA device. Maor referenced Microsoft's research on AiTM, highlighting its growing prevalence and difficulty in prevention.

Secrets in Code

A persistent and alarming issue is the ubiquitous presence of secrets in code. Despite security best practices and developer awareness programs, Maor's IR experience consistently shows that organizations harbor sensitive data—such as API keys, database credentials, cloud access keys, and SSL certificates—directly within their code repositories. This problem was starkly illustrated by the Sisense breach, a major incident that led to a CISA alert. In this case, attackers reportedly compromised a GitLab account (method unspecified, but potentially MFA circumvention). This account provided access to Sisense's code, which contained secrets. These secrets, in turn, led to S3 buckets holding "millions of secrets" belonging to Sisense's customers, alongside terabytes of customer data. The absence of a dedicated Key Management System (KMS) for these critical assets amplified the impact.

Accidental Public Code Exposure

The talk detailed how quickly secrets exposed in public repositories are exploited. Maor cited several scenarios: a developer accidentally changing repository permissions to public, pushing private code to a public personal repository, or merging a corporate Git repo into a personal one. The critical factor is speed: once a secret is public on platforms like GitHub, it is discovered by automated scanners within "single-digit seconds to 20-30 seconds." These scanners, including tools like Truffle Hog and Coral Boto (which attackers also use to validate stolen credentials), continuously trawl public repositories for sensitive patterns.

An example provided involved a developer accidentally exposing two AWS keys. Within minutes, a threat actor discovered these keys. Logs immediately showed attempts by Truffle Hog to validate the keys, followed by numerous GetCallerIdentity API calls—a common reconnaissance technique to ascertain the scope of permissions associated with the compromised credentials. While this particular incident was mitigated before significant damage due to limited key permissions, it underscored the immediate and automated nature of the threat.

Supply Chain and Software Implants

Maor introduced the sophisticated threat of supply chain attacks through open-source software, exemplified by the XZ Utils backdoor. This incident, discovered in early 2024, involved a highly obfuscated backdoor injected into the liblzma component of XZ Utils, a widely used data compression utility in Linux distributions. The attack was attributed to a malicious actor, "Jia Tan," who gained maintainer trust over several years.

The backdoor's mechanism was complex:

• It was introduced through obfuscated code within test files and build system modifications.
• The malicious code was designed to be activated only during specific build environments (e.g., Debian and Red Hat based systems using gcc) and only when compiling with automake and libtool.
• Crucially, it manipulated the configure script and Makefile to inject a malicious payload into the liblzma library.
• When sshd (the OpenSSH server daemon) dynamically linked liblzma, the backdoor could intercept and modify authentication functions, potentially allowing remote code execution via specially crafted SSH authentication requests.

The XZ Utils incident was a "nation-state attack" that was "accidentally discovered" due to unusual SSH performance issues, highlighting the extreme stealth and sophistication such attacks can achieve. It shattered the long-held belief that open-source software is inherently more secure due to public scrutiny, revealing that highly targeted and deeply embedded compromises are a tangible threat.

Demo / Proof of Concept

▶ Watch: Shocking statistics on incident discovery (5:00)

The talk did not feature a live technical demonstration or proof of concept in the traditional sense. Instead, it focused on illustrating the discussed attack vectors and their real-world impact through detailed case studies and incident response observations. Maor effectively used examples such as the GitLab password reset vulnerability, the Midnight Blizzard attack on Microsoft, the CircleCI breach, the Sisense data compromise, and the XZ Utils backdoor to demonstrate how these attack methodologies manifest in actual security incidents. These real-world accounts served as powerful evidence for the threats discussed, providing concrete examples of compromised dev tools, leaked secrets, and supply chain vulnerabilities.

Defensive Implications

▶ Watch: Identity as the primary initial access vector (6:20)

The insights from Ofer Maor’s talk necessitate a significant recalibration of defensive strategies, moving beyond traditional AppSec focuses to prioritize developer infrastructure and identity security.

1. Elevate Developer Infrastructure Security: Organizations must recognize that platforms like GitHub, GitLab, CircleCI, and cloud-based IDEs are as critical as production environments. These tools are the new perimeter, often holding the keys to the kingdom. Comprehensive security assessments, vulnerability management, and robust access controls must extend to these systems.

1. Strengthen Identity and Access Management (IAM):

• Beyond Basic MFA: While MFA is essential, defenders must acknowledge its evolving bypass techniques. Implement stronger forms of MFA, such as FIDO2 hardware keys (e.g., YubiKeys), which are more resistant to phishing and AiTM attacks.
• MFA Behavior Monitoring: Monitor for anomalous MFA activity, such as repeated push notifications from a single user or unusual MFA device registrations, which could indicate push fatigue or AiTM attacks.
• Conditional Access Policies: Implement stringent conditional access policies based on device posture, location, IP reputation, and user behavior to restrict access to sensitive developer tools.

1. Implement Robust Secret Management:

• Eliminate Secrets from Code: Enforce a strict policy against hardcoding secrets in repositories. Adopt dedicated secret management solutions like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault for storing and dynamically injecting credentials.
• Automated Secret Scanning: Deploy continuous, automated secret scanning tools (e.g., Gitleaks, Truffle Hog) within CI/CD pipelines and directly against all code repositories (public and private). Integrate these scanners to block commits with exposed secrets.
• Secret Rotation: Implement automated secret rotation policies to minimize the window of exposure for compromised credentials.

1. Enhance Supply Chain Security:

• Software Composition Analysis (SCA): Use SCA tools to identify and monitor all open-source dependencies for known vulnerabilities and licensing issues.
• Dependency Vetting: Implement rigorous processes for vetting new open-source dependencies, scrutinizing maintainer history, recent code changes, and potential red flags (as seen with XZ Utils).
• Build Process Integrity: Secure build environments and ensure the integrity of the software supply chain from source code to deployment. Consider adopting frameworks like SLSA (Supply Chain Levels for Software Artifacts).

1. Improve Detection and Incident Response (IR) for Dev Environments:

• Ingest Dev Tool Logs into SIEM: A critical step is to centralize logs from all developer tools (GitHub, GitLab, CircleCI, Jenkins, Jira, Okta, Azure AD) into a SIEM. This provides essential visibility for detection.
• Develop Specific Detections: Create alerts for suspicious activities:
• Unusual login patterns to developer platforms (e.g., from new IPs, unusual times).
• Spikes in "access denied" errors, indicating an attacker attempting to enumerate resources.
• Rapid-fire API calls (e.g., GetCallerIdentity on AWS) after a potential key compromise.
• Changes to repository permissions or the creation of new user accounts in dev tools.
• Attempts to validate credentials using tools like Truffle Hog.
• IR Playbooks for Dev Incidents: Develop specific incident response playbooks tailored for compromised developer accounts, leaked secrets, and supply chain breaches, including steps for secret rotation, forensics on dev environments, and communication with impacted customers.
• Third-Party Risk Management: Understand the security posture of all SaaS vendors, especially those handling sensitive data or integrated into critical workflows. Be prepared to respond quickly to alerts from vendors about potential breaches (e.g., the Sisense and CircleCI incidents).

1. Developer Education and Awareness:

• Educate developers about the immediate and severe consequences of accidentally exposing private code or secrets.
• Train them on phishing awareness, especially regarding MFA bypass techniques.
• Foster a security-conscious culture where developers understand their role in protecting the software supply chain.

Key Takeaways

• Shift Focus: Attackers overwhelmingly target developer tools and infrastructure, not custom application code, for scalable and high-ROI breaches.
• Identity is the New Perimeter: Compromised identity, particularly through SSO providers, is a primary initial access vector, and traditional MFA is increasingly bypassed by sophisticated techniques like push fatigue and AiTM attacks.
• Secrets Remain a Critical Weakness: Hardcoded secrets in code repositories are rampant and, when exposed, provide direct access to critical systems and sensitive data, as demonstrated by the Sisense breach.
• Speed of Exploitation: Accidental public exposure of code or secrets is discovered and exploited almost immediately (within seconds to minutes) by automated scanning tools.
• Supply Chain Vulnerability: Open-source software supply chains are vulnerable to highly sophisticated, nation-state-level attacks, as evidenced by the XZ Utils backdoor, necessitating rigorous vetting and monitoring.
• Proactive Defense is Crucial: Defenders must prioritize securing dev infrastructure, implementing advanced identity protection, enforcing strict secret management, and enhancing detection capabilities by ingesting developer tool logs into SIEMs to identify and respond to these prevalent threats.

About the Speaker(s)

Ofer Maor is the Co-Founder and CTO of Mitiga, an incident response company. With over two decades of experience in the cybersecurity industry, Maor has a deep background in application security (AppSec), having spent a significant portion of his career as a "builder, breaker, and defender" at companies like Synopsys. After years focused on preventing breaches, he made a strategic move to the incident response side, driven by the realization that prevention efforts were often failing to stop real-world attacks. At Mitiga, his work involves analyzing actual breaches and identifying the true methods and priorities of attackers. This unique perspective, bridging AppSec theory with IR reality, forms the foundation of his insights into modern cyber threats.