How to keep Open Source open without leaving our communities open to threats

How to keep Open Source open without leaving our communities open to threats

Speakers: Quintessence

Conference: 39C3

YouTube: https://www.youtube.com/watch?v=_hoeeFc7OWM

Overview

In this compelling talk, Quintessence (Q), the Executive Director of the Nively Foundation, dissects a critical and often overlooked dimension of open source security: the human element. The presentation, titled "How to keep Open Source open without leaving our communities open to threats," argues that while traditional security focuses on code vulnerabilities, the increasing frequency of crisis events within open source communities stems from "human-sided vulnerabilities" and the weaponization of organizational weaknesses. As open source software has evolved from niche hobby projects into the foundational infrastructure of global society, the resilience of its human communities has become paramount.

Q's research reveals a concerning trend: a dramatic increase in "collapse or near-collapse" events affecting open source foundations since 2010, impacting 90% of organizations surveyed. These crises, ranging from governance failures to moderation team walk-offs, highlight a systemic issue where existing community support mechanisms are proving insufficient. The talk advocates for a paradigm shift, urging the community to move beyond a sole focus on technical exploits and instead cultivate community health, responsible friction, and robust documentation practices to build truly resilient open source ecosystems.

The core message is a call to action for the open source community to acknowledge and address these human-centric threats. By centering community health, developing organizational threat awareness, and implementing structured guidelines for moderation and documentation, Q posits that open source can protect its invaluable contributions without fostering a culture of paranoia. This proactive approach aims to equip maintainers and moderators—who are often volunteers—with the tools and knowledge necessary to navigate complex social dynamics and thwart both intentional and unintentional harms that exploit human vulnerabilities.

Background

▶ Watch: Introduction: Keeping Open Source Open to Threats (0:00)

The evolution of open source software, as meticulously charted by Quintessence, provides crucial context for understanding its current vulnerabilities. In its nascent stages, prior to the 2000s, open source was characterized by limited computer access, "cluji" tools, and asynchronous versioning. Projects like the Apache web server existed, but the movement was small, young, and not running critical infrastructure. This era fostered curated communities, often centered around shared physical spaces like hacker labs, where participation was inherently selective.

The 2010s marked a significant shift. Home computer ownership became widespread, social media matured, and open source began to run increasingly critical systems, particularly in education. Tools became easier to use, expanding access and participation. However, the true inflection point arrived in the present day (2020s), where open source is deeply embedded across virtually all critical sectors: financial systems, government infrastructure, healthcare, and research. Q cites data from BlackDuck, indicating that 97% of codebases now incorporate open source components, with an average of 911 components per codebase, tripling in just five years. The industry, once valued in the billions for its supply side, now sees its demand side valued in the trillions of dollars. This ubiquitous integration means that open source is no longer a collection of hobby projects; it is a global, trillion-dollar industry, making its resilience a matter of global security.

Despite this unprecedented criticality, Q's research into 30 diverse open source organizations (a 50/50 split of for-profit and nonprofit, scaled by "t-shirt size" for data aggregation) reveals a disturbing trend: 90% of these organizations have experienced at least one "collapse or near-collapse event" since 2000. Prior to 2010, such events were rare, often making major headlines as shocking outliers. Today, they have become normalized. The data shows an alarming acceleration in the mean time to a project's first crisis: from approximately 24 years for older foundations, it has plummeted to roughly 12, then 6, and now approximately 3 years for younger projects, indicating that new foundations are being hit with crises "right out of the gate."

These crisis events manifest in various forms: Code of Conduct, Licensing, Governance, Moderation, and Leadership crises. Q highlights that the emergence of Codes of Conduct for online spaces, spearheaded by initiatives like the Contributor Covenant, did not prevent new crisis types from appearing immediately. For instance, the first Code of Conduct crisis and moderation crisis appeared almost concurrently with the widespread adoption of these policies. High-profile incidents, such as the walk-offs in the Rust and NixOS communities in 2021, while grabbing significant attention, were not unique; they represented just two of nine such collapse events in that calendar year, underscoring the systemic nature of the problem.

Crucially, Q draws a distinction between traditional actor-driven harms and amplified non-actor harms. The XCutils case serves as a stark illustration of the former: the "only public confirmed case of an actor" exploiting a human-sided vulnerability—specifically, a sole maintainer's mental health and burnout—to inject malicious code. The vulnerability had a CVSS score of 10, but the exploitation vector was human. This incident was only discovered by accident, highlighting a critical gap in detection mechanisms for human-centric exploits. Non-actor harms, amplified by social media and "social git repositories," can create similar "high high and low lows" seen in 100-comment pull requests, overwhelming maintainers and leading to burnout. Both actor and non-actor harms exploit common weaknesses: lack of documentation, poor knowledge propagation, and burned-out maintainers, all enabled by the "collapsed barriers" or "soft gates" that made earlier technologies harder to use, thus curating participation. This extensive background underscores the urgent need for new protective measures that address the human and organizational dimensions of open source security.

Key Findings

▶ Watch: Open Source Matures: Running Critical Infrastructure (2010s) (2:00)

The central findings presented by Quintessence paint a stark picture of open source's current predicament, emphasizing a critical disconnect between its economic value and its community resilience.

1. Systemic Community Crisis: Open source, now a trillion-dollar industry and the backbone of global critical infrastructure, is experiencing a widespread and accelerating crisis at the community level. Q's research on 30 organizations revealed that an overwhelming 90% had experienced at least one "collapse or near-collapse event" since the year 2000. These events are no longer anomalies but have become normalized, with the mean time to a first crisis dramatically shrinking from 24 years for older projects to as little as 3 years for newer ones.
2. Human-Sided Vulnerabilities as a Primary Attack Vector: The talk forcefully argues that traditional security models, which primarily focus on code vulnerabilities and technical exploits, are insufficient. Q introduces the concept of "human-sided vulnerabilities," such as maintainer burnout, mental health challenges, and a lack of organizational memory. The XCutils incident is cited as a prime example where an attacker exploited a sole maintainer's personal struggles to introduce malicious code, demonstrating that the human element is a direct and exploitable attack surface.
3. Ineffectiveness of Existing Community Governance Tools: Despite the widespread adoption of tools like the Contributor Covenant and the development of Codes of Conduct for online spaces, new types of crises—specifically Code of Conduct crises and moderation crises—appeared almost immediately on the timeline. This indicates that current governance and social guidelines, while well-intentioned, are not robust enough to prevent or mitigate the escalating challenges faced by communities.
4. Lack of Organizational Threat Awareness and Documentation: A significant contributing factor to these crises is the absence of organizational threat awareness and formal threat models within most open source projects. Q points out that contributor ladders rarely incorporate an understanding of project risks. Furthermore, critical information regarding moderation decisions often lacks sufficient context in audit logs, leading to organizational amnesia when experienced maintainers or moderators depart. This makes it impossible to triage past events, learn from them, or detect patterns of abuse.
5. Weaponization of Organizational Weaknesses: The ease of exploiting organizational processes or social dynamics is highlighted. Q notes that low-cost, high-payoff attacks, such as malicious GDPR or DMCA requests, can be weaponized against communities. Similarly, poorly designed or undocumented governance and contributor models can be easily manipulated, turning the community's own structures against itself.
6. The Need for a New Framework: The cumulative findings underscore the urgent need for a new framework that shifts the focus from purely technical security to a holistic approach encompassing community health and resilience. This framework must address the gaps in documentation, knowledge propagation, and protective measures for the human contributors who are the bedrock of open source.

Technical Deep Dive

▶ Watch: Present Day: Trillion-Dollar, Ubiquitous Open Source Industry (3:30)

While not a "technical deep dive" in the conventional sense of analyzing code or network protocols, Quintessence's talk provides a profound technical analysis of the mechanisms of community breakdown and exploitation within open source. This involves understanding the interplay of human factors, organizational structures, and the unique environment of open source development.

The core of this "deep dive" rests on the concept of human-sided vulnerabilities. These are not software bugs but inherent weaknesses in human cognition, social dynamics, and organizational design that can be exploited. Q identifies several critical ones:

• Maintainer Burnout and Mental Health: The XCutils case is the prime example. A sole maintainer, under immense pressure and experiencing mental health challenges, was socially engineered into merging malicious code. This was not a flaw in the software itself but an exploitation of the human behind the keyboard. The project, lacking redundancy or oversight mechanisms for a sole maintainer, had no internal system to detect this until it was discovered accidentally.
• Lack of Knowledge Propagation and Documentation: Many open source projects, particularly smaller ones, rely heavily on the tacit knowledge of experienced contributors. When these individuals burn out, walk off, or otherwise depart, this critical knowledge is lost, leading to organizational amnesia. This applies not only to technical project specifics but, more critically, to moderation and governance decisions. Audit logs often lack the necessary context, making it impossible to understand past events or identify patterns of abuse retrospectively. Q stresses that "if moderators don't know what to write down, that audit log looks fine" until a crisis necessitates investigation.
• Collapsed Barriers and Soft Gates: In the early days, "cluji" tools and less accessible technology inadvertently created "soft gates" or barriers that curated participation, making it harder for disruptive or malicious actors to engage. The proliferation of easy-to-use tools and social platforms has removed these barriers, democratizing participation but simultaneously widening the attack surface for human-centric exploitation. This enables "high high and low lows" in communication, such as pull requests accumulating 100 comments, overwhelming maintainers.

These human-sided vulnerabilities are then weaponized through various organizational weaknesses:

• Easy-to-Exploit Governance and Contributor Models: Many projects implement governance structures or contributor ladders without a clear understanding of their potential for misuse. Q notes that most organizations she surveyed did not have a threat model for their community or organization. This means that as contributors gain influence, they may not be aware of the risks posed to the project or how their actions (or inaction) could be abused, even unintentionally.
• Low-Cost, High-Payoff Attacks: Beyond direct code injection, Q highlights how non-technical processes can be weaponized. Examples include the ease of filing GDPR or DMCA requests, which are low-cost for an attacker but high-cost and difficult to respond to for a volunteer-run project. These can be used to disrupt, harass, or exhaust community resources.
• Crisis Types and Their Manifestations: The various crises identified (Code of Conduct, Licensing, Governance, Moderation, Leadership) are technical in their impact on the project's operational continuity and community health.
• Leadership Crises often mirror moderation crises, involving "step down statements" from individuals or entire leadership bodies due to sustained pressure. This pressure can originate from non-actor harms (e.g., amplified dissatisfaction over a slow pull request) or targeted campaigns.
• Moderation Crises involve teams walking off or significant internal conflict, often due to burnout, lack of support, or unclear guidelines on how to handle difficult situations.
• Licensing Crises typically involve for-profit entities attempting to close-source previously open software, leading to community fragmentation and trust erosion.

The absence of organizational threat awareness means that projects are often reactive rather than proactive. They lack the frameworks to conduct risk analysis, threat modeling, or abusability testing specific to their community dynamics. This leaves them vulnerable to both overt attacks and the insidious erosion of community health through unmanaged conflict, burnout, and unaddressed behavioral issues. The "technical" challenge here is to design and implement robust social and organizational architectures that can withstand these human-centric pressures, much as cryptographic protocols withstand mathematical attacks.

Demo / Proof of Concept

▶ Watch: Crisis Trend: Collapse Events Normalized Since 2010 (6:00)

Quintessence's presentation did not include a traditional software demonstration or a live proof of concept in the manner common to many security conference talks. The "proof" for her assertions was primarily rooted in the extensive data analysis she conducted on 30 open source organizations, which formed the bedrock of her findings on the prevalence and acceleration of community crisis events.

Instead of demonstrating a technical exploit, Q presented a call to action and outlined a proactive "proof of concept" for a collaborative solution: the launch of a Securing Open Source Communities Working Group. This working group, with its inaugural session immediately following the talk, is intended to be the mechanism through which the open source community collectively develops the missing guidance and frameworks. Its phases represent an experimental approach to address the identified gaps:

• Phase 1: Establishing Communications Standards: The initial focus is on developing standardized communication patterns for open source projects during crisis events. This acknowledges that a common failure mode is the inability of leadership bodies and volunteer teams to communicate effectively outwards, struggling with the balance of transparency versus opacity, especially when potential security threats are involved.
• Phase 2: Threat Testing Governance Models: Once communication standards are established, these patterns will be used to "threat test and abusability test governance models." Q explicitly states that the goal is not to publish a "playbook for how to attack everyone in open source," but rather to understand vulnerabilities in existing models to produce easily accessible guidance.

In essence, the entire talk, supported by its data-driven insights, serves as the "proof" that a systemic problem exists. The proposed working group is the "concept" for how the community can collaboratively build the necessary "mechanisms" to address it, moving beyond anecdotal responses to a structured, preventative approach.

Defensive Implications

▶ Watch: Common Crisis Types: Governance, Conduct, Leadership Failures (7:00)

The defensive implications stemming from Quintessence's analysis are profound, necessitating a fundamental shift in how open source communities approach security. The focus must expand beyond code vulnerabilities to encompass the resilience and health of the human communities that create and maintain the software.

1. Gaps Analysis on the Human Side: The first and most critical step is to perform a comprehensive "gaps analysis on the human side" of open source projects. This involves acknowledging, analyzing, and actively addressing the human-sided vulnerabilities and organizational weaknesses identified. It's about understanding where the community is susceptible to burnout, knowledge loss, and social engineering.

1. Integrate Threat Models into Contributor Ladders: Open source projects must develop organizational threat awareness and formal threat models that extend beyond technical exploits. These threat models should then be integrated into contributor ladders. As individuals gain influence and responsibility within a project (e.g., the ability to merge code), they should also gain an understanding of the risks posed to the project and how their roles can be misused or exploited. This ensures that increased authority comes with increased security awareness.

1. Develop Consistent Documentation Guidelines for Moderation: To combat organizational amnesia and enable effective post-crisis triage, communities need clear guidelines for documenting moderation events. This goes beyond simple audit logs, requiring moderators to record context, rationale, and relevant details consistently. This structured documentation ensures that knowledge is retained even if experienced moderators depart, allowing for pattern detection and informed decision-making over time.

1. Center Community Health, Not Just Threat Detection: Q strongly advises against building "paranoid communities" by solely focusing on threat detection. Most community members are not security specialists and should not be expected to become such. Instead, the focus should be on fostering healthier communities, which inherently instills protective measures. This approach makes security practices "doable by the community" without requiring specialized expertise from every member.

1. Implement Responsible Friction: A key defensive strategy is the introduction of responsible friction. This means implementing just enough friction in processes (e.g., contribution workflows, onboarding for new members) to slow things down slightly. This allows time for observation and intervention if something appears amiss, without hindering legitimate contributions. It also helps enthusiastic new contributors avoid immediate burnout by encouraging a more measured integration into the community. From a security perspective, responsible friction also serves to slow down an attacker, providing a window for detection.

1. Normalize Conflict Resolution: Conflict is inevitable in any human community. Instead of being conflict-averse, open source projects should normalize conflict and teach healthy resolution mechanisms. A community that knows how to heal from internal disagreements is far less susceptible to being manipulated into a state of internal conflict by an attacker. This resilience makes it harder for malicious actors to weaponize internal divisions.

1. Redefine Moderation as Boundary Enforcement: Moderators, who are often volunteers, should be equipped with a clear mandate: their role is not to identify "bad actors" or conduct complex threat analysis, but to assess whether "behavior is compatible with community health." This reframing centers moderation on boundary enforcement rather than punishment, making it a more manageable and less emotionally taxing role. Enforcing healthy boundaries inherently prevents attackers from pushing those boundaries intentionally.

1. Build Accessible Guidance for Small Projects: Recognizing that small projects and solo maintainers lack the resources for dedicated security staff, there is an urgent need to build accessible, actionable guidance. This guidance should cover risk analysis, threat modeling, and abusability testing in a format similar to CISA's Traffic Light Protocol, which provides clear directives without requiring specialized security knowledge.

1. Collaborative Working Group for Standards Development: Quintessence's call to establish a Securing Open Source Communities Working Group is a direct defensive implication. This group aims to collaboratively develop the missing standards and guidance, starting with crisis communication norms (Phase 1) and then moving to threat-testing and abusability-testing common governance models (Phase 2). This collaborative, community-driven approach is crucial for creating widely applicable and effective defensive strategies.

Key Takeaways

• Open Source is Critical Infrastructure, But Its Communities Are in Crisis: Open source software is now a trillion-dollar industry underpinning global critical infrastructure, yet its human communities are experiencing accelerating "collapse or near-collapse" events, with 90% of organizations affected.
• Human-Sided Vulnerabilities Are a Major Attack Vector: Traditional security models focused on code are insufficient. Exploits like the XCutils case demonstrate that maintainer burnout, mental health, and organizational weaknesses (e.g., lack of documentation, knowledge propagation) are directly exploitable.
• Existing Community Health Tools Are Inadequate: Widespread adoption of Codes of Conduct (like the Contributor Covenant) has not prevented new crisis types, such as moderation and leadership walk-offs, indicating a gap in current community resilience strategies.
• Shift Focus to Community Health and Responsible Friction: Defenders must move beyond a sole focus on threat detection to actively cultivate community health, normalize conflict resolution, and implement "responsible friction" in processes. This builds inherent resilience and slows down potential attackers without fostering paranoia.
• Standardized Documentation and Organizational Threat Awareness Are Essential: Most projects lack organizational threat models and consistent documentation for moderation events, leading to "organizational amnesia." Developing clear guidelines for documenting decisions and integrating threat awareness into contributor ladders is crucial for long-term health.
• Moderation is About Boundaries, Not Just Punishment: Moderators should be empowered to enforce community boundaries and assess "behavior compatibility with community health," rather than being tasked with complex threat analysis. This approach is more sustainable and effective for volunteer teams.
• Collaborative Guidance is Needed for All Projects: There is an urgent need for easily consumable guidance for projects of all sizes, particularly solo maintainers, on topics like risk analysis and abusability testing. Initiatives like the proposed Securing Open Source Communities Working Group are vital to develop these standards collaboratively.

About the Speaker(s)

Quintessence, who also goes by Q, is the Executive Director of the Nively Foundation. Her work focuses on understanding and addressing the critical challenges facing open source communities. As demonstrated in her 39C3 talk, Q has a deep interest in the human and organizational aspects of open source, particularly concerning crisis events, maintainer burnout, and the effectiveness of community governance. She emphasizes the importance of mental health within communities and advocates for practical, community-driven solutions to build more resilient and healthier open source ecosystems. Her initiative to launch a "Securing Open Source Communities Working Group" highlights her commitment to collaborative problem-solving and developing actionable guidance for the broader open source world.