Not an Impasse: Child Safety, Privacy, and Healing Together

Not an Impasse: Child Safety, Privacy, and Healing Together

Speakers: Kate Sim, Director, Children's Online Safety and Privacy Research (COSPER) program

Conference: 39C3

YouTube: https://www.youtube.com/watch?v=8P7M5QsCa7I

Overview

In "Not an Impasse: Child Safety, Privacy, and Healing Together," Kate Sim, Director of the Children's Online Safety and Privacy Research (COSPER) program, challenges the pervasive framing of online child safety as a zero-sum game against adult privacy. Sim argues that this binary, which she terms an "impass," paralyzes effective engagement with the complex social, moral, political, and technological dimensions of the issue. The talk dissects how current regulatory and industry approaches, often driven by fear and a "child protectionist" mindset, conflate harm with content, prioritize punitive measures, and ultimately fail to protect children effectively while eroding privacy for all.

Sim's presentation is a critical examination of the prevailing child sexual abuse material (CSAM) detection ecosystem, exposing its inefficiencies, inherent biases, and detrimental impacts on both survivors and those falsely implicated. Drawing on 15 years of experience in sexual violence prevention and response, including frontline work and a stint at Google's child safety policy team, she provides a unique perspective on how technological solutions are often misaligned with the realities of child sexual abuse (CSA). The core argument is that by moving beyond a narrow focus on detection and punishment, and instead investing in a public health approach centered on prevention, care, and community-building, we can foster genuinely safe and supportive digital environments for young people.

This talk is crucial for technologists, policymakers, and advocates alike, as it not only deconstructs the problematic narratives surrounding online child safety but also offers a radical paradigm shift. Sim calls upon the tech community, with its unique expertise, to reorient its efforts towards developing privacy-preserving help-seeking tools and forging critical connections with frontline child rights organizations. By doing so, she contends, we can transcend the current impasse and collaboratively build digital futures that prioritize healing and well-being for all children.

Background

▶ Watch: Challenging the child safety vs. privacy 'impasse' (0:00)

The "impass" Sim describes is a deeply entrenched narrative pitting children's online safety against adult privacy, creating a false binary that hinders constructive solutions. This framing is evident in global policy debates, from discussions on age assurance and social media bans to proposals for client-side scanning and the broader rhetoric of "moral panic versus Trojan horse." Sim emphasizes that these binaries are not only inaccurate but actively detrimental, masking the intricate realities of child sexual abuse (CSA).

Sim's personal journey grounds this critique. With 15 years in sexual violence prevention and response, she witnessed firsthand how the safety-privacy divide repeatedly undermined efforts to support survivors. Her experience spanned frontline support, community organizing, advocacy, research, and policy across the US and Seoul. She observed how systemic factors—capitalism, imperialism, patriarchy—created conditions for sexual violence, and how the criminal legal system often dismissed tech-facilitated harms, blaming victims or trivializing their experiences. Critically, her time within Google's child safety product policy team revealed the tech industry's "greed to feed its surveillance business model," where claims of privacy and safety were often reduced to marketing and compliance exercises, sidelining innovative prevention ideas in favor of "hyping up the AI bubble" (04:00).

The current legislative landscape globally reflects this protectionist approach. Australia has pioneered social media minimum age restrictions and the UK implemented age assurance methods via its Online Safety Act. The EU grapples with "child control" debates alongside the DSA and DMA, while the US considers the Kids Online Safety Act (KOSA). In the Asia-Pacific, the focus is on cyber sex crimes and financial extortion, and African nations like Nigeria, Uganda, and Ghana are enacting online harms and anti-homosexuality bills that include online content moderation. Brazil stands out for its innovative data governance framework for children, offering a potential alternative model (04:00-06:00).

These policies share three problematic commonalities:

1. Vague Definition of Child Safety: While ostensibly broad, encompassing grooming, scams, misinformation, and mental health, the language consistently evokes child sexual abuse and child sexual abuse materials (CSAM) as the primary justification for protectionist regulation (06:00). This vagueness allows CSAM to become a fixed mental model, overshadowing other harms.
2. Harm Reduced to Content: This rhetoric confines harm to "the four corners of the content," isolating it from the complex interplay of online and offline factors. It overlooks how risks differ from actual harms, how impacts vary by age, and how harm often emerges from behaviors rather than static content. This singular focus diverts attention from root causes and institutional incentives that enable abuse, instead prioritizing content removal through detection and reporting (07:00).
3. Privatized and Punitive Solutions: The proposed interventions are largely tech solutionist, including age assurance methods, client-side scanning, reporting tools, and grooming classifiers. Crucially, these measures are privatized, enabling the tech industry to centralize more data about everyone's private lives. They are also singularly focused on punishment, operating under the assumption that "catching the bad guys" equates to protecting young people (08:00).

Sim labels this overarching framework the child protectionist approach. This approach collapses meaningful differences in harms, motives, and impacts to justify a blunt, universalized "law and order" response. It frames young people as "frozen, helpless victims," neglecting their evolving needs and the fact that safety is not a special carve-out, but interconnected with broader societal well-being. This protectionism is amplified by what Sim, referencing Elizabeth Clemens, calls the "ick factor" of child sexual abuse. The discomfort and avoidance surrounding CSA, particularly CSAM, leads to a fixation on the "predator" figure, preventing nuanced engagement with the complex realities of abuse (09:00-10:00).

Contrary to the prevalent cultural image of a stranger-predator, the reality of CSA is far more complex:

• Known Offenders: Approximately 90% of CSA involves people known and trusted by the child, often within the family system (14:00).
• Juvenile Offending: Over two-thirds of CSA involves juvenile offenders, frequently in peer-to-peer contexts, with an average age difference of about six years (14:00).
• Repeated Abuse: For 78% of children, CSA happens more than once, and for 42%, it occurs more than six times, indicating a chronic rather than isolated issue (15:00).
• Co-occurring Maltreatment: CSA often happens in tandem with other forms of child maltreatment, such as neglect, physical or emotional violence, and exposure to domestic violence (15:00).
• Vulnerability: The likelihood and severity of CSA are linked to vulnerability, disproportionately affecting girls, children with disabilities (three times more likely), and those in the foster care system (16:00).
• Delayed Disclosure: Disclosure is often significantly delayed, sometimes by decades, with 66% of episodes never disclosed to an adult (16:00).

The "ick factor" and the societal fixation on the predator actively harm survivors by masking these systemic factors and delaying appropriate responses. This protectionist ecosystem, therefore, is not just inaccurate but counterproductive to genuine child safety.

Key Findings

▶ Watch: Global legislative map for children's online safety (4:00)

Sim's analysis reveals a deeply flawed and self-perpetuating ecosystem for detecting online child sexual abuse materials, driven by "big scary numbers" rather than genuine human impact. This ecosystem, while seemingly robust, ultimately prioritizes quantitative metrics over the nuanced realities of child sexual abuse and the needs of those affected.

The central finding revolves around the problematic definition and reporting of CSAM. Sim highlights the commonly cited statistic from the National Center for Missing and Exploited Children (NCMEC), which reported 36.2 million instances of suspected child sexual exploitation in 2023. This "jarring and horrifying number" (18:00) forms the bedrock of the protectionist narrative. However, Sim meticulously unpacks what constitutes "suspected CSAM" under US federal law, which obligates Electronic Service Providers (ESPs) to detect, report, and remove such content.

The US legal definition of CSAM, focusing on "visual depiction of sexually explicit conduct involving an apparent minor" (19:00), proves to be both over- and under-inclusive. Sim identifies several categories of content that fall under this broad umbrella, leading to the inflated numbers:

• Genuine CSAM: Images and content from actual abuse.
• "Baby bum on the beach": Innocuous images, like a grandparent photographing a naked child on a beach, which might technically meet the legal definition but lack sexual motivation.
• Nudes (Self-Generated CAM): Intimate imagery captured and shared by older adolescents, representing developmentally normative behavior. While potentially harmful if shared non-consensually, the image itself doesn't inherently signify abuse.
• Re-shares ("Potential Meme"): Already detected CSAM re-shared by individuals for various reasons, including awareness, outrage, or trolling.
• Obscene Visual Content: Animated or generated content designed to replicate the image of a child, which may not involve a real child.

This broad categorization leads to a significant disparity in the actionability of cyber tips. Of the 36.2 million tips in 2023, law enforcement found only 49% to be "actionable," meaning there was enough information for an investigation. Ultimately, only 3% of the total tips were referred to law enforcement, and a mere 0.18% (63,892 tips) were escalated for "likely involving real hands-on abuse of a real child" (22:00). This stark contrast between the initial "big scary number" and the final actionable instances underscores the system's inefficiency and overreach. Sim argues that to truly identify abuse, contextual information is paramount—factors like consent, the nature of harmful behavior (coercion, threats, non-consensual sharing), motivation (innocuous, curious, revenge, financial extortion), and relationality (age difference, power dynamics, relationship type) (23:00-24:00). Without this context, an image alone cannot reliably determine abuse. Consequently, Sim proposes redefining CSAM as Child Sexual Material (CSM) to accurately reflect what is actually being captured and reported.

Sim then maps out the self-reproducing and expanding CSM detection ecosystem:

• Electronic Service Providers (ESPs): Companies like Google, Meta, and Microsoft, obligated to design and apply proprietary detection technologies.
• Law Enforcement and Regulators: Investigate CSM tips for policing responses, with entities like the eSafety Commissioner and Ofcom facilitating communication.
• Lobbyists and Campaigner Organizations: Raise awareness about CSM tip reporting, working with clearinghouses.
• Clearinghouses and Tiplines: Government-appointed charities (like NCMEC) that collect, review, and refer tips, maintaining databases.
• Tech Solutions Vendors: A lucrative marketplace of third-party tech companies developing and selling compliance technologies (e.g., analyst review toolkits, intel research reports) to other ecosystem actors (26:00).

This ecosystem, Sim contends, is rife with algorithmic harms. It operates on a loop that constantly implicates young people, often without a clear understanding of the harms being perpetuated. For instance, 44% of CCAM (Child Commercial Sexual Abuse Material) offenders are under 18, and 68% are known to the child. In online commercial sexual exploitation, 92% of cases are self-negotiated by young people, primarily motivated by money (28:00). This complicates the simplistic "predator" narrative.

Furthermore, the system exhibits significant disparate impacts:

• Neurodivergence: Young people with neurodivergence are at higher risk for both victimization and offending, particularly adolescents with autism spectrum disorder (28:00).
• Racial Disparity: In the US, allegations of physical and sexual violence from white and non-Latinx children are more likely to be substantiated than those from Black and Latinx children. Globally, CSAM cyber tips disproportionately feature white children, and arrested viewers are mostly white, raising questions about internet connectivity, jurisdictional compliance, access to devices, and how "innocence" is racialized in detection ecosystems (29:00-30:00).
• Victim Awareness: Hash matching technology, used to identify known CSAM, often relies on imagery from victims who are unaware their material is being used to train detection models. A survey of 80 survivors found 60% were aware their material was retained by police, 45% by tech companies, and 35% by researchers, but 90% wanted to know (31:00).

Crucially, Sim highlights the systemic neglect of a much smaller, chronically underfunded help-seeking ecosystem. This comprises frontline service providers—therapists, shelters, caseworkers, helplines, and youth-serving organizations—who directly support young people. This vital ecosystem is often pulled into the larger detection ecosystem for funding and visibility, rather than being adequately resourced independently (32:00).

Finally, Sim powerfully argues that detection is not prevention, and reporting is not helping. The assumption that "catching the bad guys" equates to supporting young people is false. Mandated reporting, a cornerstone of the detection ecosystem, often does more harm than good. A survey of domestic violence survivors, many of whom were underage, found that over half said law enforcement intervention due to mandated reporting made their situation worse, with only 18% reporting an improvement (34:00). People seek help for CSM viewing habits, but 70% of those who sought help could not receive it due to stigma, lack of resources, and mandated reporting requirements. Young people, especially young men, are more likely to seek help from anonymous, stigma-reducing online services (35:00-36:00). These findings collectively paint a picture of a system that is not only ineffective but actively harmful, diverting resources and attention from genuine, human-centered prevention and care.

Technical Deep Dive

▶ Watch: Problem 1: Vague 'child safety' fixed on CSAM (6:00)

While the talk critiques the application and ecosystem of technology rather than delving into specific code or algorithms, it highlights several key technical concepts and their problematic implementations within the child protectionist framework. The speaker emphasizes that the proposed interventions are frequently tech solutionist, aiming to solve complex social problems with technological fixes. These include:

• Age Assurance Methods: Technologies designed to verify a user's age online. The talk references their implementation in the UK's Online Safety Act. While the specific mechanisms (e.g., facial recognition, ID verification, self-declaration with AI checks) are not detailed, Sim's critique focuses on their privacy implications and their role in a "blunt universalized law and order response" that centralizes more data (08:00).
• Client-Side Scanning: This refers to technologies that scan user content on their devices (e.g., phones, computers) before it is encrypted or uploaded to a cloud service. Though not explicitly named with a specific technology, the concept is a recurring point of contention in privacy debates, and Sim flags it as a problematic "tech solutionist" measure that enables "the tech industry to collect, consolidate, and centralize more data about everyone's private and intimate lives" (08:00).
• Reporting Tools and Grooming Classifiers: These are software-based mechanisms used by Electronic Service Providers (ESPs) to identify and flag content or behaviors that might indicate CSAM or grooming. While the talk doesn't detail the AI/ML models or heuristics used, it implies their proprietary nature ("design and apply proprietary detection technologies" - 26:00) and criticizes their output, which leads to the over-inclusive reporting numbers. The core issue is that these tools often reduce complex human interactions and motivations to content-based signals, leading to misclassification and the "great disparity in the actionability of cyber tips" (22:00).

A significant technical concept discussed is the definition and detection of CSAM itself. Sim explains that the US legal definition—"any visual depiction of sexually explicit conduct involving an apparent minor"—is a "very particular legal object" (19:00). The technical systems are built to identify "visual depiction" (often prioritizing images over text or audio, which varies by jurisdiction), "sexually explicit conduct" (defined by genitalia and penetrative acts, inscribing a heterosexual bias), and "apparent minor" (relying on the assumption that a child's body can be distinguished from an adult's). These technical interpretations are inherent limitations, leading to the inclusion of innocuous content like "baby bum on the beach" or "self-generated CAM" (nudes by older adolescents) in detection outputs. The systems struggle with context, which is crucial for distinguishing abuse from non-abusive material (e.g., a photo doesn't change, but the context of non-consensual sharing does - 24:00).

The ecosystem heavily relies on hash matching technology. This technology is used to identify materials that have already been vetted as "true positive pieces of CSAM" (30:00). When a new image or video is uploaded, its unique digital hash (a fingerprint) is compared against a database of known CSAM hashes. If a match is found, the content is flagged. While effective for known materials, Sim points out critical ethical and privacy concerns:

• Victim Awareness: Many victims of CSAM are unaware that their imagery is being used to train these detection models or is retained in databases. A survey indicated that while 60% knew police retained their material, only 45% knew tech companies did, and 35% knew researchers did, with 90% wanting to be informed (31:00). This highlights a significant transparency and consent gap in the technical process.
• Limitations: Hash matching only identifies known CSAM. It cannot identify newly created abuse material until its hash is added to a database, often after a human review process. It also doesn't provide contextual information, reinforcing the "content-as-harm" fallacy.

The talk implicitly critiques the "proprietary detection technologies" used by ESPs, suggesting a lack of transparency and external oversight regarding their internal workings and biases. The "growing very lucrative marketplace of what I call the tech solutions vendors" (26:00) further highlights how technological compliance has become an industry in itself, often without "substantive child safety expertise," prioritizing sales of tools over effective, human-centered solutions.

In essence, Sim's technical deep dive is not into the mechanics of the algorithms but into the systemic flaws and ethical implications of how these technologies are conceived, implemented, and utilized within a flawed legal and social framework. She argues that the current technological approaches, by focusing on content and punishment, generate "algorithmic harms" and exacerbate existing inequalities, rather than genuinely safeguarding young people.

Demo / Proof of Concept

▶ Watch: Problem 2: Harm reduced to content, ignoring root causes (7:00)

The talk "Not an Impasse: Child Safety, Privacy, and Healing Together" is a critical analysis and paradigm-shifting proposal, rather than a demonstration of a technical tool or a proof of concept. Kate Sim's presentation focuses on deconstructing existing systems and advocating for a new approach, making a traditional demo section inapplicable. The core of her work is theoretical and analytical, aimed at reshaping understanding and policy directions.

Defensive Implications

▶ Watch: Problem 3: Privatized, punitive tech-solutionist interventions (8:00)

The defensive implications stemming from Kate Sim's talk are profound and call for a radical reorientation of efforts within the cybersecurity and tech communities. Instead of focusing solely on detection and punitive measures, defenders—including technologists, policymakers, and frontline workers—must pivot towards a public health model of prevention, care, and community-building.

1. Re-evaluate and Challenge Current Detection Systems: Defenders must critically examine the efficacy and ethical implications of existing CSAM detection ecosystems. The talk highlights that only a tiny fraction (0.18%) of the millions of reported "suspected CSAM" cases actually lead to confirmed hands-on abuse of a real child. This inefficiency, coupled with the over-inclusivity of the definition of CSAM (which Sim renames Child Sexual Material (CSM)), indicates that current systems are resource-intensive yet largely misdirected. Technologists should advocate for greater transparency in how proprietary detection technologies are developed and deployed by ESPs, pushing for external audits and accountability for algorithmic harms and biases, particularly racial disparities.
2. Prioritize Context Over Content: A key defensive strategy is to move beyond the simplistic reduction of harm to content. Security professionals should recognize that true child safety requires understanding the contextual information surrounding an image or interaction—factors like consent, motivation, power differentials, and the nature of harmful behaviors (e.g., coercion, non-consensual sharing). This implies that purely technical, content-based detection solutions, such as hash matching technology or grooming classifiers, are insufficient on their own and can be actively harmful by removing context.
3. Invest in Privacy-Preserving Help-Seeking Tools: Sim explicitly calls upon the tech community to "develop innovative help-seeking tools" that prioritize privacy, anonymity, and confidentiality (40:00). This is a direct defensive measure, as young people are more likely to seek help from services that offer these protections. Technologists should design platforms and applications that facilitate confidential communication, provide anonymous support resources, and are stigma-reducing. This could involve secure messaging apps, encrypted peer support networks, or AI-powered chatbots that guide users to appropriate, private resources without mandated reporting obligations.
4. Advocate for Alternative Regulatory Paths: Technologists have a unique opportunity to "help shape regulatory path with tech alternatives paired with compelling messaging" (41:00). Instead of passively critiquing "moral panic" or "Trojan horse" policies, the tech community should proactively propose privacy-enhancing technical solutions that align with a prevention-first approach. This involves translating complex technical concepts for policymakers, showcasing how technology can facilitate care and support rather than just surveillance and punishment. For example, advocating for strong end-to-end encryption while also demonstrating how secure, private channels can be used for genuine help-seeking, rather than being seen solely as an impediment to detection.
5. Build Critical Connections with Frontline and Child Rights Groups: A crucial defensive strategy is to bridge the gap between the tech community and the underfunded help-seeking ecosystem of frontline service providers (therapists, social workers, youth organizations). Technologists can offer their expertise through "skillsharing," providing data audits, security audits, and data literacy training to these groups (42:00). This collaboration can help frontline workers leverage technology safely and effectively, while also informing tech development with real-world expertise on child development, trauma, and effective intervention strategies.
6. Champion a Public Health Approach to Prevention: The ultimate defensive posture is to shift investment towards prevention as a public health concept. This means moving resources away from the punitive, detection-focused ecosystem towards comprehensive, long-term support structures:

• Primary Prevention: Universal education on comprehensive sex and intimacy, including digital consent.
• Secondary Prevention: Anonymous and therapeutic support for those at higher risk, and training for youth-serving professionals to identify healthy and harmful behaviors.
• Tertiary Prevention: Providing long-term, non-stigmatizing local resources for survivors to live offense and victimization-free lives.
• Quaternary Prevention: Adopting a "do no harm" principle, which includes abolishing harmful practices like mandated reporting in favor of investment in care and community (39:00-40:00).

Defenders should advocate for policy changes that reallocate funding from surveillance-heavy tech solutions to these under-resourced prevention initiatives.

In essence, the defensive strategy is to dismantle the "impass" by challenging the foundational assumptions of current online child safety efforts. It requires technologists to actively engage, not just as builders of tools, but as ethical advocates and collaborators in building a digital world where safety and privacy are not mutually exclusive, but mutually reinforcing through care, community, and genuine prevention.

Key Takeaways

• The "Impass" is a False Binary: The notion that children's online safety and adult privacy are inherently at odds is a dangerous oversimplification that paralyzes effective action and leads to ineffective, harmful solutions.
• Current Protectionist Approaches are Flawed: Policies driven by a "child protectionist approach" and the "ick factor" of child sexual abuse (CSA) fixate on an inaccurate "predator" narrative, leading to tech-solutionist, privatized, and punitive measures that fail to address the systemic roots of abuse.
• CSAM Detection Ecosystem is Inefficient and Harmful: The current system for detecting Child Sexual Material (CSM), fueled by "big scary numbers," is over-inclusive and inefficient. A staggering 0.18% of reported "suspected CSAM" leads to actionable cases of real hands-on abuse, while generating significant algorithmic harms and perpetuating racial and other biases.
• Context is Crucial, Content is Not Enough: Determining genuine abuse requires understanding the full context—consent, motivation, power dynamics, and behavior—which static content or images alone cannot provide. Relying solely on content-based detection tools like hash matching technology overlooks these critical nuances.
• Mandated Reporting Often Worsens Situations: The assumption that reporting always helps is false. Mandated reporting systems frequently deter help-seeking, with a majority of domestic violence survivors reporting that such interventions made their situations worse. Genuine help-seeking requires privacy, anonymity, and confidentiality, which current systems often undermine.
• A Radical Paradigm Shift Towards Prevention is Needed: To achieve enduring child safety, resources must be redirected from detection and punishment to a public health model of prevention, care, and community. This includes comprehensive education, anonymous therapeutic support, long-term local resources, and the abolition of harmful practices like mandated reporting.
• Technologists Have a Critical Role: The tech community is uniquely positioned to break the impasse by developing privacy-preserving help-seeking tools, advocating for alternative regulatory paths, and building critical connections and sharing expertise with underfunded frontline child rights organizations.

About the Speaker(s)

Kate Sim is the Director of the Children's Online Safety and Privacy Research (COSPER) program. With 15 years of dedicated experience in sexual violence prevention and response, her background spans a wide array of roles, including frontline support, community organizing, advocacy, research, and policy development. Sim's professional journey led her to the intersection of sexual violence and technology, where she observed firsthand how the divide between privacy and safety repeatedly hindered effective interventions. She previously worked in Google's child safety product policy team, gaining direct insight into the tech industry's approaches to these complex issues. This presentation marked Kate Sim's first appearance at the Chaos Communication Congress (CCC).