Light in the Dark(net)

Light in the Dark(net)

Speakers: Tobias Höller

Conference: 39C3

YouTube: https://www.youtube.com/watch?v=4KsXk3RjDD8

Overview

Tobias Höller's talk, "Light in the Dark(net)," delves into the profound challenges of accurately measuring and interpreting activity on Tor's onion services, often colloquially referred to as the darknet. Höller, a seasoned Tor community member, relay operator, and researcher, highlights why "science is hard" in this domain, particularly for those attempting to quantify illicit activities like the prevalence of marketplaces. The presentation meticulously dissects the methodological pitfalls that lead to wildly contradictory research findings, offering critical insights into the limitations of current data collection and analysis techniques.

The talk is crucial for anyone seeking an empirical understanding of the darknet, from policymakers and law enforcement to fellow researchers and cybersecurity professionals. Höller exposes how fundamental changes to the Tor network, specifically the transition from version 2 (v2) to version 3 (v3) onion services, have drastically altered what can be observed and, consequently, what conclusions can be drawn. By illustrating the impact of biased data sources, flawed extrapolation, inconsistent definitions, and even human error, Höller provides a vital framework for critically evaluating existing studies and approaching future research with greater rigor and awareness of inherent limitations.

Background

▶ Watch: Introduction and the challenge of conflicting darknet research (0:00)

Tobias Höller’s extensive experience within the Tor community forms the bedrock of his insights. Having operated Tor relays for half a decade and conducted research on the network for over six years, he possesses a deep, practical understanding of its architecture and operational nuances. The talk assumes a basic familiarity with Tor's function as an anonymity network, where users connect to services without revealing their identity or location. However, it quickly zeroes in on Tor onion services, distinguished by their .onion addresses, which allow services to be hosted anonymously within the Tor network itself.

A pivotal point in understanding darknet research lies in the evolution of Tor's onion service protocol. Historically, v2 onion services were relatively easy to enumerate. The process involved a Hidden Service Directory (HSD), a distributed system of volunteer-operated Tor relays. When an onion service came online, it would publish a "descriptor" containing its address and other information to the HSD. A client wishing to connect would query the HSD, retrieve this descriptor, and then establish a connection. As a volunteer HSD operator, Höller reveals that in the v2 era, it was possible to read and even track these uploaded descriptors, allowing researchers to discover virtually every active v2 onion service. This capability enabled comprehensive studies of the v2 darknet.

However, the Tor Project intentionally phased out v2 onion services, fully sunsetting them in 2021, primarily for security and privacy enhancements. They were replaced by v3 onion services, which represent a fundamental paradigm shift. V3 addresses are significantly longer and incorporate cryptographic elements designed to prevent passive enumeration. With v3, the service still publishes a descriptor to the HSD, but this descriptor is now encrypted. The HSD only sees an encrypted blob and a unique identifier; it does not know the actual onion address. For a client to connect, they must already possess the full v3 onion address, which contains the necessary cryptographic information to decrypt the descriptor. This means that unlike v2 services, v3 services are not enumerable by simply monitoring the HSD. Researchers can only study v3 services whose addresses have been explicitly shared or discovered through other means. This change has profound implications: while v2 studies could analyze the entire population of services, v3 studies are inherently limited to analyzing a sample, the size and representativeness of which are often unknown. The scale of this shift is immense: from approximately 40,000 v2 onion services in 2014 to an estimated 800,000 to 1 million v3 services today, a twenty-fold increase that vastly complicates comprehensive analysis.

Key Findings

▶ Watch: Understanding why scientific darknet research is difficult (3:00)

Höller's primary finding is that seemingly contradictory research results regarding darknet usage are often not contradictory in their underlying data but in their methodologies, interpretations, and the inherent limitations imposed by the Tor network's design. He illustrates this with a compelling example: studies claiming marketplace prevalence on the darknet ranging from 3% to 50%. This vast discrepancy, Höller explains, arises from several critical factors:

1. Biased Data Sources: Researchers rely on various methods to collect onion addresses, each introducing specific biases:

• Google/ChatGPT: General search engines are poor sources, often outdated or unspecialized.
• Ahmia: A specialized Tor search engine, Ahmia filters out known illegal content, particularly child abuse sites. Studies using Ahmia will, by definition, underrepresent these categories.
• GitHub: Often contains onion addresses related to distributed networks (e.g., blockchain bootstrapping nodes) and ransomware operations (as threat intelligence tracks these). This skews results towards these specific types of services.
• Certificate Transparency (CT) Logs: These logs record SSL/TLS certificates issued for websites. Onion services that obtain certificates (e.g., Facebook, ProtonMail) are typically legitimate and do not prioritize anonymity for the service itself. Studies using CT logs will predominantly find legally operating, non-anonymous services, likely underrepresenting illicit activities.

These varied sources mean that depending on where a researcher gets their list of addresses, they can often predict the study's outcome.

1. Flawed Extrapolation: A common mistake is to extrapolate findings from a small, biased sample to the entire, unknown population of onion services. For instance, if 50% of 30,000 GitHub-sourced addresses are marketplaces, it's erroneous to assume 50% of all 800,000+ onion services are marketplaces. Höller points out that services like illegal marketplaces and scam sites actively want to be found, so they widely disseminate their addresses. This makes them overrepresented in easily discoverable samples, rendering simple extrapolation highly inaccurate and "at the very least a daring assumption."

1. Inconsistent Definitions: The lack of standardized definitions for categories like "marketplace" or "illegal content" further exacerbates discrepancies. Some studies might define a marketplace as any site with a shopping cart and account creation, while others might consider any site with the word "buy" a marketplace. Höller cites a particularly egregious example where a paper defined child abuse sites as any containing the words "boy," "girl," and "hot," which could lead to vastly inflated numbers. Such definitional variances make cross-study comparisons extremely difficult.

1. Human Error and Dynamic Networks: Höller candidly admits to his own published errors (a factor of four mistake in his calculations) and points out similar issues in other reputable sources (Tor metrics being off by 10-15%). The dynamic nature of the Tor network, which constantly changes (e.g., v2 to v3 transition, services coming and going), also makes reproducing and validating studies challenging, as the "thing being studied" is never static.

Höller then synthesizes the current state of knowledge for v3 onion services:

• The HSD remains the best source for estimating the total number of existing onion services.
• However, only about one-third (1/3) of these uploaded services are discoverable by researchers using current methods. The remaining two-thirds (2/3) are effectively unknown.
• Based on old v2 data, Höller speculates that perhaps one-third of these unknown services might be botnets, but stresses this is an unconfirmed guess.
• Certain categories, like onion chat/messaging apps (e.g., Briar, Ricochet), are inherently undiscoverable as their addresses are never published.
• To gain a more accurate understanding of usage rather than mere existence, Höller advocates focusing on successful downloads. This filters out services that are offline or those receiving failed connection attempts.
• Bitcoin nodes constitute a significant share of successful downloads, largely because their addresses are easily discoverable via the public blockchain.
• Clearnet mirrors (e.g., Facebook, Pornhub) and ransomware sites are other readily attributable categories.

Technical Deep Dive

▶ Watch: How Tor's Hidden Service Directory works for onion services (4:00)

The core technical challenge discussed by Höller revolves around the architectural differences between Tor's v2 and v3 onion services and how these impact data collection.

In the v2 onion service model, a service would generate an onion address (e.g., facebookcorewwwi.onion) and a descriptor containing its public key and introduction points. This descriptor was then uploaded to a subset of the Hidden Service Directory (HSD) relays. When a client wanted to connect, it would query the HSD for the service's descriptor. The HSD would return the descriptor, revealing the onion address and enabling the client to establish a connection. Critically, as Höller, a volunteer HSD operator, explains, the HSD relays could read these descriptors. This meant that by monitoring HSD traffic, researchers could enumerate virtually all active v2 onion services.

The transition to v3 onion services introduced fundamental cryptographic changes specifically designed to prevent this enumeration. V3 onion addresses are significantly longer (e.g., facebookwkhpilnemxj7asjfvzh6tvutadgjmklpqg7npb5c6ixx2f4hccgyn5ryad.onion) and incorporate concepts like key blinding and other cryptographic magic. When a v3 service comes online, it still uploads a descriptor to the HSD. However, this descriptor is now encrypted. The HSD only receives an encrypted blob and a randomly generated identifier for that blob. The HSD itself has no knowledge of the actual onion address or the content of the descriptor.

For a client to connect to a v3 service, it must already know the full v3 onion address. This address contains embedded cryptographic material that allows the client to:

1. Derive the correct identifier for the service's encrypted descriptor.
2. Request this identifier from the HSD.
3. Upon receiving the encrypted blob from the HSD, decrypt it using information derived from the onion address.
4. Extract the introduction points and establish a connection.

This design means that the HSD, while still facilitating the connection, no longer reveals the onion address to third parties. Only the service operator, by distributing the address, can make the service discoverable. This makes passive enumeration of v3 services by monitoring the HSD practically impossible, a deliberate security and privacy enhancement by the Tor Project.

Höller's own research methodology leverages his position as a Tor relay operator. He operates 50 Tor relays, which collectively constitute approximately 1% of the entire HSD. The HSD design ensures that relays are randomly assigned a share of the network's load daily. To collect data, Höller made a minimal modification to the Tor source code – adding "two lines of code" to log descriptor uploads and downloads. He then compiled and ran his custom Tor binary as a relay.

His data collection focused on a year-long observation period (which he stated was for 2024, likely a forward-looking statement or a slip of the tongue given the 39C3 talk in late 2023). This allowed him to observe a random 1% sample of all onion service uploads and downloads. A crucial distinction in his analysis is between "uploads" (measuring existence) and "successful downloads" (measuring usage). He found that a significant portion of download attempts fail. His method for detecting failed downloads is straightforward: if a client requests a descriptor via an identifier, and Höller's relay, as the responsible HSD node, cannot provide it (because it was never uploaded or is no longer active), it returns a "404 not found" error. This allows him to filter out non-existent or offline services, providing a more accurate picture of active usage.

Ethical considerations were paramount in Höller's research. He consulted the Tor Project's research safety ethics board to ensure his data collection practices were responsible. He also implemented measures to add noise to his collected data, making it less useful for potential law enforcement investigations and protecting Tor users' privacy.

Demo / Proof of Concept

▶ Watch: Impact of Tor v2 to v3 onion service migration (6:00)

The talk did not feature a live demo or a demonstration of a specific tool or exploit. Instead, Tobias Höller provided a detailed explanation of his research methodology, including the modifications he made to the Tor source code and how he collected data as a Tor relay operator. While no interactive demonstration was part of the presentation, the technical deep dive effectively served as a conceptual proof of concept for his research approach to understanding Tor network activity.

Defensive Implications

▶ Watch: Research methods changed with Tor v2 vs. v3 (7:20)

The insights presented by Tobias Höller carry significant implications for various stakeholders involved in cybersecurity and darknet monitoring:

1. For Law Enforcement and Intelligence Agencies:

• Enumeration is Largely Obsolete: The era of easily enumerating all active onion services is over with the deprecation of v2 and the design of v3. Any intelligence relying on comprehensive darknet enumeration without external leads is fundamentally flawed.
• Targeted Discovery: Discovery of v3 onion services now requires active investigation, relying on addresses shared in public forums, leaked data, or through specific intelligence operations. Passive monitoring of the HSD is no longer sufficient.
• Beware of Data Biases: Law enforcement agencies must critically evaluate any research or intelligence reports about darknet activity, understanding the inherent biases of the data sources used (e.g., GitHub will overrepresent ransomware, CT logs will overrepresent legitimate services).
• Focus on Usage Metrics: Shifting focus from mere "existence" (uploads) to "successful downloads" provides a more realistic understanding of active service usage, helping to prioritize investigations.

1. For Cybersecurity Researchers:

• Methodological Rigor is Paramount: Researchers must be meticulously transparent about their data sources, collection methodologies, and the limitations these impose.
• Acknowledge Sampling Bias: It is crucial to explicitly state that current v3 onion service research is based on samples, and these samples are often biased. Extrapolating from these samples to the entire darknet population is highly problematic.
• Standardize Definitions: The community needs to work towards standardized definitions for categories like "marketplace," "scam site," or "child abuse content" to enable meaningful comparison across studies.
• Reproducibility Challenges: The dynamic nature of the Tor network makes direct reproducibility difficult. Researchers should focus on validating methodologies and assumptions rather than expecting identical results over time.
• Ethical Considerations: Researchers operating Tor relays or collecting network data must adhere to strict ethical guidelines, consulting bodies like the Tor Project's research safety ethics board and implementing privacy-preserving measures for user data.

1. For Tor Users and Onion Service Operators:

• Enhanced Privacy for V3: The v3 onion service protocol effectively achieves its goal of preventing passive enumeration. This means that an onion service operator who does not publicly share their address gains a significant privacy advantage against external observers trying to discover their service.
• Trust in Tor's Design: The changes implemented by the Tor Project are demonstrably working as intended to enhance the anonymity and unreachability of services that wish to remain private.

In essence, the defensive implication is a call for greater realism and critical thinking. The "darknet" is not a monolith that can be easily measured or understood through simplistic methods. Its inherent design, particularly with v3 onion services, ensures a significant degree of opacity, making informed decisions reliant on a deep understanding of methodological limitations and data biases.

Key Takeaways

• Quantifying activity on Tor's onion services (the darknet) is exceptionally challenging, often leading to wildly contradictory research results due to inherent limitations and methodological flaws.
• The shift from v2 to v3 onion services fundamentally altered the landscape of darknet research; v3 services are designed to prevent passive enumeration, making comprehensive data collection impossible.
• Discrepancies in research findings often stem from biased data sources (e.g., Ahmia, GitHub, Certificate Transparency logs), flawed extrapolation from limited samples, and inconsistent definitions of content categories.
• Current research methods can only discover approximately one-third (1/3) of all existing v3 onion services; the vast majority (two-thirds) remain unknown and largely unknowable by design.
• Focusing on "successful downloads" provides a more accurate measure of active onion service usage compared to simply counting "uploads" (existence), filtering out inactive or unreachable services.
• For defenders, understanding these inherent limitations and biases is crucial for realistic assessments of darknet threats and activities, as direct enumeration for intelligence gathering is largely obsolete.

About the Speaker(s)

Tobias Höller is a dedicated and long-standing member of the Tor community. He has actively contributed to the network's operation as a relay operator for half a decade and has been conducting research on the Tor network for the better part of the last six years. His work primarily focuses on various aspects of Tor, with a particular emphasis on the usage and characteristics of Tor onion services. His deep technical knowledge and practical experience from operating parts of the Tor infrastructure provide a unique and authoritative perspective on the challenges of understanding and measuring activity within the network.