Revealing the Black Box of Device Search Engine: Scanning Assets, Strategies, and Ethical Consideration

Revealing the Black Box of Device Search Engine: Scanning Assets, Strategies, and Ethical Consideration

Speakers: Mengying Wu, D University

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=5OLdqP7BVVE

Overview

In an increasingly interconnected world, the proliferation of internet-connected devices has introduced unprecedented convenience alongside significant security vulnerabilities. This presentation, "Revealing the Black Box of Device Search Engine: Scanning Assets, Strategies, and Ethical Consideration," delivered by Mengying Wu from D University at the NDSS Symposium, delves into the clandestine operations of device search engines. These powerful platforms, often likened to Google for the Internet of Things (IoT), continuously scan the internet to index open ports, identified services, product information, and even screenshots of exposed devices. The talk critically examines the methodologies employed by these engines, their impact on internet security, and the profound ethical implications of their scanning practices.

The core motivation behind this research stems from a fundamental question: Are these device search engines always operating as "white hats" to enhance security awareness, or do their actions inadvertently, or even directly, contribute to privacy breaches and expose critical infrastructure to malicious actors? The paper presented is the first comprehensive measurement study focused on understanding the internal working strategies of these engines. By meticulously analyzing their scanning patterns, protocol identification techniques, and the resulting data exposure, the research sheds light on how these platforms operate, the risks they introduce, and what ethical responsibilities they should uphold.

This talk is crucial for anyone involved in cybersecurity, from network defenders and system administrators to policymakers and device manufacturers. It highlights the often-overlooked "black box" nature of tools that are widely used by both security researchers and malicious actors. Understanding the intricacies of how these engines discover and catalog internet-connected devices is paramount to developing effective defensive strategies, mitigating privacy risks, and fostering a more ethically responsible approach to large-scale internet scanning.

Background

▶ Watch: Key research questions: scanning strategies, service identification, privacy ... (1:00)

The pervasive presence of internet-connected devices, ranging from smart home gadgets to industrial control systems, has dramatically expanded the attack surface for cyber adversaries. Attackers no longer need to conduct brute-force scanning themselves; instead, they leverage sophisticated device search engines to efficiently identify vulnerable targets. As highlighted in the talk, botnets like Shulu can pinpoint numerous victims in seconds, largely due to the comprehensive indexing performed by these engines. These platforms, including prominent names like Shodan, Censys, Fofa, and ZoomEye, function by continuously scanning vast swathes of the internet, cataloging open ports, identifying running services (e.g., SSH on port 22), product banners, and even capturing screenshots of web interfaces.

The problem, as the speaker articulates, is that while these engines can be invaluable tools for security researchers to identify exposed assets and improve defensive postures, their powerful capabilities raise significant questions about their operational ethics and potential for misuse. Prior to this research, there was a notable gap in understanding the specific working strategies employed by these engines. Most studies focused on the results of their scans (i.e., what devices they find), but not how they achieve these results, what specific techniques they use to identify services, or the direct security and privacy consequences of their scanning activities. This lack of transparency makes it difficult for organizations to understand why their assets appear on these platforms and how to best protect themselves. The research presented aims to fill this critical gap by providing the first measurement study to dissect these internal strategies and their ethical implications.

Key Findings

▶ Watch: Revealed scanning strategies: neighbor port probing and fallback (4:39)

The research addresses three fundamental questions concerning device search engines: their scanning strategies, their service identification methods, and the security and privacy implications of their activities.

To overcome the challenge of differentiating engine scanning activities from other internet traffic, the researchers devised an ingenious methodology based on IP mirror services. These are network services that, for debugging or error reporting purposes, reflect the connecting client's IP address back in their response (e.g., a SIP service showing the connection address). By identifying and monitoring these services, the research team could accurately record the IPs of the scanning engines. They identified three primary formats for these reflected IPs: standard IPs (e.g., MySQL connection errors), reverse IPs (often used in SMTP for reverse DNS records), and URL-encoded IPs (common in HTTP services).

Over a year-long data collection period, the study identified 1,447 distinct scanner IPs across four major device search engines: Shodan, Censys, Fofa, and ZoomEye. Honeypots were then deployed to monitor the actions of these identified scanners, providing granular insights into their operational tactics. Key observations included:

• IP Lifespan and Management: Censys and Fofa were observed to use relatively static IP addresses, while Shodan demonstrated a strategy of regularly changing its scanning IPs approximately every three months. ZoomEye's IP management was less clearly defined but did not exhibit the "vex scanning assets" pattern seen in Shodan.
• Protocol Identification Strategies:
• Neighbor Strategy: Engines don't just probe default ports. For instance, for RDP (port 3389), they also scan adjacent ports like 3388 and 3390, attempting to discover services migrated to non-standard, but predictably close, ports.
• Fallback Strategy: If initial attempts to identify a protocol on a specific port fail (e.g., HTTP/S), engines will "guess" other common services like FTP. ZoomEye was specifically noted for its aggressive attempts to identify every possible RDP service.
• Infiltration Scanning: Beyond simple port and protocol identification, many engines perform deeper probes, akin to "trying to open the door" rather than just knocking. This includes attempting to access sensitive paths, configurations, databases, and file lists, and then displaying these findings on their public websites.
• Ethical Dimensions: The study rigorously evaluated the engines against a framework derived from five ethical references, concluding with three key dimensions: transparency, harmlessness, and anonymity.
• Transparency: Fofa and ZoomEye failed to provide clear identification for their scans, making it difficult for users to determine the source. Surprisingly, despite advocating for ethical scanning, Censys did not fully implement transparency measures.
• Harmlessness: Many engines, particularly Shodan, go beyond benign scanning. They actively infiltrate databases, nodes, configurations, and file lists. This "infiltration scanning" exposes weakly protected hosts, revealing authentication details and sensitive data, a practice the researchers argue violates the principle of data minimization. The study quantitatively showed Shodan's significant "sensitive passive success" in infiltrating honeypots.
• Anonymity: The research uncovered alarming instances of privacy leakage. Engines were found to display phone numbers from LDAP and database indexes, CA keys, and even screenshots of live systems. Shodan, in particular, was highlighted for having a website dedicated to browsing screenshots, which requires membership to access, effectively "selling the privacy" of exposed systems. The community section of Shodan also showed top-voted queries focused on web cameras, including "live sex camera," indicating the potential for abuse for illicit spying and content sales.

These findings collectively paint a detailed picture of device search engine operations, revealing a complex landscape where powerful data collection methods often intersect with significant ethical shortcomings and security risks.

Technical Deep Dive

▶ Watch: Ethical framework: transparency, harmlessness, and anonymity dimensions (6:40)

The methodology employed by Mengying Wu and her team to "reveal the black box" of device search engines is both innovative and robust. The primary challenge was to distinguish the scanning activities of specific engines from the vast background noise of internet traffic. Their insight centered on leveraging IP mirror services. These are services designed to echo the client's connecting IP address back to them, often for diagnostic or logging purposes.

The process began with a primitive study where researchers manually identified initial mirror-type services. An example given was a MySQL server that would return an error message indicating "this IP is not allowed to connect to the server," thereby revealing the scanner's IP. Similarly, SMTP services often include the client's IP in reverse DNS records, and HTTP services might embed it in URL-encoded formats. Once an engine's IP was identified through one of these mirror services, that IP was then used to find new mirror types. This iterative process allowed the researchers to progressively expand their catalog of scanner IPs and mirror service types.

Over a year, this iterative collection yielded 1,447 unique scanner IPs associated with Shodan, Censys, Fofa, and ZoomEye. With these IPs, the researchers then deployed honeypots – decoy systems designed to attract and record scanning activity – to monitor and analyze the specific actions of these engines. This allowed them to capture the actual network traffic and understand the engines' protocol identification strategies.

The study revealed distinct IP management strategies among the engines. Censys and Fofa appeared to use relatively stable IP addresses for their scanning operations. In contrast, Shodan was observed to regularly rotate its scanning IPs, changing them approximately every three months, a tactic that could be interpreted as an attempt to evade detection or mitigate blocking. ZoomEye's pattern was less defined in terms of dynamic IP changes.

Beyond basic port scanning, the engines employed sophisticated protocol identification techniques:

• Neighbor Strategy: This involves probing ports adjacent to well-known default ports. For instance, instead of exclusively targeting RDP on port 3389, engines would also scan 3388 and 3390. The implication for defenders is clear: simply shifting a service to a slightly different, but predictable, port offers minimal security through obscurity.
• Fallback Strategy: When an initial probe fails to identify a service (e.g., an HTTP request to a non-HTTP port), engines don't give up. They "fallback" to guessing other common protocols. For example, if an HTTP probe fails, they might try FTP. ZoomEye was specifically highlighted for its persistent attempts to identify RDP services, suggesting a targeted focus on this protocol.
• Infiltration Scanning: This is perhaps the most concerning technical aspect. Unlike benign scanners that might send a simple "ping" or a service-specific query (e.g., "are you okay?" to a ZooKeeper instance), these engines often engage in deeper, more intrusive probes. The analogy used was "trying to open the door" rather than just confirming its presence. The honeypot data revealed that engines, particularly Shodan, were attempting to access sensitive directories, database nodes, configuration files, and even full file lists. This goes beyond mere service identification; it's an active attempt to gather intelligence from the target system.

The ethical framework developed by the researchers, based on five references and distilled into transparency, harmlessness, and anonymity, provided a critical lens through which to evaluate these technical operations.

• Transparency: The lack of clear identification in Fofa and ZoomEye scans means network defenders cannot easily distinguish legitimate security research scans from potentially malicious activity, hindering incident response. Even Censys, despite its ethical stance, did not fully implement transparency.
• Harmlessness: The data minimizing principle suggests that scanners should only confirm a service's presence, not extract sensitive data. However, the infiltration scanning observed directly contradicts this. The successful infiltration of databases, exposed configurations, and authentication details, particularly by Shodan, demonstrates a direct disregard for harmlessness. The "sensitive passive success" metric quantified the extent to which Shodan extracted sensitive information from the honeypots.
• Anonymity: The technical ability to capture and display screenshots, coupled with the extraction of Personally Identifiable Information (PII) like phone numbers from LDAP or database indexes, and even CA keys, represents a profound failure in anonymity. Shodan's practice of offering screenshot browsing behind a membership paywall was explicitly called out as "selling the privacy" of exposed systems. The existence of community queries for "live sex camera" further underscores the potential for severe privacy violations and illicit exploitation facilitated by these technical capabilities.

In essence, the technical deep dive reveals that device search engines are not merely indexing services; they are actively probing, inferring, and often infiltrating systems to extract detailed information, much of which poses significant security and privacy risks.

Demo / Proof of Concept

▶ Watch: Harmlessness concern: engines infiltrate and expose sensitive data (7:20)

The talk primarily focuses on a measurement study of existing device search engines and their observed behaviors. As such, the presentation did not include a live demonstration or a proof-of-concept of a new tool or exploit developed by the researchers. Instead, the "proof" of their findings came from the extensive data collected through their IP mirror service methodology and honeypot monitoring, which provided direct evidence of the scanning strategies and data infiltration activities of the prominent device search engines discussed. The results presented, such as the "sensitive passive success" rates for Shodan and the examples of leaked PII and screenshots, serve as empirical evidence supporting their claims.

Defensive Implications

▶ Watch: Anonymity concern: privacy leakage, exploitation, and illicit spying (8:20)

The detailed insights into the strategies of device search engines provide critical intelligence for network defenders seeking to protect their assets. Understanding how these engines operate allows for more informed and effective defensive postures.

1. Randomize Service Ports Effectively: The neighbor strategy employed by engines means that simply shifting a service to a port adjacent to its default (e.g., RDP from 3389 to 3388 or 3390) offers negligible security. Defenders should instead migrate services to truly random, non-sequential ports if they intend to rely on security through obscurity. However, this should not be the primary defense.
2. Implement Strong Access Controls and Authentication: Given the prevalence of infiltration scanning, relying solely on port obscurity is insufficient. All services, regardless of their port, must be protected by robust authentication mechanisms. Default credentials must be changed immediately, and multi-factor authentication (MFA) should be enforced wherever possible, especially for administrative interfaces and critical services.
3. Practice Data Minimization: Device search engines often expose sensitive configurations, database structures, and even PII. Defenders must ensure that services are configured to expose the absolute minimum amount of information necessary.

• Databases: Do not expose database indexes, table names, or entries to unauthenticated access. Ensure proper firewall rules restrict access to only trusted IPs.
• APIs and Configuration Files: Secure API endpoints and configuration files. They should not be publicly accessible or easily discoverable. Avoid hardcoding sensitive information.
• Banners and Headers: While less critical, overly verbose service banners can provide attackers with version numbers and other details. Configure services to minimize this information.

1. Regularly Review Publicly Exposed Assets: Defenders should proactively use device search engines (like Shodan, Censys, Fofa, ZoomEye) themselves to audit their own external-facing assets. Search for their organization's IP ranges, domain names, and common service banners to identify what information is publicly visible. This "attacker's perspective" is crucial for discovering unintended exposures.
2. Be Wary of Diagnostic Services: The reliance on IP mirror services for identifying scanner IPs highlights that even seemingly benign diagnostic services can inadvertently aid in scanner identification. While essential for debugging, ensure these services are not publicly exposed without strict access controls.
3. Secure Screenshots and Web Interfaces: The alarming finding about screenshots and web camera access underscores the need for stringent security on web-facing interfaces. Ensure all web cameras, NVRs, and IoT devices are behind firewalls, use strong, unique passwords, and are updated regularly. If a web interface is not intended for public access, it should not be public.
4. Advocate for Ethical Scanning Practices: The research highlights significant ethical shortcomings. Defenders and security researchers should advocate for greater transparency from device search engine operators, pushing for clear identification of scans, adherence to data minimization principles, and robust anonymization of sensitive data.

By adopting these defensive strategies, organizations can significantly reduce their attack surface and mitigate the risks posed by the pervasive and often intrusive scanning activities of device search engines.

Key Takeaways

• Device search engines are powerful but operate as black boxes: Platforms like Shodan, Censys, Fofa, and ZoomEye continuously scan the internet, indexing vast amounts of information about connected devices. However, their specific scanning strategies and ethical considerations have historically been opaque.
• Sophisticated scanning strategies are employed: Engines use advanced techniques such as the neighbor strategy (scanning adjacent ports) and fallback strategy (guessing protocols if initial probes fail) to identify services, even those moved from default ports.
• Infiltration scanning poses significant risks: Beyond simple port identification, many engines, notably Shodan, engage in infiltration scanning, attempting to access sensitive paths, configurations, databases, and file lists. This exposes weakly protected hosts and can leak critical authentication details and PII.
• Major ethical shortcomings exist across transparency, harmlessness, and anonymity: The study found that engines often lack transparency in identifying their scans, violate the principle of data minimization by collecting excessive sensitive data, and fail to anonymize information, leading to the exposure of phone numbers, CA keys, and even live screenshots.
• Privacy leakage is a severe concern: The public availability of screenshots, PII from databases and LDAP, and the existence of communities sharing queries for "live sex cameras" demonstrate how device search engines can facilitate illicit activities and severe privacy violations.
• Defenders must be proactive: To mitigate risks, organizations should randomize service ports effectively, enforce strong authentication, practice data minimization, regularly audit their public-facing assets using these very engines, and advocate for higher ethical standards in internet-scale scanning.

About the Speaker(s)

Mengying Wu is a researcher affiliated with D University. Her work focuses on understanding the complex landscape of internet-connected devices and the security implications of large-scale scanning operations. This presentation at the NDSS Symposium highlights her expertise in measurement studies, network security, and the critical ethical considerations surrounding data collection on the internet. Her research aims to shed light on the practices of device search engines to inform better defensive strategies and promote more responsible conduct in the cybersecurity community.