“We can’t change it overnight”: Understanding Industry Perspectives on IoT Product Security Compliance and Certification

“We can’t change it overnight”: Understanding Industry Perspectives on IoT Product Security Compliance and Certification

Speakers: Prianka Mandal; Adwait Nadkarni William & Mary

Conference: IEEE S&P

YouTube: https://www.youtube.com/watch?v=Ng6aD0QoVWg

Overview

The proliferation of Internet of Things (IoT) devices has introduced a complex landscape of security and privacy challenges. In response, regulatory bodies worldwide have begun to introduce initiatives such as the US Cyber Trust Mark, aiming to standardize and improve the security posture of IoT products. This talk, presented by Prianka Mandal, a PhD student at William & Mary, and advised by Professor Adwait Nadkarni, delves into the critical, yet often overlooked, human element of these efforts: the perspectives of IoT practitioners. The research explores how industry professionals perceive, practice, and experience IoT product security compliance and certification, shedding light on the practical hurdles and opportunities for enhancing device trustworthiness.

This study is particularly pertinent given prior research indicating that traditional certification models often fall short, certifying products that still harbor common vulnerabilities. The talk highlights a crucial gap in understanding how the primary stakeholders — IoT vendors, developers, and small-to-medium enterprises (SMEs) — navigate this evolving regulatory environment. By conducting an in-depth interview study, Mandal and Nadkarni provide invaluable insights into budget constraints, time pressures, the efficacy of current certification processes, and the contentious issue of liability, ultimately aiming to inform the development of more effective and widely adopted IoT security standards and frameworks.

The findings from this research are vital for regulators, standard-setting organizations, and the cybersecurity community. Understanding the industry's hesitations, challenges, and proposed solutions is essential for creating compliance and certification models that are not only robust in theory but also practical and achievable in real-world development and deployment cycles. The talk underscores the need for a collaborative approach to uplift IoT security, moving beyond a "checkbox" mentality to foster genuine security by design across the entire IoT ecosystem.

Background

▶ Watch: Flaws in current IoT product security certification model. (0:00)

The rapid expansion of the Internet of Things has brought with it an exponential increase in potential attack surfaces. From smart home devices to industrial sensors, IoT products often lack fundamental security controls, making them attractive targets for malicious actors. Recognizing this pervasive threat, governments and industry consortia globally have initiated various regulatory and standardization efforts. A prominent example is the US Cyber Trust Mark, which aims to provide consumers with an easily recognizable label indicating a product's adherence to certain cybersecurity standards. Similar initiatives exist in other regions, all striving to establish a baseline of security and privacy for IoT devices.

These policy initiatives inherently create a demand for IoT-specific security standards and product certification. The traditional model for security certification involves vendors submitting their products to certification labs, which then evaluate the product against defined standards and issue a certificate. This model is appealing because it promises scalability, allowing labs to handle the technical assessment while regulators focus on licensing the labs. However, a significant flaw in this traditional approach, as identified in the speaker's prior work on IoT app security certification, is that regulators often license these labs based on procedural criteria (e.g., number of employees, testing facilities) rather than their actual performance in finding vulnerabilities. This can lead to a scenario where certified products may not have been rigorously analyzed and could still contain exploitable weaknesses, offering a false sense of security to consumers.

The core problem, therefore, is two-fold: first, the inherent security challenges of a diverse and rapidly evolving IoT landscape, and second, the potential inadequacy of current certification models to truly guarantee product security. While the need for an IoT compliance and certification model is clear, there has been a significant knowledge gap regarding the perspectives, practices, and experiences of the primary stakeholders – the IoT practitioners within the industry. Understanding their views is crucial to designing effective frameworks that bridge the gap between regulatory intent and practical implementation, thereby moving beyond superficial compliance to achieve genuine security improvements.

Key Findings

▶ Watch: Budget and time significantly impact product security. (2:40)

The research, based on interviews with 17 IoT practitioners from diverse backgrounds, yielded 16 findings, a selection of which were highlighted in the talk. These findings offer critical insights into the real-world challenges and perceptions surrounding IoT product security compliance and certification.

Firstly, budget and time emerged as paramount factors significantly impacting product security. Participants consistently noted that without adequate budget, companies struggle to hire essential security experts or invest in robust security testing tools and processes. Time constraints also play a critical role; while some organizations prioritize delivering secure products, independent contractors and those under intense market pressure often prioritize deadlines over comprehensive security measures. Furthermore, the researchers found that pursuing security certification itself is heavily constrained by both time and budget, often restricting companies from even attempting the process.

A particularly salient finding was the disproportionate challenge faced by Small and Medium-sized Enterprises (SMEs). These organizations typically operate with limited budgets, often lack dedicated security expertise within their teams (some even reported not having a security expert for extended periods), and may not possess proper testing facilities. Compounding this, the IoT market is highly competitive, forcing SMEs to rapidly introduce new products. This environment, when coupled with the pressure for certification, can lead to "forum shopping". Forum shopping, in this context, refers to SMEs intentionally choosing certification labs known for cheaper and less rigorous testing, primarily to obtain a certificate rather than genuinely enhance product security. This practice, while yielding a "certified" product, ultimately provides only a false sense of security to consumers.

Regarding practitioners' knowledge and current practices, the study revealed a noticeable gap. Most participants were unaware of specific IoT security standards, though they did certify their products using more general certifications like ISO standards. Despite this, there was strong support among practitioners for the development of dedicated IoT-specific security standards and certifications, believing these could provide better assurance to both vendors and consumers. However, a significant skepticism persists regarding the effectiveness of the current certification process; many practitioners believe it is imperfect and may not genuinely improve product security. Consequently, they are often hesitant to pursue certification unless it becomes mandatory.

On the issue of responsibility, participants generally agreed that both developers and certification labs should share the burden for maintaining proper security compliance. Developers, as the creators of the product, are expected to integrate security standards from the development phase. Certification labs, on the other hand, are expected to rigorously test products to ensure compliance. An interesting and somewhat controversial finding was the industry's stance on third-party libraries. Despite acknowledging that these libraries frequently contain vulnerabilities, participants were firmly against assuming responsibility for vulnerabilities found within them.

Finally, the question of liability in case of certification failure was explored. The overwhelming sentiment among participants was that the certification lab should bear the primary liability if a vulnerability is found in a certified product. Their reasoning was that the lab's core responsibility is to identify such vulnerabilities during testing, and failure to do so indicates a lapse in their duty. This highlights a significant expectation gap and potential legal quagmire in the current certification ecosystem.

Technical Deep Dive

▶ Watch: Industry supports IoT-specific standards, but current process is flawed. (4:20)

While the talk itself presents a qualitative social science study rather than a deep dive into specific code or protocol vulnerabilities, the "technical deep dive" here refers to the underlying technical subject matter being investigated: IoT product security compliance and certification frameworks. The research methodology itself, centered on understanding industry perspectives, serves as a critical technical investigation into the practical implementation and challenges of these frameworks.

The study's focus on IoT-specific security standards is a direct technical concern. The existing landscape often relies on general security certifications like ISO 27001, which while valuable for information security management systems, may not adequately address the unique constraints and attack vectors inherent in IoT devices (e.g., constrained resources, long deployment cycles, physical access, over-the-air update mechanisms). The call from practitioners for dedicated IoT standards underscores a recognition of this technical specificity. Such standards would ideally cover areas like secure boot, secure storage, secure communication protocols (e.g., TLS 1.3, DTLS), secure update mechanisms, data privacy considerations, and robust authentication/authorization for devices and cloud services. The US Cyber Trust Mark and similar initiatives are attempting to define these technical baselines, often drawing from established frameworks like NIST SP 800-213A (IoT Device Cybersecurity Guidance) or ETSI EN 303 645 (Cyber Security for Consumer IoT).

The discussion around the certification process itself is also deeply technical. A rigorous certification process for IoT products would involve a multifaceted approach:

1. Threat Modeling: Identifying potential threats and vulnerabilities specific to the device's function and environment.
2. Code Review: Manual and automated analysis of firmware and software for common vulnerabilities (e.g., buffer overflows, injection flaws, insecure cryptographic implementations).
3. Vulnerability Scanning and Penetration Testing: Actively attempting to exploit weaknesses in the device, its network interfaces, and associated cloud services. This would involve specific IoT attack vectors like hardware tampering, side-channel attacks, and exploitation of proprietary protocols.
4. Configuration Audits: Verifying secure default configurations, strong password policies, and disabled unnecessary services.
5. Supply Chain Security: Assessing the security of third-party components and libraries, a point of contention highlighted by the practitioners. This could involve Software Bill of Materials (SBOM) analysis and vulnerability scanning of dependencies.
6. Privacy Impact Assessments: Ensuring compliance with data privacy regulations (e.g., GDPR, CCPA) for data collected and processed by the device.

The study implicitly critiques the current technical rigor of some certification labs, suggesting that "cheap and lightweight testing" (indicative of forum shopping) bypasses these critical technical assessments. This leads to certified products that technically meet a minimal, often procedural, standard but fail to withstand real-world attacks. The lack of validated performance criteria for labs directly impacts the technical quality of the certification.

Another critical technical aspect is liability for third-party libraries. Modern IoT devices are complex assemblies of open-source and proprietary components. Vulnerabilities in widely used libraries (e.g., OpenSSL, curl, FreeRTOS) can affect millions of devices. The industry's reluctance to assume responsibility for these vulnerabilities presents a significant technical debt and risk. Technically, developers can mitigate this through robust dependency scanning tools (like OWASP Dependency-Check, Snyk, Black Duck), regular updates, and thorough vetting of third-party components. However, this requires dedicated resources and expertise, which, as the study points out, are often lacking, especially in SMEs. The call for a clear legal framework for liability in this context is not just a legal issue but a technical one, as it directly impacts the incentives for adopting secure development practices throughout the software supply chain.

In essence, the "technical deep dive" here is into the design and efficacy of security assurance mechanisms for IoT products, revealing the technical and organizational hurdles that prevent these mechanisms from achieving their intended goal of securing the connected world.

Demo / Proof of Concept

▶ Watch: Industry resists responsibility for third-party library vulnerabilities. (6:00)

This research presents a qualitative study based on interviews with industry practitioners rather than a traditional technical demonstration or a proof of concept of an exploit. Therefore, there was no live demonstration of a vulnerability, a security tool, or a novel attack technique.

Instead, the "demonstration" of this work lies in its methodology and findings. The researchers effectively demonstrated the utility of a qualitative interview study as a powerful tool for uncovering nuanced industry perspectives and systemic challenges in cybersecurity. By interviewing 17 diverse IoT practitioners, they provided empirical evidence of the disconnect between regulatory aspirations and real-world implementation. The "proof of concept" is that a structured sociological inquiry can reveal critical insights into the human and organizational factors that impede or facilitate the adoption of secure practices, an area often overlooked in purely technical security research. The presentation of the 16 findings, particularly concerning budget constraints, SME challenges, forum shopping, and liability disagreements, serves as the output of this research "demonstration," illustrating the practical implications of current IoT security compliance models.

Defensive Implications

▶ Watch: Key takeaways and recommendations for improving IoT security. (6:40)

The findings of this study provide crucial insights for various stakeholders involved in the defense of the IoT ecosystem. Understanding industry perspectives is paramount for designing effective and adoptable defensive strategies.

1. For Regulators and Standard-Setting Bodies:

• Validate Certification Labs: The most significant defensive implication is the urgent need to move beyond procedural criteria for licensing certification labs. Regulators must develop and implement metrics to assess labs' actual performance in finding vulnerabilities. This could involve periodic audits, "red team" exercises against certified products, or requiring labs to disclose their methodologies and vulnerability detection rates. This would directly counter "forum shopping" and ensure that certification genuinely enhances security.
• Accessibility of Standards: Security standards and regulations must be made more accessible and understandable for average developers and SMEs. This means providing clear guidelines, practical examples, and potentially even toolkits that integrate compliance checks into the development lifecycle. Overly complex or abstract standards will be ignored or misinterpreted, leading to insecure products.
• Clear Liability Frameworks: Establishing a clear legal framework for assigning liability in cases of compliance failure is essential. This would provide incentives for both developers and certification labs to take their responsibilities seriously, especially concerning third-party components. Clarity on who is accountable for vulnerabilities, particularly in certified products, would drive better security practices.

1. For IoT Product Developers and Vendors (especially SMEs):

• Proactive Security Planning: The finding that budget and time heavily impact security underscores the need for proactive security planning. Security should be integrated into the product development lifecycle (Security by Design) from the very beginning, rather than being an afterthought. This includes allocating dedicated budget for security experts, secure coding training, and robust testing from the initial phases.
• Embrace IoT-Specific Standards: Even if not mandatory, actively pursuing and understanding IoT-specific security standards is crucial. These standards provide a roadmap for building more resilient products. Developers should advocate for and contribute to the development of such standards.
• Rethink Third-Party Library Responsibility: While practitioners are hesitant, the reality is that vulnerabilities in third-party libraries become vulnerabilities in their products. Developers must implement robust Software Supply Chain Security practices, including using Software Bill of Materials (SBOMs), automated dependency scanning tools, and regular patching cycles for all components, regardless of origin.
• Invest in Expertise and Tools: SMEs, in particular, need support and resources to overcome their lack of security expertise and testing facilities. This could involve leveraging external security consultants, open-source security tools, or participating in industry-sponsored training programs.

1. For Certification Labs:

• Increase Rigor and Transparency: To regain industry confidence, certification labs must demonstrate increased rigor in their testing methodologies. This involves comprehensive vulnerability assessments, penetration testing, and adhering to the highest standards, not just the minimum required for certification. Transparency about their testing processes and findings, while respecting client confidentiality, can build trust.
• Educate and Consult: Labs have an opportunity to educate their clients, especially SMEs, about the true value of robust security and the specifics of IoT security standards. They can move beyond being mere auditors to becoming trusted security advisors.

1. For Consumers:

• Beware of False Security: Consumers should be aware that a "certified" label does not automatically guarantee absolute security. While certifications aim to improve security, the current system has limitations. Consumers should still practice good security hygiene (e.g., strong passwords, keeping devices updated) and demand transparency from manufacturers regarding security practices.

In summary, the defensive implications highlight a systemic need for improved collaboration, clearer guidelines, and stronger accountability across the entire IoT product lifecycle. The goal is to shift from a reactive, compliance-driven approach to a proactive, security-first mindset, ensuring that the next generation of IoT devices is built on a foundation of genuine trustworthiness.

Key Takeaways

• Budget and Time are Critical Enablers/Inhibitors: Adequate budget and time must be allocated for security from the very beginning of the IoT product development lifecycle, including for dedicated security experts and certification processes, to ensure secure products.
• SMEs Face Unique Challenges: Small and Medium-sized Enterprises (SMEs) are particularly vulnerable due to limited budgets, lack of security expertise, and competitive market pressures, often leading to "forum shopping" for superficial certifications.
• Industry Lacks Confidence in Current Certification: Many practitioners are skeptical about the effectiveness of existing certification processes in genuinely improving product security, leading to hesitation in adoption unless mandatory.
• Strong Support for IoT-Specific Standards: Despite current awareness gaps, there is a clear demand from practitioners for dedicated IoT-specific security standards and certifications, believing they can provide better assurance for both vendors and consumers.
• Liability Requires Clarification: A clear legal framework is urgently needed to assign liability for vulnerabilities in certified products, especially concerning third-party libraries, as practitioners largely believe certification labs should bear primary responsibility.
• Education and Accessibility are Key: To improve adoption and perception of IoT security compliance, developers need better education about standards, accessible tools, and clear information explanations.

About the Speaker(s)

Prianka Mandal is a PhD student at William & Mary, where her research focuses on critical aspects of cybersecurity, particularly within the Internet of Things domain. Her work, as highlighted in this talk, investigates the practical challenges and perceptions of industry practitioners regarding IoT product security compliance and certification. She is advised by Professor Adwait Nadkarni.

Adwait Nadkarni is a Professor at William & Mary. He advises Prianka Mandal's PhD research. His work often involves exploring complex security challenges, contributing to a deeper understanding of software and system security.