Fueling The Future: Building Robust Engines - Daniel Cuthbert (Keynote)

Fueling The Future: Building Robust Engines

Speakers: Daniel Cuthbert, Keynote Speaker

Conference: Nullcon

YouTube: https://www.youtube.com/watch?v=_gal2jNmXbs

Overview

In his compelling Nullcon keynote, "Fueling The Future: Building Robust Engines," Daniel Cuthbert challenges the cybersecurity industry to fundamentally rethink its approach, advocating for a shift from a reactive, bug-centric mindset to a proactive safety engineering paradigm. Cuthbert, a veteran security researcher and co-founder of the OWASP Testing Project, argues that the tech sector has largely ignored critical lessons from centuries of industrial development regarding safety and robustness, leading to a pervasive culture of "safety third." He contends that this oversight is increasingly perilous given humanity's growing reliance on interconnected digital systems and the dawn of the AI era.

Cuthbert's talk is a critical examination of why, despite decades of effort, the industry continues to grapple with persistent, fundamental vulnerabilities that have real-world, often devastating, consequences. He draws parallels between the early, unregulated industrial revolutions—marked by horrific accidents and loss of life—and the current state of software development, where critical bugs are routinely shipped to production. By juxtaposing the stringent safety standards of industries like manufacturing and structural engineering with the often-lax practices in software, Cuthbert highlights a profound disconnect. This keynote serves as an urgent call to action, urging security professionals and developers to embrace a safety-first philosophy to build a truly defendable and resilient digital future.

Background

▶ Watch: From offensive cyber to safety engineering (0:00)

The concept of safety, as Cuthbert highlights, is not a modern invention but a historical imperative, tracing its linguistic roots back to the 15th century. Across various industries, stringent safety standards have evolved over centuries, often in response to catastrophic failures. For instance, designing and manufacturing children's toys today requires adherence to specific standards like EN 71.1 in the UK, ensuring they do not harm users. Similarly, the design and construction of lifts, despite their inherent danger, involve incredibly intense processes, from detailed specifications and material regulations (e.g., specific steel types, welding standards) to rigorous, multi-stage testing. This meticulous approach has resulted in remarkably low accident rates, with only 27 lift-related deaths in the US last year, 14 of which were workers not following safety protocols, translating to an infinitesimal 0.0015% chance of death per ride.

Cuthbert contrasts this with the tech industry's "ship to production and then worry about security" mentality. He traces this historical negligence through the lens of industrial revolutions:

• First Industrial Revolution (Steam & Textiles): Marked by the shift from agriculture to machines, it brought concerns about control and the impact of new, powerful technologies, as depicted in Philip Jacques de Loutherbourg's 1801 painting, showing furnaces replacing traditional farming.
• Second Industrial Revolution (Electricity & Mass Production): This period saw the rise of oil, gas, and the internal combustion engine, amplifying human capabilities but also leading to horrific industrial accidents. Jean Eugène Buland's 1888 painting, seemingly depicting an apprentice and master, subtly references an era of rampant child death in factories due to unguarded machinery. The tragic Triangle Shirtwaist Factory fire in New York in 1911, which killed 140 people due to inadequate safety controls (e.g., open furnaces, locked exits), finally prompted America to implement stronger industrial regulations.
• Third Industrial Revolution (Computers & Automation): The era of transistors, computers, and the birth of the internet.
• Fourth Industrial Revolution (Cyber-physical, IoT): From 2010 to the present, characterized by the convergence of digital and physical systems.
• Fifth Industrial Revolution (Artificial Intelligence): The current, emerging era that Cuthbert views with apprehension due to the potential for new, unaddressed risks.

Historical incidents outside of tech further underscore the industry's failure to learn. The Chernobyl disaster in 1986, where reactor issues were initially kept secret, led to widespread radiation exposure and delayed evacuations. Similarly, the Bhopal chemical incident in 1984 was a massive industrial disaster attributed to a lack of security controls. These events highlight a pattern in other industries: catastrophic failures lead to systemic changes and improved safety protocols. In contrast, Cuthbert argues, the cyber security sector often fails to look at its own history or build things in a secure, safe way, instead glorifying offensive operations and giving "cute little names" to threat actors who cause damage. This "transformation ambivalence" leaves the industry at a crossroads, where the fundamental "bug journey" has evolved from relatively harmless early web vulnerabilities to those capable of causing immense financial and societal harm.

Key Findings

▶ Watch: Tracing safety through industrial revolutions (2:40)

Cuthbert's analysis reveals several critical findings about the current state of cybersecurity and software development:

Firstly, fundamental vulnerabilities persist despite decades of awareness and effort. Despite the maturity of the internet and the cybersecurity industry, basic bug classes like SQL injection (CWE-89), cross-site scripting (XSS) (CWE-79), and path traversal (CWE-22) remain rampant. He cites a recent example of a major VPN provider found with a path traversal vulnerability within the last four weeks, underscoring that even critical infrastructure is not immune. This persistence, 25-26 years after Rainforest Puppy (Jeff Forristal) first popularized SQL injection in 1998, indicates a systemic failure to address root causes, often masked by "crutches" like Web Application Firewalls (WAFs) rather than building inherently secure systems.

Secondly, web application vulnerabilities are on the rise again. While average CVSS scores for web vulnerabilities showed a slight decrease between 2002 and 2024, the number of reported vulnerabilities increased in 2023, and early 2025 data from NVD (National Vulnerability Database) shows a "massive spike" in web application vulnerabilities. This resurgence is concerning, especially as the internet's attack surface continues to grow. Cuthbert notes that while some less common web app vulnerabilities like XML External Entity (XXE) injection (CWE-611) have decreased, the most dangerous and common ones remain stubbornly high.

Thirdly, the financial incentives around bugs are perverse, fueling a global surveillance industry. Vulnerabilities are no longer minor nuisances; they are worth "life-changing money." Cuthbert references current Zerodium price lists, showing desktop bugs valued up to $500,000, server bugs up to $500,000, and virtualization software bugs up to $1,000,000. Full zero-click payloads for mobile devices can fetch "many millions." This lucrative market incentivizes the discovery and sale of exploits, often to state-sponsored actors, rather than their remediation. As Kenneth Geers aptly put it, we are in a "golden age of Espionage," where technology has enabled anyone to spy on anyone, and bugs are the primary currency.

Fourthly, there is a significant imbalance between offensive and defensive security research. Cuthbert, who helps run Black Hat, observes that while thousands of submissions often feature offensive exploits like "popping calc.exe," defensive talks are "really, really slim on the ground." This imbalance suggests that the industry collectively prioritizes finding and demonstrating vulnerabilities over developing robust, proactive defenses. He echoes Halvar Flake's 2016 assertion that the industry is not building a defendable internet, a conclusion Cuthbert initially tried to disprove but ultimately found to be true.

Finally, human error remains a primary attack vector, exacerbated by insecure systems. Phishing continues to be the number one vector, as evidenced by the recent North Korean hack that stole $1.4 billion from a crypto exchange, likely through a phishing campaign. This highlights that simply blaming users for clicking links is insufficient; systems must be designed to be resilient against human fallibility, a standard that is currently not met. The talk argues that the collective failure to build secure systems proactively leads to these recurring issues, costing billions and eroding trust.

Technical Deep Dive

▶ Watch: Early industrial revolution's impact on people (4:40)

Cuthbert delves into several key technical historical points and modern vulnerability trends to substantiate his arguments. The very term "bug" originates from a physical incident: in the 1940s, the ENIAC, one of the first electronic general-purpose computers, utilized light bulbs to represent ones and zeros. A moth, attracted by the heat, caused a short circuit in relay number 70 in panel F, leading to the coining of the term "bug" for a system malfunction. This early example highlights the unexpected interactions between physical and computational systems.

The evolution of computing from these early light-bulb-based systems to modern transistors and powerful chips like Apple's M2 Ultra has created immense global capabilities. However, this growth has also introduced geopolitical concerns regarding access to and control over these semiconductor technologies and the data they process, all of which are intrinsically linked to the presence and exploitation of bugs.

One of the most striking technical examples of a software bug's catastrophic impact is the Arianne 5 rocket failure in 1996. During its maiden flight, a software error caused the rocket to veer 90 degrees off course shortly after liftoff and self-destruct. The root cause was a seemingly innocuous conversion: a 64-bit integer representing horizontal velocity was implicitly converted to a 16-bit integer. As the rocket's velocity increased, the 64-bit value exceeded the maximum capacity of the 16-bit variable, leading to an integer overflow (specifically, a buffer overflow in the context of data representation). This unhandled exception effectively crashed the guidance system, resulting in a $370 million loss. This incident serves as a stark reminder that seemingly minor programming choices can have devastating real-world consequences.

Another compelling example of a critical bug with real-world impact is a trucker safety program Cuthbert cited. Designed to allow drivers under duress (e.g., during a hijacking) to send an emergency alert, the system had a queue with a "very small buffer." When the buffer filled, instead of sending the critical alert, it displayed "SQ is nearly full, will retry soon," rendering the system useless in a life-threatening situation. This illustrates how even simple buffer management issues can have profound safety implications.

On the web application front, the talk revisits the enduring problem of SQL Injection. First publicly detailed by Rainforest Puppy (Jeff Forristal) in 1998, this vulnerability allows attackers to manipulate database queries through user input. Despite being over 25 years old, SQL injection remains a "massive problem," highlighting a persistent failure to implement secure coding practices and input validation. This long-standing issue underscores the industry's inability to eradicate fundamental flaws.

Cuthbert also touches on the origins of structured web security efforts, noting the founding of OWASP (Open Web Application Security Project) in 2001, shortly after Bill Gates' Trustworthy Computing memo. He recounts co-authoring the OWASP Testing Project guide in 2002, one of the first comprehensive guides for web application hacking and security testing. This initiative aimed to change the status quo of web vulnerabilities, yet, as Cuthbert notes, OWASP's continued relevance 24 years later indicates that the "bugs" are far from fixed.

To quantify the current state, Cuthbert presents an analysis of vulnerability data from the NVD (National Vulnerability Database):

• The average CVSS score for web vulnerabilities has slightly decreased from 2002 to 2024, which is a positive trend.
• However, the number of vulnerabilities per year saw an increase in 2023, and early 2025 data shows a "massive spike" in reported web application vulnerabilities, suggesting a resurgence.
• The prevalence of better tools, such as the AFL++ fuzzer (American Fuzzy Lop Plus Plus), which now includes a quick start guide, contributes to easier bug discovery.
• Analyzing CISA's Known Exploitable Vulnerabilities (KEV) dataset, Cuthbert points out the most dangerous CWEs (Common Weakness Enumerations) currently exploited in the wild:
• CWE-787 (Out-of-bounds Write): This memory-related vulnerability is the most common.
• CWE-94 (Code Injection): Represents traditional injection flaws.
• Specific web application CWEs that remain persistent problems include CWE-79 (Cross-site Scripting), CWE-89 (SQL Injection), and CWE-22 (Path Traversal). Conversely, CWE-611 (XML External Entity Injection) has seen a positive decline.
• The MITRE Most Dangerous Software Weaknesses list further corroborates that many bugs, like SQL injection and use-after-free, continue to exist despite well-known mitigations.

This detailed technical review underscores that the industry is not only failing to learn from historical mistakes but is also struggling to address fundamental vulnerabilities that have been understood for decades, leading to a continuously insecure digital landscape.

Demo / Proof of Concept

▶ Watch: Factory fire: Catalyst for modern safety laws (6:00)

The keynote "Fueling The Future: Building Robust Engines" by Daniel Cuthbert is a high-level, analytical presentation focused on historical context, industry trends, and philosophical shifts in cybersecurity. As such, it did not include a live demonstration or a specific proof of concept of a vulnerability or exploit. The speaker relied on historical examples, data analysis, and anecdotes to illustrate his points about the persistence of bugs and the need for a safety engineering approach.

Defensive Implications

▶ Watch: Chernobyl: The human cost of delayed information (6:40)

Cuthbert's keynote provides a clear mandate for defenders: a radical shift from reactive patching to proactive safety engineering. This involves fundamentally rethinking how software is designed, built, and deployed, moving away from the "ship it and fix it later" mentality.

Firstly, embrace stringent standards and regulations. Just as other industries have developed robust safety standards (e.g., EN 71.1 for toys, rigorous building codes for lifts), the software industry needs to establish and adhere to similar, non-negotiable security and safety standards. This means defining specifications for secure design, coding practices, and deployment, ensuring that security is an inherent quality, not an add-on.

Secondly, prioritize rigorous, upfront testing. The analogy of lift manufacturing, where "stringent, stringent testing" occurs before any human enters the system, is paramount. Software development must adopt a similar approach, integrating comprehensive testing—including fuzzing (like with AFL++), static analysis, dynamic analysis, and penetration testing—early and continuously throughout the development lifecycle, rather than as a post-deployment afterthought.

Thirdly, address root causes instead of relying on "crutches." The persistence of vulnerabilities like SQL injection, even with the prevalence of Web Application Firewalls (WAFs), highlights a failure to fix fundamental architectural and coding flaws. Defenders must advocate for and implement secure coding practices, robust input validation, and secure design principles that eliminate entire classes of bugs at their source, rather than deploying perimeter defenses that merely mitigate symptoms.

Fourthly, invest significantly more in defensive research and development. Cuthbert's observation that defensive talks are "really, really slim on the ground" at major conferences points to a systemic imbalance. The industry needs to foster an environment that rewards and promotes research into defensive architectures, secure development methodologies, threat modeling, and proactive security controls. This includes funding, academic programs, and industry initiatives that elevate the profile and impact of defensive innovation.

Fifthly, learn from history, both within and outside of tech. Defenders must actively study past security incidents (like the Arianne 5 rocket failure or the North Korean crypto heist) and industrial disasters (Chernobyl, Bhopal, Triangle Shirtwaist Factory fire) to understand their root causes and implement lessons learned. This historical perspective can inform better risk assessment, incident response planning, and preventive measures.

Finally, empower developers with the knowledge and tools to build securely. Cuthbert emphasizes that "you have the power to help eradicate a vast amount of bugs." This means providing developers with continuous security training, secure coding guidelines, access to automated security tools, and fostering a culture where security is a shared responsibility, not solely relegated to a security team. By focusing on critical CWEs like CWE-787 (Out-of-bounds Write) and CWE-94 (Code Injection), and ensuring their eradication, the industry can make significant strides towards building truly robust and defendable systems.

Key Takeaways

• The cybersecurity industry must adopt a safety engineering mindset, drawing lessons from other industrial revolutions where safety standards were developed through centuries of learning from catastrophic failures.
• Fundamental vulnerabilities like SQL Injection (CWE-89), Cross-Site Scripting (CWE-79), and Path Traversal (CWE-22) persist for decades, indicating a systemic failure to address root causes rather than merely mitigating symptoms.
• The lucrative bug bounty market, with zero-day exploits fetching millions, inadvertently fuels a global surveillance industry by rewarding the discovery and sale of vulnerabilities rather than prioritizing their swift and permanent eradication.
• There is a critical imbalance between offensive and defensive security research, with defensive strategies and innovations being underrepresented and under-resourced at major industry conferences and within organizations.
• Major incidents, from the Arianne 5 rocket failure due to a 16-bit integer overflow to recent multi-billion dollar crypto heists facilitated by phishing, underscore the severe real-world consequences of software bugs and the urgent need for a more robust approach.
• The collective power of security professionals and developers, when focused on proactive security, stringent testing, and learning from history, is capable of eradicating many common vulnerabilities and building a genuinely secure and defendable internet.

About the Speaker(s)

Daniel Cuthbert is a seasoned keynote speaker and a highly respected veteran in the cybersecurity industry, boasting over 28 years of experience. Initially starting his career in art and art history, and later venturing into fashion design, fashion photography, and documentary photography (including documenting the Chernobyl anniversary), Cuthbert's diverse background informs his holistic perspective on safety and risk.

He describes himself as having evolved from a security researcher primarily focused on offensive campaigns and bug exploitation to a safety engineer, emphasizing the importance of building robust and secure systems from the ground up. Cuthbert was a co-founder of hack.co.za in 1997-99, one of the earliest exploit sharing sites, and confesses to writing exploits in Perl in his early days. A pivotal figure in web application security, he co-authored the OWASP Testing Project guide in 2002, a foundational resource for web application hacking. Today, he is actively involved in running conferences like Black Hat, where he observes and champions the discussions around building a more defendable internet, advocating passionately for a safety-first approach in software development.