Human Dignity in AI and Tech Policy

Human Dignity in AI and Tech Policy

Speakers: Jan Trzaskowski, Law Professor, Copenhagen Business School and Aalborg University

Conference: DEF CON 32

YouTube: https://www.youtube.com/watch?v=BIjgd2mOEks

Overview

In this thought-provoking DEF CON 32 talk, Jan Trzaskowski, a distinguished law professor from Copenhagen Business School and Aalborg University, delves into the profound ethical and societal implications of artificial intelligence and data-driven business models. The presentation, based on his book "Human Dignity in AI and Tech Policy," challenges the prevailing market-centric view of technological development, arguing that the current trajectory of AI and data exploitation fundamentally undermines human dignity and individual empowerment. Trzaskowski posits that what we are witnessing with AI is not entirely new but rather an exacerbation of manipulative practices honed over a decade of data-driven business models.

Trzaskowski emphasizes that the core problem lies not in the technology itself, but in the incentives driving its application. He argues that the relentless pursuit of profit, user attention, and data acquisition by "big tech" companies leads to sophisticated forms of manipulation that erode our agency and freedom. This talk is crucial for anyone in the security and technology fields, as it provides a critical legal and philosophical lens through which to understand the often-overlooked human cost of unchecked technological advancement, urging a re-evaluation of our priorities from mere economic gain to the preservation of human values.

Background

▶ Watch: Introduction, topic overview, and book reference (0:00)

The evolution of technology law, as outlined by Trzaskowski, reveals a cyclical pattern. In the mid-1990s, the digitalization of media like music and film brought copyright to the forefront of legal discourse. This was followed by the rise of social media, which digitalized the "human experience," shifting the legal focus to privacy and personal data. Curiously, with the latest surge in artificial intelligence (AI), legal concerns have reverted to copyright and privacy, highlighting persistent challenges in adapting legal frameworks to rapid technological change.

Trzaskowski draws a stark parallel to the warnings issued by Norbert Wiener over 75 years ago. In 1948, the same year the UN Declaration of Human Rights articulated the concept of human dignity, Wiener described "thinking machines" as akin to the atomic bomb. Wiener’s prescient insights suggested two crucial points: technology, once created, will inevitably be used, and the ultimate solution to its challenges lies not in market economics but in prioritizing humanity. This historical context sets the stage for Trzaskowski's central argument: data and AI lack intrinsic value; their true impact is determined by how they are used and the business models they serve. He contends that the past decade of data-driven business models has served as a "dress rehearsal" for manipulation, a practice now amplified by new AI technologies.

The speaker identifies three primary incentives driving personalized marketing and, by extension, many data-driven AI applications:

1. Maximizing Users: Businesses aim for network effects, striving to become the dominant "winner-takes-all" platform.
2. Maximizing Attention: Platforms are designed to be addictive, capturing as much user attention as possible to increase advertising revenue.
3. Maximizing Data: The more data collected on individuals, the more precisely advertisements can be targeted, leading to higher ad prices.

The advent of generative AI introduces additional complexities. Trzaskowski highlights two critical, often overlooked, issues:

1. Content Scarcity: Generative AI models are rapidly consuming available online content for training, leading to a potential "Babel" where new, original content becomes scarce. This also raises ethical questions about the uncompensated use of user-generated content for commercial AI training, which Trzaskowski refers to as "big tech arrogance."
2. Energy Consumption: The immense computational demands of training and operating large AI models place an unprecedented strain on the global power grid. This raises fundamental questions about resource allocation: should energy be distributed based purely on market forces (i.e., those who can pay the most, potentially concentrated in wealthy regions like the "West Coast"), or should there be a broader societal discussion about its equitable use and the externalities involved?

A core premise of liberal democracies and market theory is empowerment – the idea that citizens and consumers can make rational choices aligned with their goals, values, and preferences. Trzaskowski fundamentally challenges this premise, arguing that contemporary technological landscapes actively undermine it. He points out that while regulators often resort to providing more information as a solution, this approach is flawed. Information and transparency are not synonymous; merely providing more data, especially in complex and overwhelming digital environments, often leads to cognitive overload and shifts the burden of decoding onto the already "bounded rational" individual, rather than genuinely empowering them.

Key Findings

▶ Watch: Data's lack of intrinsic value and business incentives (2:26)

Trzaskowski's research culminates in several key findings that reframe our understanding of technology's impact on human behavior and autonomy:

Firstly, he introduces an extended model of information asymmetry, building upon George Akerlof's seminal 1970 work. Akerlof's original "Tier One" information asymmetry describes situations where a seller knows more about a product or themselves than the buyer (e.g., selling a used car). Traditional legal remedies address this by imposing a duty to inform. However, Trzaskowski identifies a "Tier Two" information asymmetry, which he argues is far more insidious in the digital age. This tier describes the vast knowledge businesses possess about how to persuade people in general. This isn't just about product knowledge, but about deep psychological insights into human decision-making, leveraged through technology.

This "Tier Two" asymmetry is where the work of behavioral scientists like BJ Fogg (whose model focuses on motivation, ability, and prompts) and Robert Cialdini (known for his principles of persuasion like reciprocity, commitment, social proof, authority, liking, and scarcity) becomes critical. Trzaskowski asserts that technology allows these influence techniques to be applied "on steroids," creating online influence mechanisms that are extraordinarily powerful. The speaker even draws a parallel to the foundations of magic, noting that both magic and online manipulation rely on misdirection and creating the impression of having a choice where none truly exists.

Secondly, Trzaskowski delivers a sharp critique of the regulatory reliance on providing more information as a means to achieve transparency and empower consumers. He argues that regulators favor this approach because it is "almost free to do." However, this strategy places an undue burden on individuals, who are already experiencing cognitive overload in the internet age. Simply providing more terms and conditions or privacy policies does not equate to genuine transparency; it merely shifts the responsibility of decoding complex information to users, who are inherently "bounded rational." This distinction—that information is not transparency—is a cornerstone of his argument, emphasizing that true transparency would necessitate a fundamental shift in how information is presented and how markets operate.

Thirdly, Trzaskowski articulates the three essential conditions for meaningful empowerment in a digital society:

1. Agency: Individuals must possess the capability to understand the world they inhabit and act effectively within it. This is directly challenged by the psychological concept of bounded rationality, which acknowledges that human decision-making is limited by cognitive constraints.
2. Transparency: Beyond mere information provision, there must be genuine clarity about how systems operate, how data is used, and how influence is exerted.
3. Absence of Manipulation: True freedom and empowerment are antithetical to manipulation. If individuals are being subtly or overtly manipulated, their choices are not truly their own.

Finally, the talk highlights the pervasive role of storytelling and frames in shaping our perception of reality. Trzaskowski notes that humans inherently understand the world through narratives. Businesses have become exceptionally adept at crafting and disseminating stories that influence our beliefs, preferences, and behaviors, often outcompeting individuals' own capacity for critical narrative construction. This mastery of storytelling, combined with the power of algorithms and personalized delivery, becomes a potent tool for manipulation.

Technical Deep Dive

▶ Watch: Generative AI's content depletion and energy demands (4:00)

While Trzaskowski's talk is rooted in legal and philosophical frameworks rather than traditional computer science, it offers a "technical deep dive" into the mechanisms of influence and data utilization that underpin modern digital platforms. This "technical" aspect refers to the systematic and algorithmic application of behavioral psychology to human interaction, rather than specific code or network protocols.

At its core, the technical deep dive revolves around the digital amplification of psychological principles. The speaker highlights how the work of BJ Fogg and Robert Cialdini is implemented "on steroids" through technology. Fogg's model, which posits that behavior occurs when motivation, ability, and a prompt converge, can be algorithmically optimized. For example, a social media platform's notification system (the "prompt") is designed to arrive when a user's motivation (e.g., boredom, desire for social connection) and ability (e.g., holding a smartphone) are high, triggering a desired action like opening the app. Similarly, Cialdini's principles of persuasion – such as social proof (showing how many others like a product), scarcity (limited-time offers), or reciprocity (giving free content to encourage engagement) – are integrated into user interface (UI) and user experience (UX) design, content recommendations, and targeted advertising. These aren't just design choices; they are engineered psychological interventions.

The personalized marketing algorithms are the technical engines driving these influence campaigns. These algorithms ingest vast quantities of individual user data – browsing history, click patterns, dwell times, location data, social connections, and even biometric information – to construct highly detailed psychological profiles. With these profiles, platforms can:

• Optimize for User Acquisition: A/B testing different onboarding flows, referral programs, and ad creatives to maximize the network effects that lead to market dominance.
• Optimize for Attention: Employing recommender systems that surface content most likely to trigger engagement, infinite scroll designs, variable reward schedules (like notifications for likes or comments), and even dark patterns to maximize addictiveness and screen time. These systems are constantly learning and adapting to individual user responses.
• Optimize for Data Acquisition: Designing interactions that subtly encourage users to provide more personal information, often framed as improving user experience or personalization. The more data points an algorithm has, the more accurately it can predict behavior and tailor influence tactics, leading to higher valuations for advertising space.

The rise of generative AI introduces new technical challenges and ethical considerations. The training of large language models (LLMs) and diffusion models requires colossal datasets. Trzaskowski points out the technical reality that these models are "running out of content to train their models on." This technical limitation drives companies to scrape vast swathes of the internet, including user-generated content, raising significant copyright and ethical concerns about consent and compensation. The sheer scale of these models also translates into staggering energy consumption. The computational infrastructure – data centers, GPUs, cooling systems – demands immense electrical power. Trzaskowski highlights the technical and societal question of whether existing power grids can sustain this demand and how the allocation of such a fundamental resource should be determined, moving beyond purely market-driven decisions. This is a technical externality with profound societal implications, requiring a deep understanding of infrastructure capabilities and energy policy.

Ultimately, the "technical deep dive" in this context reveals how seemingly innocuous design elements and algorithms are, in fact, sophisticated tools for behavioral manipulation. They exploit human bounded rationality and cognitive biases, effectively creating an environment where individual agency is subtly, yet systematically, undermined. The "tech" here is not just about building tools, but about building systems that hack human behavior at scale, turning psychological vulnerabilities into profit centers.

Demo / Proof of Concept

▶ Watch: Online influence leveraging magic's misdirection (6:01)

Jan Trzaskowski's talk is an academic and policy-oriented discussion, focusing on legal frameworks, ethical considerations, and societal implications of AI and data-driven business models. As such, the presentation did not include a live technical demonstration or a proof of concept in the traditional sense of showcasing code, exploits, or system vulnerabilities. Instead, the "proof" for his arguments is drawn from established psychological research, legal theory, and observed market behaviors of major technology companies.

Defensive Implications

▶ Watch: Model explaining three tiers of information asymmetry (6:33)

The insights presented by Jan Trzaskowski offer critical defensive implications for individuals, policymakers, and even technologists, urging a shift in perspective from passive acceptance to active engagement with the ethical challenges posed by AI and data-driven manipulation.

For Individuals:

• Cultivate Self-Awareness of Bounded Rationality: Recognize that human decision-making is inherently limited and susceptible to biases. Understanding that platforms are designed to exploit these cognitive shortcuts (e.g., through dark patterns, infinite scroll, or variable reward schedules) is the first step toward resisting manipulation.
• Distinguish Information from Transparency: Be critical of the sheer volume of information provided (e.g., privacy policies, terms of service). Understand that more information does not automatically lead to more transparency or empowerment. Instead, question what is not being made transparent and demand clarity.
• Critically Evaluate Online Narratives and Choices: Be aware that businesses are masters of storytelling and framing. Question the narratives presented by platforms and advertisers. Similarly, be skeptical of "choices" offered, as they might be a form of misdirection designed to give the impression of control rather than genuine agency.
• Manage Cognitive Load: Consciously limit exposure to overwhelming digital environments to combat cognitive overload. Practice digital hygiene and seek out diverse, non-algorithmically curated sources of information.

For Regulators and Policymakers:

• Shift Regulatory Focus from Information to Manipulation: Move beyond mere information-disclosure regimes, which place the burden on consumers. Implement regulations that directly address and prohibit manipulative design practices and business models that undermine agency. This could involve mandating human-centric design, requiring ethical impact assessments, or even banning certain persuasive techniques.
• Regulate Data-Driven Business Models, Not Just Technology: Recognize that the problem lies in the incentives of these models (maximizing users, attention, and data) rather than the AI technology itself. Policy interventions should target these underlying economic drivers, potentially exploring alternative business models or imposing stricter limits on data collection and use.
• Prioritize Human Dignity as a Foundational Principle: Elevate human dignity and empowerment above purely market-driven considerations in tech policy. This means moving beyond a sole focus on economic efficiency to consider the broader societal and individual well-being impacts of technology.
• Address Externalities of Generative AI: Proactively engage with the environmental impact (energy consumption) and content ownership issues (training data scarcity, copyright) posed by generative AI. This requires global cooperation and potentially new legal frameworks for resource allocation and intellectual property in the age of AI.

For Technologists and Developers:

• Embrace Ethical Design Principles: Move beyond "engagement at all costs" metrics. Design products and services that genuinely empower users, prioritize user agency, and foster transparency. This includes avoiding dark patterns and implementing clear, comprehensible interfaces.
• Question Business Incentives: Reflect on the ethical implications of the business models they are building and supporting. Advocate for and explore alternative models that do not rely on constant data extraction, attention maximization, or behavioral manipulation.
• Advocate for Responsible AI Development: Consider the broader societal impact of AI systems, particularly in terms of data sourcing, energy consumption, and potential for misuse. Push for industry standards and best practices that prioritize ethical AI development and deployment.
• Foster True Transparency: Work towards designing systems that genuinely communicate their operations and data handling practices in an understandable way, going beyond legalistic disclosures to achieve true user comprehension.

Key Takeaways

• AI and Data-Driven Business Models are a "Dress Rehearsal" for Manipulation: Modern AI exacerbates long-standing techniques of influence and control refined over a decade of data-driven market strategies.
• Empowerment Requires Agency, Transparency, and Absence of Manipulation: True individual freedom in the digital age hinges on the ability to understand and act, genuine clarity about system operations, and freedom from coercive or deceptive influence.
• Information is Not Transparency; More Data Can Lead to Cognitive Overload: Regulatory reliance on providing more information often burdens individuals, who are already overwhelmed, rather than genuinely empowering them.
• Businesses Leverage Psychological Principles and Design to Hack Behavior: Techniques from behavioral psychology (e.g., Cialdini's persuasion principles, Fogg's behavior model) are digitally amplified through sophisticated design and algorithms to exploit human bounded rationality.
• Human Dignity, Not Just Market Economics, Must Guide Tech Policy: A fundamental shift is needed to prioritize human values and well-being over the relentless pursuit of profit, user acquisition, and attention maximization.
• Generative AI Introduces New Ethical and Resource Challenges: The immense demand for training content and energy consumption by generative AI models raises critical questions about content ownership, environmental impact, and equitable resource allocation.

About the Speaker(s)

Jan Trzaskowski is a distinguished Law Professor affiliated with both Copenhagen Business School and Aalborg University. His academic and research focus lies at the critical intersection of law, technology, and human behavior. Trzaskowski's work extensively explores the ethical and regulatory challenges posed by data-driven business models and artificial intelligence, particularly concerning their impact on individual autonomy and human dignity. He is the author of the book "Human Dignity in AI and Tech Policy," which serves as the foundation for his insights presented in talks like this one, advocating for a human-centric approach to technology governance.