51 Ways to Spell the Image Giraffe: The Hidden Politics of Token Languages in Generative AI

51 Ways to Spell the Image Giraffe: The Hidden Politics of Token Languages in Generative AI

Speakers: Ting-Chun Liu, Media Artist, Bahuni Enva; Leon-Etienne Kühr, Media Artist, AI Lab Offenbach

Conference: 39C3

YouTube: https://www.youtube.com/watch?v=PTlcyYsi-Es

Overview

This talk, "51 Ways to Spell the Image Giraffe," delves into the often-overlooked yet foundational process of tokenization in generative AI, particularly its profound impact on text-to-image models. Presented by media artists Ting-Chun Liu and Leon-Etienne Kühr, the research explores how the seemingly neutral conversion of human language into computable units embeds significant political, cultural, and economic biases, shaping what AI can generate and how it understands the world. The speakers highlight that tokenization, a process preceding neural network involvement, is a critical bottleneck that dictates the accessibility, cost, and even the semantic interpretation of prompts, revealing a hidden "token language" distinct from natural language.

The core of their inquiry lies in dissecting the token vocabulary and its underlying ideologies. Through artistic research and reverse engineering, Liu and Kühr demonstrate how specific word choices, language origins, and even social media trends are encoded into the fabric of AI models like Stable Diffusion, leading to unintended consequences such as linguistic discrimination, brand prominence, and the creation of "unreachable images" through conventional prompting. Their work serves as a crucial reminder that the technical choices made at the earliest stages of AI development have far-reaching implications, influencing not just the aesthetics of generated content but also the broader societal impact of these powerful technologies.

Background

▶ Watch: Introduction and talk overview: '51 Ways to Spell Giraffe' (0:00)

The journey into understanding generative AI often begins with the prompt, the seemingly simple text input that guides image generation. However, as Liu and Kühr reveal, a complex and largely opaque process known as tokenization precedes any neural network activity. This process converts natural language into discrete, computable units called tokens, forming the fundamental vocabulary that AI models understand. Unlike the linguistic concept of morphemes—the smallest meaningful units of language—tokens are primarily designed for computational efficiency, often splitting words in ways that lack human semantic meaning (e.g., "giraf" and "fe"). This efficiency-driven design, while practical, introduces a layer of abstraction that profoundly influences AI behavior.

The dominant algorithm for tokenization in many large language and image models is Byte Pair Encoding (BPE), first proposed in 2015. BPE works by iteratively identifying the most frequent character combinations in a vast training corpus and adding them as new tokens to a dictionary. This process creates a vocabulary where common words or phrases become single tokens, while less frequent or complex words might be broken down into multiple, smaller units. The specific token dictionary examined in this talk is from CLIP (Contrastive Language-Image Pre-training), an OpenAI model published in 2021, which boasts around 49,470 tokens. A critical point of concern, as highlighted by the speakers, is the lack of transparency surrounding CLIP's training data, described only as "400 million image-text pairs collected from the internet," with no details on how the tokenizer itself was trained. This opacity sets the stage for the embedded biases that the artists uncover.

Key Findings

▶ Watch: Unexpected AI output from 'www big bald head' prompt (3:40)

Liu and Kühr's investigation into tokenization yields several crucial findings that expose the hidden mechanics and inherent biases within generative AI:

1. Linguistic Discrimination and Cost Imbalance: The BPE algorithm, by prioritizing frequency, heavily favors English and other Latin-alphabet-based languages. Words in non-Latin languages, such as German or Traditional Chinese, often require significantly more tokens to represent the same meaning. For instance, the German word "Schutzbeutel" (fire safety officer) translates to nine tokens, while its English equivalent "fire safety officer" is just three. Similarly, the Traditional Chinese word for "giraffe" (literally "long-necked deer") becomes nine tokens, whereas "giraffe" in English is a single token. This disparity translates directly into higher computational costs (API usage, GPU processing, energy consumption) and increased latency for prompts in less-represented languages, effectively creating a linguistic hierarchy within AI.

1. Embedded Cultural and Commercial Biases: The token dictionary reflects the dominant cultural and commercial landscapes of its training data. Common social media hashtags (e.g., "Monday motivation," "Wine Wednesday"), brand names, and popular culture references (celebrities, movie characters) frequently exist as single tokens. This gives them disproportionate "power" in image generation, as they are more efficiently processed and potentially more strongly weighted in the model's internal representation. The infamous www big bald head example, which consistently generates images of actor Norman Reedus (whose Twitter handle is @wwwbigbaldhead), vividly illustrates how specific, high-frequency online data points become hardcoded into the token vocabulary.

1. The Existence of a "Token Language": The research reveals that the internal representation of text within AI models, what the speakers term "token language," is distinct from human natural language. Different sequences of tokens can decode to the same natural language word (e.g., "giraffe" vs. "gira" + "ffe"), yet produce vastly different images. This implies that the model interprets the raw token IDs and their embeddings, not just the semantic meaning of the human-readable word. This "token language" can generate images that are "unreachable" through conventional, well-formed natural language prompts, opening up new avenues for artistic expression but also for manipulating AI outputs.

1. Influence of Misspellings and Unconventional Sequences: The models demonstrate an ability to handle misspellings or fragmented token sequences, often still producing recognizable images (e.g., many of the "51 ways to spell giraffe" still yielded giraffes). This suggests that the training data likely included noisy or misspelled text, allowing the model to generalize. However, some unconventional token splits led to entirely unrelated images, highlighting the unpredictable nature of this underlying token language.

1. Lack of Transparency and Controllability: The opaque nature of tokenizer training, combined with the embedded biases, creates a system where the fundamental building blocks of AI understanding are largely uncontrolled and unexamined by end-users or even many developers. This lack of transparency makes it challenging to predict, debug, or mitigate the biases and unintended behaviors that arise from the tokenization process.

Technical Deep Dive

▶ Watch: Abstract process of generative image creation (5:00)

The technical foundation of this research lies in understanding the three interconnected stages of text-to-image generation: tokenization, embedding, and conditioning a diffusion model.

1. Tokenization: The Pre-Neural Network Process

As discussed, tokenization is the initial step, converting raw text into numerical token IDs. This process occurs before any neural network processing. The primary algorithm employed is Byte Pair Encoding (BPE). BPE is a data compression technique adapted for subword tokenization. It begins by treating each character as an individual token. It then iteratively merges the most frequent adjacent pairs of tokens into new, single tokens until a predefined vocabulary size is reached or no more merges can be found. For instance, starting with "a rose is a rose is a rose," BPE might first combine "o" and "s" into "os," then "r" and "o" into "ro," and so on, eventually forming compound tokens like "rose" or "is_a." The resulting token dictionary (or vocabulary) maps these unique tokens to specific numerical token IDs. The CLIP tokenizer, central to this talk, has approximately 49,470 such entries. Crucially, tokens can represent single characters, parts of words, full words, or even sequences of words and emojis, all based on their frequency in the training data. For example, specific emoji combinations like two celebration emojis 🎉🎉 function as a single token, as do long phrases like think big Sunday with Marsha or international women's day.

1. Embeddings: Semantic Representation in High-Dimensional Space

Once text is tokenized into a sequence of token IDs, these IDs are then converted into embeddings. An embedding is a high-dimensional vector (in the case of Stable Diffusion, a 512-dimension vector) that numerically represents the semantic meaning of each token. These vectors are learned during the training of a text encoder, such as the one within CLIP. The key idea behind embeddings is that tokens with similar meanings or contexts should have vectors that are "closer" to each other in this high-dimensional space. While a single token's embedding might appear as a "bunch of random numbers," the relationships (distances) between these vectors are what create meaning. For example, the embedding for "dog" would be closer to "puppy" than to "cat." The speakers illustrate this by projecting these high-dimensional embeddings onto a 2D graph, showing clusters of related tokens. It's important to note that a token like "what" (ID 349) is distinct from "what " (ID 349, with a trailing space/end-of-word marker), each having a unique ID and a distinct embedding, reflecting their different contextual roles.

1. CLIP: Bridging Text and Image

CLIP (Contrastive Language-Image Pre-training) is fundamental to how text-to-image models understand prompts. CLIP consists of two main components: an image encoder and a text encoder. Both encoders are trained to project their respective inputs (images and text) into a shared embedding space. During training, CLIP learns to associate images with their descriptive text captions. When given an image and multiple text descriptions, CLIP can output a similarity score, indicating how well each text description matches the image. The more descriptive the text, the higher the score. This shared embedding space allows text prompts to "condition" the image generation process. The tokenized and embedded prompt acts as guidance for a diffusion model, directing it from an initial state of random noise towards an image that semantically aligns with the prompt's embedding. This conditioning is how text "steers" the image generation, transforming random visual noise into coherent, prompted imagery. The speakers specifically mention that many open-source models, including various Stable Diffusion versions and Flux, utilize the same CLIP tokenizer, even if they employ different text encoders, underscoring its pervasive influence.

Demo / Proof of Concept

▶ Watch: How meaning emerges from token distances and embeddings (8:30)

The core of Liu and Kühr's demonstration involved directly manipulating the tokenization pipeline to expose its hidden language. Their project unfolded in several stages:

1. Direct Token ID Injection: The speakers "hijacked" the image generation process by patching various Stable Diffusion models (including Stable Diffusion 1, XL, 3.5, and Flux) to accept raw token IDs directly, bypassing the natural language tokenizer. This allowed them to input numerical sequences instead of text strings. For example, directly inputting the token ID 22826 generated an image of a giraffe, as this ID corresponds to the token "giraffe". They extended this to multi-token sequences, successfully generating "a giraffe wearing sunglasses" by feeding the corresponding token IDs.

1. Exploring "Orphan" Tokens: They downloaded the complete CLIP token dictionary (approximately 50,000 entries) and filtered out all English words and tokens ending with a word-termination character. This left them with 3,832 "orphan" tokens—sequences that don't form standard English words or complete linguistic units but exist in the dictionary due to BPE's frequency-driven training. They then generated images for all these filtered tokens. This revealed fascinating insights: many orphan tokens corresponded to characters from popular movies, celebrities, sports terms, popular social media hashtags (e.g., wine Wednesday, my Dubai), brands, and events. Others produced seemingly random or abstract images, such as the token "carell," which in some models generated abstract patterns, while in others (like Stable Diffusion XL), it produced images resembling fish, suggesting a connection to the English word "mackerel" that was split into "mac" and "erel" during tokenization.

1. The "51 Ways to Spell Giraffe" Experiment: This was a central demonstration. The speakers identified all possible ways to segment the word "giraffe" into multiple tokens from the CLIP dictionary (e.g., "gira" + "ffe," "gir" + "affe," "gi" + "raff" + "e," etc.). Despite all these token sequences decoding back to the human-readable word "giraffe," when fed directly into the patched models with the same random seed, they produced different images. While many still resulted in giraffes (likely due to the model learning to cope with misspellings in its training data), some combinations generated entirely unrecognizable imagery. This experiment unequivocally showed that the model processes the raw token sequence, not just the semantic meaning of the decoded word, revealing the "token language" as a distinct layer of interpretation.

1. Reverse Engineering Prompts with Genetic Algorithms: To further explore the token language, Liu and Kühr adapted genetic algorithms (a concept from the 1960s) to reverse engineer the "token prompt" for a given image.

• Goal: Find a sequence of 16 tokens that best describes a target image.
• Method:
• They initialized a "population" of 248 random sequences of 16 token IDs (the "genome").
• For each sequence, they calculated a CLIP score (a measure of semantic similarity between the token sequence and the target image). This served as the "fitness function."
• Over 200 iterations, the algorithm applied genetic crossover (combining parts of high-scoring token sequences) and mutation (randomly altering tokens) to evolve new generations of token sequences. The "fittest" sequences (those with higher CLIP scores) were more likely to "survive" and reproduce.
• Results: The algorithm successfully generated seemingly random sequences of tokens (e.g., kati, K, p, L, _) that, when fed into the patched image generators, reliably reproduced the original target image, even for generic or unusual images. This demonstrated that the model's internal representation could be manipulated with non-natural language token sequences to achieve precise visual outcomes. When compared to human-generated or even ChatGPT-generated text descriptions, the genetically engineered token sequences consistently achieved higher CLIP scores, proving their superior alignment with the image in the model's internal representation.

Defensive Implications

▶ Watch: 'Giraffe on a surfboard' example: tokenization vs. meaning (10:00)

The detailed exploration of tokenization by Liu and Kühr uncovers several critical implications for developers, users, and security researchers working with generative AI:

1. Bias Mitigation through Tokenizer Auditing: The most significant defensive implication is the urgent need for transparency and auditing of tokenizer training data and algorithms. The fact that current tokenizers embed cultural, linguistic, and commercial biases (e.g., favoring English, popular brands, specific emojis) means that these biases propagate throughout the entire AI pipeline. Developers should advocate for or implement methods to inspect and potentially fine-tune token vocabularies to ensure equitable representation across languages and cultures, thereby reducing inherent discrimination and cost disparities.

1. Understanding and Addressing Linguistic Inequity: Users and developers working with multilingual applications must be aware of the tokenization cost disparity. Generating content in non-Latin languages can incur significantly higher computational costs and latency. This knowledge can inform architectural decisions, resource allocation, and prompt engineering strategies for diverse user bases. It also highlights the need for AI models to be trained with more balanced datasets that account for linguistic diversity at the tokenization level.

1. Prompt Hiding and Obfuscation: The "51 ways to spell giraffe" experiment revealed that models respond to specific token sequences rather than just the semantic meaning of a word. This opens a new vector for prompt hiding or obfuscation. Adversaries could craft token sequences that decode to an innocuous word (e.g., "giraffe") but generate an entirely different, potentially harmful or censored image. While the speakers noted limitations with current censorship mechanisms that operate at the embedding or post-generation stage, this technique offers a novel way to bypass text-based content filters. Defenders need to consider inspecting the token sequence itself, not just the decoded text, for potential malicious intent.

1. Robustness Against Adversarial Token Sequences: The ability to reverse-engineer images into token sequences using genetic algorithms demonstrates that models can be highly sensitive to non-natural language inputs. This implies a potential vulnerability to adversarial token sequences that could intentionally manipulate image generation for misinformation, propaganda, or other harmful purposes. Research into detecting and mitigating such adversarial token inputs is crucial for maintaining the integrity and safety of generative AI systems.

1. Rethinking "Natural Language" Interfaces: The existence of a distinct "token language" challenges the notion that natural language is the most effective or direct way to interact with generative AI. While natural language interfaces are user-friendly, they might obscure the full capabilities or vulnerabilities of the underlying model. Future AI interfaces could potentially offer "token-aware" prompting tools, allowing advanced users or researchers to directly manipulate token sequences for more precise control, artistic exploration, or security testing.

Key Takeaways

• Tokenization is a foundational, pre-neural network process that converts human language into computable units for AI, directly impacting how models understand and generate content.
• Byte Pair Encoding (BPE) creates biases based on data frequency, leading to linguistic discrimination where non-Latin languages are often more "expensive" (requiring more tokens) than English.
• The token dictionary embeds cultural and commercial influences, with popular brands, social media hashtags, and celebrities often represented as single, powerful tokens due to their high frequency in training data.
• A hidden "token language" exists, distinct from natural language, where different token sequences can decode to the same word but produce vastly different images, enabling generation of "unreachable" content through conventional prompts.
• Genetic algorithms can reverse-engineer images into specific token sequences, demonstrating a powerful method to understand and precisely control AI outputs at a sub-linguistic level.
• Transparency and auditing of tokenizer training data are crucial to mitigate inherent biases, understand cost implications, and defend against novel forms of prompt manipulation or censorship bypass.

About the Speaker(s)

Ting-Chun Liu and Leon-Etienne Kühr are media artists and researchers deeply engaged with the intersection of art and computer science. Both previously associated with the KH (likely Kunsthochschule für Medien Köln), they now pursue their work at different institutions. Ting-Chun Liu is currently affiliated with the Bahuni Enva, while Leon-Etienne Kühr works at an AI lab in Offenbach. Their artistic research methodology, as highlighted in the talk, involves playful yet rigorous exploration of AI systems to uncover their underlying mechanisms and aesthetic potentials. Their previous work included an exploration of "imaginary words" that exist outside conventional language but within AI models, setting the stage for their current deep dive into tokenization. They approach their research with an open mind, acknowledging the possibility of error and inviting collaboration from experts in the field.