Costume Jewelry: How Capital One Uses Tokenization to Protect Data

Costume Jewelry: How Capital One Uses Tokenization to Protect Data

Speakers: Andy Ozment, Chief Technology Risk Officer, Capital One

Conference: RSAC 2024

YouTube: https://www.youtube.com/watch?v=iVI6avYmu8w

Overview

In this compelling talk at RSAC 2024, Andy Ozment, Chief Technology Risk Officer at Capital One, detailed the financial institution's extensive and evolving journey into tokenization as a cornerstone of its data protection strategy. Titled "Costume Jewelry," the presentation eloquently metaphorized Capital One's approach: rather than guarding priceless crown jewels, they transform sensitive customer data into "costume jewelry" – tokens – that hold no intrinsic value to an attacker. This strategy dramatically reduces the blast radius of a potential data breach, rendering stolen data useless.

The talk highlighted Capital One's unique position as a major financial institution that has fully migrated to the public cloud, a move that has significantly accelerated their ability to implement such a comprehensive tokenization program. Ozment shared candid insights into the challenges and triumphs of this multi-year endeavor, emphasizing that securing customer data is their paramount concern. The discussion delves into the technical intricacies, operational hurdles, and strategic lessons learned from deploying tokenization across an enterprise with hundreds of millions of customers and thousands of applications.

This presentation is crucial for any organization grappling with the complexities of protecting vast amounts of sensitive data, especially those in highly regulated industries or those undertaking significant cloud transformations. Capital One's experience offers a real-world blueprint for how a large enterprise can proactively neutralize the value of sensitive information, shifting from a reactive breach response to a proactive data-centric security posture. It underscores the importance of a holistic, enterprise-wide approach to data security that goes beyond traditional encryption methods.

Background

▶ Watch: Capital One's "costume jewelry" approach to data protection (0:25)

Capital One's journey to pervasive tokenization is deeply rooted in its corporate philosophy and technological evolution. Founded in 1994, Capital One distinguished itself early on by leveraging data and technology to innovate in the credit industry. This foundational ethos of embracing technology and challenging conventional banking practices permeates its approach to cybersecurity. A key tenet of Capital One's operational strategy is its "hate legacy technology" stance, which drove a monumental decision in 2015 to go "all in" on the public cloud. This ambitious migration was completed by 2020, resulting in the shutdown of all on-premise data centers, with under 1% of applications remaining in co-location facilities. This full cloud adoption, primarily on AWS, provided the agility and scalability necessary to embark on an enterprise-wide tokenization initiative.

Before delving into tokenization, Ozment clarified a critical concept: readable sensitive data. While data is often encrypted at rest (e.g., on encrypted hard drives, S3 buckets, or via database-level/column-level encryption), Capital One considers it "readable" if a legitimate administrator or an application can access and view the unencrypted form. The primary threat scenarios addressed by this definition include malicious insiders or, more commonly, adversaries compromising an admin's account or an application itself, thereby gaining access to the cleartext data.

Capital One considered various solutions but ultimately gravitated towards tokenization, viewing it as their second preferred option after deletion – their absolute favorite. The decision to prioritize tokenization over traditional encryption methods was driven by several compelling benefits. Tokenization dramatically simplifies key management, a notoriously complex and labor-intensive aspect of encryption, even with modern advancements. Furthermore, tokenization eliminates the possibility of compromising or brute-forcing an encryption key, as no cryptographic key is directly involved in transforming the sensitive data into a token in the same way. A more subtle but significant advantage is the ability to maintain format preservation. For instance, a social security number (SSN) can be tokenized while retaining its XXX-XX-XXXX format, allowing existing applications to process and display it without extensive code changes, even though the underlying data is now a meaningless token. This ease of adoption for applications that don't need to see the cleartext data was a major factor in its widespread enterprise deployment.

Key Findings

▶ Watch: Capital One's all-in public cloud migration strategy (2:40)

Capital One’s tokenization program is not merely a point solution but an enterprise-wide initiative designed to protect sensitive data across all storage types. The core of their strategy is a four-bucket process:

1. Inventory: For existing applications, a manual inventory of sensitive data was conducted, which, surprisingly, yielded valuable insights despite initial skepticism about its trustworthiness. For new applications, data inventory is integrated into build readiness and launch readiness gates, ensuring proper data handling from inception.
2. Automated Scanning: Recognizing the limitations of manual inventories, Capital One built its own proprietary scanning tool, starting in late 2018. This custom tool was deemed necessary to achieve the required accuracy and cost-effectiveness at Capital One’s immense scale, focusing specifically on their use cases and managing false positive rates. The team consciously accepted high false positive rates (around 30% for SSNs) to minimize the risk of missing true positives. The scanning primarily targets S3 buckets and the data lake (also built on S3), with a process to trace upstream to the original data source (e.g., a database) for remediation. For databases not directly scanned, a one-time dump to S3 for scanning and remediation is performed.
3. Dispositioning Alerts: The process of handling findings is highly structured. A single sensitive data instance is a "finding." Multiple findings (e.g., an entire column of SSNs in a database) are grouped into an "alert." Related alerts within a dataset form a "case," and all cases assigned to an individual’s responsibility constitute a "bucket." This aggregation ensures accountability and manageable workloads.
4. Addressing Readable Sensitive Data (Remediation): Capital One has established clear Service Level Agreements (SLAs) for remediation. Teams have 10 days to disposition an alert (true vs. false positive) and 120 days to resolve it by deleting data or adopting an approved protection method. If a resolution isn't met within 120 days, a formal remediation plan must be approved. The preferred remediation methods, in order of preference for structured data, are tokenization and field-level encryption. For unstructured data, object or message-level encryption or file-level encryption are acceptable. Default solutions like database-level or bucket encryption, or TLS 1.2 for data in transit, are considered exceptions requiring approval and continuous tracking, as they are not deemed satisfactory long-term outcomes.

A significant finding was the necessity of a custom-built tokenization engine. Commercial offerings were found to be inadequate for Capital One's scale and performance requirements, which demand 160 billion operations per month (tokenization or detokenization calls) compared to a leading commercial provider's advertised 2 billion operations per month. This custom engine was also designed for easy adoption, with case studies showing new applications integrating it in as little as two weeks, and existing applications in about a month, often with part-time developer effort.

The program's progression shows a substantial chunk of applications reaching a "destination state" (using approved protection methods), with a smaller number "in flight." However, the talk candidly acknowledged the existence of "exceptions," particularly with third-party applications or engagements with external government entities that cannot accept anything other than unencrypted or minimally encrypted data. The biggest strategic lesson learned was viewing the program not as a typical security adoption project with a finite end, but as a continuous detection and response program, akin to a Security Information and Event Management (SIEM) feed, due to the dynamic nature of data, new data types, country expansions, and acquisitions.

Technical Deep Dive

▶ Watch: Specific sensitive data types Capital One tokenizes (3:20)

Capital One's tokenization architecture is designed for immense scale, low latency, and ease of integration. The core concept revolves around replacing sensitive data elements with non-sensitive substitutes, or tokens, which are meaningless without access to the tokenization engine.

Data Types and Scope: Capital One tokenizes a wide array of sensitive data, including Social Security Numbers (SSNs), taxpayer IDs (in other nations), credit card numbers, bank account numbers, passport numbers, and driver's license numbers. Interestingly, date of birth is a data type they've debated but decided against tokenizing due to its frequent need in cleartext. The decision to tokenize is driven by the concept of "readable sensitive data," which includes data encrypted at storage or database levels if an application or legitimate administrator can still access its cleartext form.

Tokenization Process Flow:

1. An application receives raw sensitive data (e.g., from a user or another application).
2. The application sends this raw data to the Capital One tokenization engine.
3. The engine generates a unique token corresponding to the raw data.
4. The application receives the token and uses it for transactions, stores it in databases, or displays it to users who don't require the original value.
5. If the untokenized version is needed, the application sends the token back to the engine for detokenization.

A key technical feature is format-preserving tokenization (FPT). For example, an SSN can be tokenized to a value that maintains the XXX-XX-XXXX format, allowing legacy applications or user interfaces to function without significant modification, even though the underlying data is now a token. This significantly lowers the barrier to adoption for existing systems.

Data Discovery and Scanning:

The foundation of the program is its robust data discovery capability. Capital One's custom-built scanning tool focuses primarily on S3 buckets, including their vast data lake which is built on S3. The rationale is that most data eventually flows into the data lake. When sensitive data is found in S3, the tool traces it upstream to its origin, typically a database used by an application. Remediation then occurs at the source database. For databases that don't routinely dump to S3, a process exists to perform a one-time dump to S3 for scanning and remediation. The scanning process is stateful, meaning it skips previously identified false positives to improve efficiency over time, although continuous full scans are not performed. The cost for this scanning is significant, estimated at $50,000 per petabyte.

Remediation Hierarchy:

Capital One provides a clear hierarchy of acceptable protection methods for "readable sensitive data":

• Structured Data (Preferred):

1. Tokenization
2. Field-level encryption

• Unstructured Data (Preferred):

1. Object or message-level encryption
2. File-level encryption

• Exceptions (Require Approval & Tracking):
• Database-level encryption
• Bucket encryption
• TLS 1.2 as the sole data-in-transit protection

These exceptions are not considered satisfactory long-term solutions and are continuously tracked with the goal of transitioning to stronger controls.

Custom Tokenization Engine:

The decision to build a custom tokenization engine was driven by Capital One's unique scale. The engine supports 160 billion tokenization/detokenization operations per month, far exceeding the capacity of commercial tools available at the time of evaluation (e.g., a leading provider advertised 2 billion operations/month). This custom solution ensures low latency and high availability, critical for a financial institution. While the specific cryptographic approach (e.g., format-preserving encryption (FPE) or substitution) was not disclosed due to legal sensitivities, the speaker mentioned trying both stateless and stateful approaches, favoring stateless local instances per application. The engine is also designed to integrate robust logging and detection capabilities, looking for anomalies in detokenization call volumes as an indicator of potential compromise.

Challenges in Implementation:

Two significant technical challenges highlighted were:

1. Free Text Fields: Customers or call center agents sometimes inadvertently enter sensitive data into unstructured text fields. Capital One is exploring browser plugins for internal agents to redact or prompt users at the point of entry, but customer-initiated input remains a difficult problem.
2. Clean Room for Investigations: To prevent "pollution" of sensitive data during alert dispositioning, Capital One is developing a clean room approach. This provides automated, temporary, and securely managed environments for investigators to validate true positives without exposing the sensitive data to unapproved storage or systems.

The underlying success of this large-scale technical migration was attributed to Capital One's prior cloud migration, which involved significant application modernization rather than simple lift-and-shift, making it easier for applications to integrate with new API-driven security services like the tokenization engine.

Demo / Proof of Concept

▶ Watch: Defining "readable sensitive data" and its implications (4:05)

While the talk did not feature a live technical demonstration or a traditional proof of concept, Andy Ozment presented compelling case studies that function as real-world proofs of the tokenization engine's effectiveness and ease of adoption within Capital One.

These case studies highlighted three distinct teams:

1. New Application (Greenfield): A team developing a new application serving 100 million customers successfully integrated the tokenization tool in just two weeks. The developer's feedback was exceptionally positive: "Onboarding was fairly easy, I have nothing bad to say. Tokenization is highly resilient and highly available." This underscores the successful design for developer experience and operational reliability.
2. Existing Application (Early Project Phase): Another team, early in the project, implemented tokenization for an application that performs 5 million detokenization calls daily. This integration took approximately four weeks with one developer working part-time. The extended timeline was attributed to early project back-and-forth, not inherent difficulty.
3. Existing Application (Back Book Challenges): A third team also completed their front-book implementation in about a month, with one developer for less than a month. While they faced some challenges with the "back book" (legacy data), the overall sentiment for new integrations was positive, reinforcing the "under a month" adoption goal.

These examples collectively serve as a robust proof of concept, demonstrating that Capital One's custom-built tokenization engine is not only capable of handling massive scale and traffic but is also designed for rapid and relatively straightforward integration by development teams, a critical factor for enterprise-wide adoption.

Defensive Implications

▶ Watch: Advantages of tokenization over traditional encryption (5:30)

Capital One's journey provides several critical defensive implications for organizations aiming to strengthen their data protection posture:

1. Prioritize Data Deletion: The first line of defense should always be deletion. Organizations should actively identify and eliminate data that is no longer needed. If data doesn't exist, it can't be stolen.
2. Embrace Tokenization for Sensitive Data: Tokenization offers a powerful control for reducing the blast radius of a breach. By replacing sensitive data with meaningless tokens, organizations can dramatically lessen the impact of a successful attack, as stolen data will be useless to adversaries.
3. Comprehensive Data Inventory and Discovery: A robust and continuous data inventory and discovery program is essential. This includes both manual processes for existing systems and automated scanning for all data stores (e.g., S3 buckets, data lakes). The ability to trace sensitive data back to its source is crucial for effective remediation.
4. Custom Tools for Scale (If Necessary): While commercial products should always be evaluated first, organizations operating at extreme scale or with unique performance requirements may need to build custom solutions. Capital One's experience highlights that off-the-shelf tools may not always meet the demands of truly massive enterprises (e.g., 160 billion operations per month).
5. Accept High False Positives in Scanning: When building or configuring data discovery tools, it can be strategically advantageous to accept a higher false positive rate to ensure that true positives (actual sensitive data) are not missed. This shifts the operational burden to dispositioning, but prioritizes risk reduction.
6. Define Clear Remediation Paths and SLAs: Establish a clear hierarchy of acceptable data protection methods (e.g., tokenization > field-level encryption > object-level encryption > exceptions). Implement strict SLAs for dispositioning and remediating findings to drive accountability and progress. Track exceptions rigorously.
7. Treat as a Detection and Response Program: Data protection, especially with discovery tools, should be viewed as an ongoing detection and response program rather than a finite adoption project. New data types, acquisitions, and evolving business needs will continuously introduce new challenges. Integrate data findings into existing security operations (e.g., SIEM).
8. Address Free Text Fields: Free text fields are a persistent challenge for sensitive data leakage. Implement controls at the point of entry (e.g., browser plugins for internal users) and educate users on appropriate data handling. This is an area requiring continuous innovation and vigilance.
9. Secure Investigation Environments: When security teams need to investigate potential sensitive data findings, provide dedicated, secure clean room environments. This prevents further proliferation of sensitive data during the investigative process and ensures proper data hygiene.
10. Leverage Cloud Modernization: For organizations undergoing cloud migration, prioritize application modernization over simple lift-and-shift. Modernized applications are significantly easier to integrate with advanced security services like tokenization engines, accelerating adoption and effectiveness.
11. Logging and Anomaly Detection on Tokenization Engine: Implement comprehensive logging on the tokenization engine itself. Monitor for anomalous detokenization calls, unusual volumes, or patterns that could indicate a compromise or unauthorized access, providing a critical layer of detection.

Key Takeaways

• Tokenization is a powerful control to minimize breach impact: By transforming sensitive data into meaningless "costume jewelry," Capital One dramatically reduces the value of stolen information, making it useless to attackers.
• Enterprise-wide adoption requires custom solutions for scale: Commercial tokenization tools often fall short for organizations operating at Capital One's scale (160 billion operations/month), necessitating custom-built engines that prioritize performance, low latency, and ease of developer adoption.
• Data protection is a continuous detection and response program, not a one-time project: Organizations must adopt an operational mindset, continuously scanning, identifying, and remediating sensitive data, as new data types, systems, and integrations will always emerge.
• Prioritize deletion and establish clear remediation hierarchies: The first and best defense is deleting unnecessary data. For data that must be retained, define a clear set of preferred protection methods (e.g., tokenization, field-level encryption) with strict SLAs and rigorous tracking for exceptions.
• Cloud migration and application modernization are accelerants: Moving to the cloud and modernizing applications significantly simplifies the integration of advanced security controls like tokenization, enabling faster and more widespread adoption across the enterprise.
• Operational challenges like free text fields and secure investigations require dedicated solutions: Unstructured data in free text fields poses a persistent challenge, and secure "clean room" environments are essential for investigators to safely disposition sensitive data alerts without further data exposure.

About the Speaker(s)

Andy Ozment is the Chief Technology Risk Officer at Capital One, a position he has held since 2020. In this role, he leads the second line of defense for technology risk within the financial institution. Prior to joining Capital One, Ozment had a distinguished career in both government and the private sector. He served as President Obama's Deputy Cyber Czar, playing a critical role in national cybersecurity policy. He also led the cybersecurity portion of what is now CISA (Cybersecurity and Infrastructure Security Agency) at the Department of Homeland Security (DHS) in its earlier iteration. Before his tenure at Capital One, he was the first-line CISO at Goldman Sachs for approximately four years. As a senior leader, Ozment emphasizes that the extensive work detailed in his presentation is the result of dedicated teams at Capital One, whose efforts he gratefully acknowledged during the talk.