Open Sesame! On the Security and Memorability of Verbal Passwords

Open Sesame! On the Security and Memorability of Verbal Passwords

Speakers: Eunsoo Kim, Researcher, Sanguan University; Kiho Lee; Doowon Kim; Hyoungshick Kim

Conference: IEEE S&P

YouTube: https://www.youtube.com/watch?v=cUZjxpyI3qc

Overview

This talk, "Open Sesame! On the Security and Memorability of Verbal Passwords," presented by Eunsoo Kim from Sanguan University, delves into the critical aspects of security and memorability for verbal passwords, an increasingly relevant authentication method. Conducted in collaboration with Etri in South Korea and the University of Tennessee in the United States, the research addresses the growing impracticality of traditional text-based passwords in contexts like VR headsets, smart home interfaces, and mobile banking, where typing is cumbersome or impossible.

The core of the study investigates whether verbal passwords, despite their ease of use and accessibility, can withstand sophisticated guessing attacks while remaining memorable for users over time. The researchers conducted a large-scale user study involving over 2,000 participants, complemented by advanced AI-driven password guessing models. Their findings reveal that while verbal passwords initially exhibit significant vulnerabilities without proper policies, they can become a secure and highly memorable alternative to text passwords when simple, well-designed creation policies are applied. This work is crucial for guiding the development of more user-friendly and robust authentication systems in an era of diverse computing interfaces.

Background

▶ Watch: Why verbal passwords are a practical alternative (0:22)

The ubiquitous text password, despite its widespread use, presents significant usability challenges in modern computing environments. Devices such as virtual reality (VR) headsets, smart home systems, and even some mobile applications make text entry awkward, slow, or entirely impractical. This limitation has spurred the development of alternative authentication methods, including graphical passwords, gesture-based passwords, and the focus of this research: verbal passwords.

While graphical and gesture passwords offer some relief, they often necessitate specific screen interfaces or sensors, which can restrict their applicability across various devices and contexts. Verbal passwords, by contrast, stand out due to their inherent simplicity and ease of use—users merely speak their password. This method requires no special hardware beyond a microphone, making it a highly accessible option for hands-free or screen-less interactions. Verbal passwords are not a nascent concept; they are already gaining traction and are popular in sensitive applications such as financial services and smart home security systems, underscoring the urgent need to rigorously evaluate their security and memorability before broader adoption. Prior work has largely focused on the linguistic aspects or basic recall, but a comprehensive study combining large-scale user behavior with advanced AI-driven attack simulations was lacking, which this research aims to fill.

Key Findings

▶ Watch: Verbal passwords are highly vulnerable to guessing attacks (2:47)

The research yielded several critical findings regarding the security and memorability of verbal passwords, structured around two main user studies.

The first study involved over 2,000 participants who created and practiced their verbal passwords. The initial recall tests showed promising results:

• Mid-term recall: Approximately 64% of participants successfully recalled their passwords after two days.
• Long-term recall: 52% of participants still remembered their passwords after nine days.
• For those who failed to recall, the primary reasons were complete forgetting or remembering only a partial phrase. This demonstrates a decent inherent memorability for user-chosen verbal phrases.

However, the security assessment of these initial, policy-free verbal passwords revealed significant vulnerabilities against guessing attacks:

• The researchers trained two sophisticated password guessing models, Passphrase GPT (based on GPT2) and Passphrase Llama (based on Llama 3). These models were trained on a vast corpus of over 25 million phrases compiled from public sources like Wikipedia and movie reviews.
• Each model was then used to generate up to 1 billion guesses against the collected passwords.
• The attack success rates were alarmingly high: nearly 40% of the passwords were correctly guessed by Passphrase GPT, and 30% by Passphrase Llama.
• A surprising and concerning finding was that approximately 40% of these correctly guessed passwords were also among the ones successfully recalled by users, indicating a direct correlation between memorability and guessability in the absence of constraints.
• Analysis of the compromised passwords showed that most were very short, consisting of less than three words. Common patterns included numeric sequences like "one two three" and phrases resembling natural speech patterns, such as "I love my." When passwords were sorted by entropy, simple choices like "poppy" or "banana" were easily compromised, while more complex, longer phrases demonstrated resilience. This clearly highlighted that verbal passwords, when chosen without restrictions, are inherently weak.

The second study focused on improving security through password creation policies:

• Based on the finding that four-word verbal passwords struck the best balance between security and memorability, two policies were introduced:
• VPL (Verbal Password Length): Required verbal passwords to be at least four words long.
• VPLB (Verbal Password Length + Blacklist): Combined the VPL policy with blacklist filtering, similar to what's used for text passwords, to prevent the use of common or easily guessable phrases.
• For comparison, a control group was assigned a text password policy based on NIST guidelines.
• Security Evaluation with Policies: The same guessing attacks were re-run. The VPLB policy demonstrated the lowest guess rate at just 7.5%, significantly outperforming the other policies. While the differences in guess rates between policies were not statistically significant, the VPLB policy consistently showed the best performance.
• Memorability Evaluation with Policies: Crucially, verbal passwords created under these policies were recalled significantly better than text passwords, both in midterm and long-term recall tests. This indicates that security improvements did not come at the cost of memorability.
• User Feedback: Participants expressed a willingness to use verbal passwords for casual and frequently used accounts (e.g., social media, shopping). However, concerns were raised regarding the risk of being overheard or having their voices recorded and misused, pointing to the need for careful application context.

In summary, the research definitively concludes that while raw, unconstrained verbal passwords are insecure, applying simple policies such as a minimum word count and blacklist filtering can dramatically enhance their security without compromising their inherent memorability advantage over text passwords. This makes them a viable and practical alternative for various authentication scenarios.

Technical Deep Dive

▶ Watch: Designing policies for secure and memorable verbal passwords (4:25)

The technical rigor of this research is primarily evident in the design and execution of the password guessing attacks and the subsequent policy development. The researchers leveraged state-of-the-art language models to simulate realistic and potent adversaries.

The core of the attack infrastructure comprised two distinct password guessing models: Passphrase GPT and Passphrase Llama.

• Passphrase GPT was built upon the GPT2 architecture, a transformer-based language model known for its ability to generate human-like text.
• Passphrase Llama utilized the Llama 3 architecture, a more recent and powerful large language model.

Both models were chosen for their advanced natural language processing capabilities, making them highly effective at predicting sequences of words that resemble natural human speech, which is precisely what verbal passwords often are.

To train these sophisticated models, a massive dataset of over 25 million phrases was meticulously compiled. This dataset was sourced from publicly available texts, including Wikipedia articles and movie reviews. This diverse collection ensured that the models were exposed to a wide array of linguistic patterns, common expressions, and natural sentence structures, enabling them to generate highly plausible verbal password candidates. The sheer volume of training data contributed to the models' ability to identify subtle statistical relationships and common collocations within natural language.

During the attack phase, each trained model was configured to generate an astounding up to 1 billion unique password guesses. This extensive guessing space was designed to thoroughly test the resilience of the verbal passwords, simulating a highly motivated and resource-rich attacker. The comparison between the generated guesses and the actual user-created passwords allowed for a quantitative measure of guessability.

The analysis of initial, policy-free verbal password choices revealed several structural weaknesses:

• Short Length: A significant proportion of compromised passwords consisted of fewer than three words. This short length inherently limits the entropy and makes brute-forcing or dictionary attacks more feasible.
• Predictable Sequences: Many users opted for simple numeric sequences (e.g., "one two three") or common, easily guessable words ("poppy", "banana").
• Natural Language Patterns: Passwords often resembled full sentences or common phrases (e.g., "I love my..."), reflecting natural speech. While memorable, these patterns are precisely what large language models excel at generating. The models could exploit the grammatical and semantic coherence often present in user-chosen phrases.
• Entropy Analysis: The researchers explicitly sorted passwords by entropy in ascending order, demonstrating a clear correlation: passwords with lower entropy (simpler, shorter, more predictable) were invariably compromised, whereas those with higher entropy (more complex, longer, less common combinations) withstood the attacks.

This detailed understanding of password weaknesses informed the design of the improved verbal password policies:

• Minimum Word Count: The study empirically determined that a minimum of four words offered the optimal balance between security and memorability. This policy directly addresses the observed vulnerability of short passwords by enforcing a greater length, thereby increasing the potential entropy.
• Blacklist Filtering: Similar to text password policies, this involves preventing users from choosing verbal passwords that are known to be common, easily guessable, or have appeared in breaches. This proactive filtering mechanism directly counters the models' ability to predict common phrases and numeric sequences.

The comparison of verbal passwords created under these policies with text passwords created under NIST guidelines (e.g., minimum length, character variety) highlighted a fundamental difference in their structure. Verbal passwords, even when constrained by policies, tended to be "generally longer and readable phrases," emphasizing their linguistic nature, in contrast to the often arbitrary and less memorable character sequences of strong text passwords. This linguistic coherence, while a potential security risk without policies, becomes a memorability advantage when combined with appropriate constraints.

Demo / Proof of Concept

▶ Watch: Policies drastically improve verbal password security (4:57)

The talk commenced with a concise and effective demonstration showcasing the practical application of a verbal password in a mobile banking context. This served as a compelling proof of concept for the user experience that verbal passwords aim to provide.

The demonstration depicted a user attempting to complete a transaction within a mobile banking application. Instead of navigating a virtual keyboard or inputting a PIN, the user simply spoke their password aloud. The specific phrase used in the demo was "Please let me in." Upon speaking the password, the system processed the verbal input. If the spoken password matched the stored verbal credential, the system validated it, and the bank transaction was successfully completed. This short demonstration effectively illustrated the ease, speed, and hands-free nature of verbal authentication, highlighting its potential to streamline secure interactions, particularly in environments where traditional text entry is inconvenient.

Beyond this initial demo, the user study itself served as a large-scale practical demonstration of both memorability and security. Participants were tasked with recalling their verbal passwords to "complete a bank transaction," mimicking the real-world usage scenario from the initial demo. This setup for midterm and long-term recall tests (after two and nine days) further validated the practical feasibility and user experience aspects of verbal passwords in a controlled environment.

Defensive Implications

▶ Watch: Verbal passwords are a practical, secure, and memorable alternative (6:20)

The research provides clear and actionable guidance for developers, security architects, and organizations considering the adoption of verbal passwords. The findings underscore that while verbal passwords offer significant usability advantages, their implementation requires careful consideration of security policies.

1. Mandate Minimum Word Count: The most critical defensive measure is to enforce a minimum word count for verbal passwords. The study specifically identified four words as striking the best balance between security and memorability. Organizations should not allow verbal passwords shorter than this threshold, as shorter phrases were found to be highly susceptible to guessing attacks.
2. Implement Blacklist Filtering: Similar to text passwords, a robust blacklist filtering mechanism must be applied to verbal passwords. This involves compiling and regularly updating a list of common, easily guessable words, phrases, numeric sequences ("one two three"), and patterns (e.g., simple sentences like "I love my dog"). Users should be prevented from choosing passwords that appear on this blacklist.
3. Educate Users on Strong Verbal Password Creation: While policies help, user education is crucial. Users should be advised to:

• Choose phrases that are unique and not directly related to personal information.
• Avoid common idioms, proverbs, or well-known quotes.
• Opt for slightly longer, more complex, or even nonsensical phrases that are memorable to them but unpredictable to an attacker or language model.
• Avoid simple numeric sequences.

1. Contextual Application of Verbal Passwords: Defenders must carefully assess the risk profile of the application before deploying verbal passwords.

• For casual and frequently used accounts (e.g., social media, shopping), where the risk of financial loss or severe privacy breach is lower, verbal passwords with strong policies can be a practical and user-friendly alternative.
• For high-security applications (e.g., critical financial transactions, highly sensitive data), the concerns raised by participants regarding being overheard or voice recordings being misused are significant. In such scenarios, verbal passwords might need to be augmented with a second factor of authentication (MFA) or reserved for specific, controlled environments. The possibility of eavesdropping or voice recording and subsequent replay attacks remains a distinct threat that standard verbal password policies alone cannot fully mitigate.

1. Leverage Memorability for Adoption: The inherent memorability of verbal passwords, even with policies, is a powerful advantage. Defenders can leverage this by promoting verbal passwords as a more user-friendly alternative to complex text passwords, thereby potentially improving user adherence to security practices and reducing password fatigue.
2. Continuous Monitoring and Policy Adjustment: As language models and attack techniques evolve, organizations should continuously monitor the effectiveness of their verbal password policies and be prepared to adjust them (e.g., updating blacklists, refining word count requirements).

By adopting these defensive strategies, organizations can harness the usability benefits of verbal passwords while significantly mitigating their inherent security risks, making them a viable and practical authentication method in a diverse digital landscape.

Key Takeaways

• Verbal passwords offer a highly convenient and accessible alternative to text passwords, particularly for devices where text entry is difficult or impractical.
• Without proper creation policies, verbal passwords are highly insecure, with up to 40% susceptible to guessing attacks by sophisticated AI models like Passphrase GPT and Passphrase Llama.
• The insecurity of policy-free verbal passwords is linked to their brevity (often less than three words), use of numeric sequences, and reliance on natural, predictable speech patterns.
• Implementing simple policies, such as a minimum of four words and blacklist filtering, dramatically improves the security of verbal passwords, reducing guess rates to as low as 7.5%.
• Crucially, these security enhancements do not compromise memorability; verbal passwords with policies are recalled significantly better than text passwords in both midterm and long-term tests.
• While suitable for casual accounts, concerns about eavesdropping and voice recording necessitate careful consideration of verbal passwords for high-security applications or the integration of multi-factor authentication.

About the Speaker(s)

The primary speaker for this presentation was Eunsoo Kim, affiliated with Sanguan University in South Korea. The research was a collaborative effort, also involving Kiho Lee, Doowon Kim, and Hyoungshick Kim. The work was conducted in partnership with Etri in South Korea and the University of Tennessee in the United States, highlighting a multi-institutional and international collaboration in addressing critical cybersecurity challenges. Eunsoo Kim presented the findings with confidence, demonstrating a deep understanding of the user study methodologies, the technical aspects of AI-driven password guessing, and the implications for future authentication systems.