Hackers Challenge Shakedown

Hackers Challenge Shakedown

Speakers: Santiago Gimenez Ocano (Santi - Hackers Challenge GameMaster)

Conference: SAINTCON

YouTube: https://www.youtube.com/watch?v=-hI1izhlml0

Overview

This article provides a detailed technical retrospective of the SAINTCON 2025 Hackers Challenge, presented by GameMaster Santiago Gimenez Ocano (Santi) and several key challenge creators. The talk delves into the design philosophy, statistical outcomes, and intricate technical solutions behind a diverse array of challenges, ranging from traditional web and hardware hacking to novel social engineering AI and interpreted reverse engineering puzzles. It serves as an invaluable resource for both aspiring CTF participants seeking to understand complex solutions and challenge designers looking for inspiration and best practices.

The session highlights the record-breaking participation in SAINTCON's CTF, with 350 registered users, while also emphasizing the increased difficulty of this year's challenges, evidenced by a significant drop-off in players solving more than a handful of puzzles. Speakers unpack the specific vulnerabilities exploited, the defensive implications of these attack vectors, and the unexpected ways players approached the challenges. A particular focus is placed on the evolving role of Artificial Intelligence (AI) in both creating and solving CTF puzzles, and the strategies employed to "AI-proof" challenges.

Beyond a mere recap of flags, this talk offers a deep dive into the creative process behind complex security puzzles. It showcases the ingenuity required to craft engaging and realistic scenarios that test a broad spectrum of cybersecurity skills, from low-level binary exploitation and hardware interaction to high-level social engineering and application security. The insights shared by the challenge creators provide a unique glimpse into the current state of offensive and defensive security techniques in a competitive, educational environment.

Background

▶ Watch: Welcome, challenges overview, and community thanks (0:00)

The SAINTCON Hackers Challenge is an annual Capture The Flag (CTF) event designed to test participants' cybersecurity skills across various domains. Each year, the GameMasters and a community of contributors craft a fresh set of challenges, aiming to balance difficulty, innovation, and educational value. The context for the 2025 shakedown is a reflection on the successes and challenges of the most recent iteration, particularly in light of evolving attack techniques and the increasing prevalence of AI tools in the security landscape.

Historically, CTFs serve as crucial training grounds, allowing participants to apply theoretical knowledge to practical problems. They expose players to common vulnerabilities, exploit development, reverse engineering, and defensive strategies in a controlled environment. The problem statement for CTF organizers is multifaceted: how to create puzzles that are challenging enough to be engaging, yet solvable; how to cover a broad range of technical disciplines; and how to adapt to new technologies and adversarial tactics. The 2025 challenge aimed for a high bar, with 70 main-stage challenges and three for the finals, drawing on expertise from various communities including AppSec, hardware hacking, lockpicking, and RF/NFC.

A significant background element this year was the rising influence of AI. Challenge creators were keenly aware that participants might leverage AI tools like ChatGPT to assist in solving puzzles. This presented a new dimension to CTF design: how to craft challenges that resist trivial solutions by AI, or conversely, how to integrate AI as a tool or target within the challenge itself. The discussions during the shakedown reveal a community grappling with these questions, sharing both successes and ongoing efforts to refine AI-resistant or AI-aware challenge design. The statistics presented, such as the number of attempts on specific challenges and the overall solve rates, provide a quantitative measure of the difficulty and player engagement, offering valuable feedback for future CTF iterations.

Key Findings

▶ Watch: Announcing top 5 players and opening for audience questions (4:00)

The SAINTCON 2025 Hackers Challenge saw a record-breaking 350 registered users, indicating a high level of interest and participation. However, the difficulty of the challenges was notably high this year, with only 221 participants solving at least one challenge and a mere 54 solving ten or more. This stark contrast suggests that while engagement was broad, the depth of challenge required significant skill and perseverance.

Several challenges stood out in terms of their difficulty or popularity:

• Easiest Challenge: "Bot Sync Level 4" was solved by 131 people, making it the most accessible puzzle.
• Unsolved "Fun" Challenges: "Forge Master Cinder" and "Judge the Judge Meridian" remained unsolved, highlighting their extreme difficulty.
• Most Attempted/Hardest to Solve: "Reading from Zlag," created by Josh, garnered an astounding 181 attempts by a single player without a successful solve, with only three total solves. This challenge involved finding hidden characters, brute-forcing an MD5 hash of a phone number, and deciphering a computer-generated voice riddle.
• Top Player Performance: One anonymous player solved an impressive 43 challenges (61% of the total), demonstrating exceptional skill. Professor Plum was also lauded for his contributions, particularly in hardware hacking, though he opted to remove himself from the competition leaderboard.

The discussion also revealed insights into the Hackers Challenge Finals, a separate competition featuring top players "blank" and "dark matter" tackling challenges in lockpicking and blockchain, which ultimately resulted in a tie broken by tiebreakers.

A significant finding revolved around the role of AI in CTF solving. While some challenge creators attempted to make their puzzles "AI-proof" by making them obscure or manually obfuscated, it was noted that a main contestant had considerable success using AI, even for complex tasks like interpreted reverse engineering. This suggests that while AI can be a powerful tool for solvers, thoughtful design, such as requiring specific context, manual deobfuscation, or understanding subtle behavioral differences (e.g., JavaScript execution in a console vs. a file), can still present significant hurdles for AI-assisted approaches. Conversely, challenge creators themselves used AI for code generation, but emphasized the need for manual architecting and rigorous testing to prevent AI from inadvertently leaking flags or solutions.

Technical Deep Dive

▶ Watch: Creator explains complex 'Reading from Zevlag' challenge solution (5:00)

The SAINTCON 2025 Hackers Challenge featured a rich array of technically sophisticated puzzles, each designed to test specific skill sets and often incorporating multiple layers of complexity.

Web and Application Security

Dice Roller (Seth - AppSec Carnival): This PHP application presented a seemingly simple game: roll 30 twenty-sided dice (d20s) and achieve 30 successful rolls of '20'. The underlying vulnerability stemmed from a loose comparison in PHP code, where == was used instead of ===. The application would take a timestamp from the frontend, compute its MD5 hash, and then convert this hash to base20. If this base20 hash loosely equated to 0, the player would win. The trick was that in PHP, a string like "0E12345" (representing 0 times 10 to the power of 12345) is loosely compared to 0. Thus, participants needed to brute-force a timestamp that, when MD5 hashed and converted to base20, would result in a string starting with 0E followed by entirely integer characters. Seth noted that while his PHP code could be reused, it was too slow, and many successful solvers optimized their brute-forcing efforts using Rust.

Cryptography

Matrix Decoded (Santi): This challenge combined historical technology with cryptographic principles. Participants were presented with an audio file containing DTMF (Dual-Tone Multi-Frequency) tones. The first step involved decoding these tones into phone digits (0-9). The second, more thematic step, required interpreting these digits using a T9 keyboard—a nod to older Nokia-era cell phones and the "old movie" aesthetic of The Matrix. The sequence of T9 key presses then revealed the final message, or flag.

RSA Signature (LegoClones - Crypto): Designed by a math major, this challenge was described as highly mathematical. The core problem involved manipulating RSA signatures. Participants were tasked with finding a highly factorable number that shared the same last eight bytes as the hash of a target message. The solution then required finding a two-byte hash collision for each of these factors, signing them individually, and subsequently multiplying these signed hash collisions to construct a valid signature for the original message. This technique exploits properties of multiplicative homomorphic signatures in RSA, where sign(m1) sign(m2) = sign(m1 m2).

Hardware Hacking

Badge CTF (Redacted): This multi-component challenge centered around the SAINTCON badge, involving firmware, physical hardware, and an exposed API.

1. Firmware Analysis: The initial stage required participants to dump the badge's firmware to extract strings, specifically to find the DNS name for the API server.
2. Unique Identifier: A key stumbling block for many was identifying the unique identifier for the badge, which turned out to be its MAC address. This could be found by scanning a QR code on the chip or by listening for Wi-Fi packets emitted by the powered-on badge.
3. API Interaction: Once the MAC address was identified, participants used custom API headers for self-identification. The core of the challenge involved interacting with an I2C bus to communicate with an ATECC security crypto chip, which stored sensitive secrets. The API leveraged HMAC encryption to build signatures and employed single-use nonces for challenge-response authentication, replicating the security mechanisms of the main badge game. The speaker mentioned providing a 240-line Python script for interacting with the I2C bus and the ATECC chip, utilizing the cryptoauthlib C and Python libraries.

EM4000 (Mike - RFID/NFC): This RFID challenge involved a low-frequency (LF) EM4000 tag. Participants were given a Proxmark 3 (PM3) file containing a frequency capture of a full transaction between a reader and the tag. The technical task was to open this file in the Proxmark client, analyze the graphical representation of the signal, and interpret the Manchester signal encoding to extract the raw data, which then revealed the low-frequency card information.

Social Engineering & AI

Social Engineering AI (Clipper): A novel challenge where participants attempted to social engineer trained AI agents via physical analog phones. There were 12 AI personas (two from each of six factions), each with 10 flags, totaling 120 potential flags, though only five agents held "Hackers Challenge" flags. Agents would ask quiz questions, the answers to which had to be obtained by skillfully social engineering the AI based on its persona (e.g., feigning compliance for a "Dawn Accord" agent). Statistics included 2,513 questions asked, 124 badge codes requested, and 4 Hackers Challenge flags given out. A unique technical challenge for the creators was preventing the AI from dialing 911—a problem ingeniously solved by physically cutting the outbound dialing wires in each phone.

Pwn and Forensics (Mashup)

Pwn + Forensics 1 (Rajio): This challenge combined binary exploitation with network forensics. Rajio created a vulnerable binary and then exploited it, capturing the network traffic.

• Vulnerabilities: The binary had a stack buffer overflow in a read function and a format string vulnerability in a printf function within a check_password function. Username and password were hardcoded (admin:password).
• Exploitation: Mitigations like PI (Position Independent Executables) and NX (No-eXecute) were disabled. The exploit involved two stages:

1. An initial connection using the format string vulnerability to leak a stack address. Because the server forked for each connection, addresses remained consistent across connections.
2. A second connection to send the actual exploit payload.

• Shellcode & Forensics: The shellcode would P_open and cat the flag, store it in a stack buffer, XOR it with a hardcoded key, and then send it over the network. Participants had to analyze the packet capture, disassemble the shellcode to understand its logic, identify the XOR key, and then decrypt the exfiltrated flag from the network traffic.

Pwn + Forensics 2 (Rajio): This was a more advanced version, with NX enabled (PI remained disabled).

• Exploitation with NX:

1. RET to PLT (Procedure Linkage Table): The initial step involved a ROP chain to call a function from the PLT (specifically set_stack_options), which would then print out a libc address, allowing the attacker to determine the libc base address. The ROP chain then returned execution to the vulnerable function for a second exploitation.
2. ROP Chain & mprotect: A second ROP chain was built using gadgets from libc. This chain invoked mprotect to change the permissions of the last section of the executable to read-write-execute (R-W-X).
3. Two-Stage Shellcode: The exploit then used a read command to transfer two-stage shellcode from the network connection into the newly R-W-X memory region and then jumped to it.

• Stage 1: This shellcode decrypted the second stage. The encryption key was derived from the value of the RCX register at a specific point, multiplied by 0x1337. Participants needed to replay the exploit in a debugger (like GDB) within the provided Docker container and libc to determine the RCX value, which was based on a libc address.
• Stage 2: Once decrypted, this shellcode was similar to Forensics 1, catting the flag and XORing it with a hardcoded key for exfiltration. The challenge required intricate analysis of memory, registers, and network traffic.

Interpreted Reverse Engineering

Among Us (Rajio): This JavaScript challenge aimed to make interpreted reverse engineering as difficult as compiled binary analysis.

• Setup: A website with a fake login form. JavaScript on the client-side would parse username/password and send them to a non-existent /xfill endpoint. The goal was to determine the correct username and how the password was obfuscated.
• Obfuscation Techniques:

1. JSFuck: Extensive use of JSFuck, a technique to write arbitrary JavaScript code using only six characters ([]()!+). This made the code visually unintelligible.
2. Invisible Unicode Characters: Variable names, particularly an "Among Us" character, were visually identical but contained different invisible Unicode code points after the visible character. This made simple copy-pasting or visual inspection misleading.
3. Funky XOR Logic: The password processing involved a recursive XOR function that set up internal variables and derived an XOR key.
4. Asynchronous Hashing Glitch: The challenge used an online synchronous SHA1 hashing library. Rajio discovered that due to JavaScript's asynchronous nature, this function behaved differently when run in a browser's console versus within the actual JS file, giving incorrect hash outputs in the console. This forced participants to use a debugger or console.log within the script's execution flow.

• Solution: The specific username was hackerschallenge2025username. If the password was at least 34 characters long, it would be Base64 encoded and then XORed with the flag itself before exfiltration. The "attacker" (malicious web server owner) would then simply XOR the exfiltrated data with the known flag to retrieve the original password. The difficulty lay in unraveling the layers of obfuscation to understand this final XOR logic. Rajio deliberately used manual obfuscation to defeat automated deobfuscators.

Demo / Proof of Concept

▶ Watch: Explanation of the new, intense Hackers Challenge Finals (6:40)

While the talk itself was a retrospective and did not feature live demonstrations of exploits or tools, the discussions heavily revolved around the proof-of-concept (PoC) nature of the challenges and their solutions.

For instance, the hardware hacking challenges for the badge CTF involved a 240-line Python script that could interact with the I2C bus and the ATECC crypto chip, serving as a working PoC for communicating with the secure element. Similarly, the Pwn + Forensics challenges were built upon Rajio's own exploits, which were developed and captured as network traffic. The speaker explicitly mentioned that for Forensics 2, the easiest path to solve was to replay the provided exploit within a Docker container and use GDB to inspect register values, effectively demonstrating the exploit's mechanics and allowing participants to extract the necessary keys. In the context of the social engineering AI, the physical phones and AI agents served as a functional demonstration of the concept. Although not presented as live demos during the talk, the challenges themselves were built on and required the creation of functional PoCs by both the creators and the solvers.

Defensive Implications

▶ Watch: Detailed solution for 'Morpheus's Calling' Matrix challenge (9:00)

The technical details and findings from the SAINTCON 2025 Hackers Challenge offer several critical defensive implications for developers, security architects, and incident responders:

1. Strict Type Comparisons in Web Development: The "Dice Roller" challenge highlighted the danger of loose comparisons (==) in PHP. Developers must consistently use strict comparison operators (===) to prevent unexpected type juggling and comparison behaviors that can lead to bypasses, especially in security-sensitive contexts like authentication or flag validation. This principle extends to other languages with similar quirks.

1. Hardware Security and Firmware Analysis: The Badge CTF underscored the importance of secure hardware design and the reality that firmware can be reverse-engineered. Sensitive information like API endpoints, unique identifiers (e.g., MAC addresses), and cryptographic interaction logic can be extracted from firmware dumps. Defenders must ensure that secrets are properly provisioned and protected within secure elements (like the ATECC chip) and that API interactions are robustly secured with HMAC encryption and single-use nonces to prevent replay attacks or unauthorized access.

1. Network Forensics for Post-Exploitation Analysis: The Pwn + Forensics challenges demonstrated the critical role of packet capture analysis in understanding and responding to binary exploitation. Defenders should be proficient in inspecting network traffic for shellcode, command execution, data exfiltration (especially if XORed or encrypted), and other indicators of compromise. The ability to reconstruct exploit stages and identify encryption keys from network flows is paramount for incident response.

1. Binary Exploitation Mitigations and Bypasses: The Pwn challenges illustrated common binary exploitation techniques (stack buffer overflows, format string vulnerabilities) and how modern mitigations like NX (No-eXecute) and PI (Position Independent Executables) are bypassed. Defenders need to understand techniques like ROP (Return-Oriented Programming), RET to PLT, and memory region re-protection (mprotect) to adequately assess the risk of their compiled binaries and implement robust defenses, including Address Space Layout Randomization (ASLR), Canaries, and stricter compiler flags.

1. Understanding Obfuscation in Reverse Engineering: The "Among Us" challenge showcased advanced JavaScript obfuscation techniques (JSFuck, invisible Unicode, recursive XOR logic). While used for CTF difficulty, these are commonly employed by malware authors. Defenders involved in malware analysis or web application security must develop skills in manual deobfuscation, leveraging browser debuggers, and understanding subtle language behaviors (like asynchronous execution) to uncover malicious payloads or logic. Relying solely on automated deobfuscators against custom, manually obfuscated code can be ineffective.

1. AI in Security Operations: The discussion on AI's role highlighted its dual nature. For defenders, AI can be a powerful tool for code generation, vulnerability scanning (though with caveats), and automating analysis. However, it also means attackers may leverage AI to find vulnerabilities or solve puzzles more efficiently. Defenders need to consider how their systems and processes might be targeted by AI-assisted adversaries and explore ways to make their defenses resilient to such attacks, potentially by designing security controls that require human-specific reasoning or context.

1. Protocol Deep Dives: Rajio's experience with the Web Authentication challenges, where he found vulnerabilities by building his own server and client, emphasizes that a deep understanding of security protocols can reveal subtle flaws. Security architects and engineers should not just implement protocols but also understand their underlying mechanisms and potential edge cases to identify and mitigate novel attack vectors.

Key Takeaways

• CTFs are Evolving with Complexity and Mashups: Modern CTFs are moving beyond single-category puzzles, integrating multiple disciplines like pwn and forensics, or appsec and crypto, to create more realistic and challenging scenarios. This forces participants to think holistically.
• AI Presents a Dual Challenge and Opportunity: AI tools can significantly assist CTF solvers, but thoughtful, obscure, and manually obfuscated challenge design can still provide an "AI-proof" experience. Challenge creators also leverage AI for code generation but stress the importance of human architecting and rigorous testing to prevent inadvertent solution leaks.
• Fundamental Vulnerabilities Persist: Even in highly complex challenges, classic vulnerabilities like PHP loose comparisons, stack buffer overflows, and format string bugs remain potent. Mastery of these basics is still crucial for both offensive and defensive security.
• Obfuscation Requires Deep Understanding, Not Just Tools: Advanced, manual obfuscation techniques (e.g., JSFuck, invisible Unicode characters) can defeat automated deobfuscation tools, forcing manual reverse engineering. This highlights the need for deep language and runtime environment understanding in threat analysis.
• Hardware and Low-Level Interaction Remain Critical: Challenges involving direct interaction with hardware (I2C, crypto chips) and analysis of low-level signals (Manchester encoding for RFID) demonstrate that physical and embedded systems are vital attack surfaces requiring specialized skills.
• Social Engineering is a Persistent Threat, Even to AI: The AI social engineering challenge illustrates that even trained AI agents can be manipulated, reflecting the continued importance of understanding human psychology and interaction in cybersecurity, regardless of the target's nature.

About the Speaker(s)

The Hackers Challenge Shakedown was led by Santiago Gimenez Ocano (Santi), the Hackers Challenge GameMaster for SAINTCON 2025. Santi expressed his passion for creating challenges, witnessing participants' "aha moments," and fostering a community around the CTF. He personally contributed several challenges, including a GraphQL puzzle, a syncbot game, a middleware challenge, and the Matrix Decoded puzzle, which involved DTMF tones and T9 keyboard decoding.

Various challenge creators also spoke, providing invaluable insights into their work:

• Josh, creator of the "Reading from Zlag" challenge, detailed its intricate design involving hidden characters, MD5 hash brute-forcing, and a computer-generated voice riddle.
• Seth, representing the AppSec community, discussed his "Dice Roller" challenge, which exploited a loose comparison vulnerability in PHP.
• An anonymous speaker, referred to as "redacted," presented the hardware hacking Badge CTF, which involved firmware analysis, MAC address identification, and interaction with an ATECC security crypto chip via an I2C bus. He noted Professor Plum's exceptional skill in solving these challenges.
• Mike discussed the RFID/NFC challenges, specifically the EM4000 puzzle that required interpreting a Manchester signal from a Proxmark 3 frequency capture.
• "The Clipper," along with his partner Kyote, introduced the innovative social engineering AI challenge, where participants attempted to manipulate AI agents via analog phones, drawing on their experience from Defcon's social engineering village.
• LegoClones spoke about the crypto challenges, including the RSA Signature puzzle, which was designed by his brother, a math major.
• Rajio provided deep technical dives into his "Pwn + Forensics" mashup challenges and the "Among Us" interpreted JavaScript reverse engineering puzzle. He is passionate about combining CTF categories and challenging the status quo of interpreted reverse engineering, often using manual obfuscation techniques to make his puzzles "AI-proof."