🤖 AI

Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

This paper introduces "Capture the Flags," a framework and tool called Evolve-CTF that generates families of semantically equivalent cybersecurity challenges to evaluate agentic LLMs, revealing that while models are robust to simple code changes, their performance degrades under complex obfuscation and that explicit reasoning offers little benefit.

Original authors: Shahin Honarvar, Amber Gorzynski, James Lee-Jones, Harry Coppock, Marek Rei, Joseph Ryan, Alastair F. Donaldson

Published 2026-04-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shahin Honarvar, Amber Gorzynski, James Lee-Jones, Harry Coppock, Marek Rei, Joseph Ryan, Alastair F. Donaldson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a detective to solve a mystery. In the world of Artificial Intelligence, these "detectives" are Agentic LLMs (AI models that can think, write code, and use tools like a human would). To test if they are good detectives, researchers usually give them a series of puzzles called Capture the Flag (CTF) challenges. In these puzzles, the AI has to find a hidden secret (the "flag") inside a piece of code by exploiting a weakness.

However, there's a problem with how we usually test them. It's like giving a detective the exact same mystery 100 times, just with the names of the characters changed. If the detective memorized the solution to the first one, they might pass all 100 tests without actually understanding the mystery. They might just be recognizing patterns rather than truly solving the problem.

This paper introduces a new way to test these AI detectives called "CTF Challenge Families."

The Core Idea: The "Same Cake, Different Frosting"

Instead of giving the AI 100 different, unrelated puzzles, the researchers take one puzzle and create a whole "family" of variations from it. They use a tool called Evolve-CTF to automatically rewrite the code in ways that don't change the actual logic or the solution, but make the code look very different.

Think of it like baking a cake:

The Original Cake: A delicious chocolate cake with a hidden cherry in the middle.
The Transformations:
- Renaming (R): You change the label on the box from "Chocolate Cake" to "Mystery Dessert #42." The cake inside is identical.
- Inserting Loops (T1): You add a layer of decorative frosting that looks complicated but is actually empty space. It doesn't change the cake, but it makes it look bigger.
- Inserting Conditionals (T2): You add a fake "trapdoor" in the kitchen that leads nowhere. It looks like a secret passage, but it's just a dead end.
- Obfuscation (O): You wrap the whole cake in layers of aluminum foil, shrink-wrap it, and encrypt the box. The cake is still there, but it's incredibly hard to get to.

The goal is to see: Can the AI detective still find the cherry (the solution) even when the cake looks weird, confusing, or wrapped in foil?

How They Tested It

The researchers took 16 real-world cybersecurity puzzles (from existing benchmarks) and used their tool to generate 24 variations for each one. They then asked 13 different AI models (like GPT-5, Claude, Gemini, etc.) to solve these puzzles.

They watched closely to see:

Did the AI get confused by the new names?
Did it get lost in the fake "dead end" code?
Did it know how to use its tools (like a screwdriver or a flashlight) to unwrap the foil?

What They Discovered

1. The AI is great at ignoring "Cosmetic" Changes
When the researchers just changed the names of variables (like changing x to banana) or added some fake, useless comments, the AIs didn't care. They solved the puzzles just as easily as the originals.

Analogy: If you tell a detective, "The suspect is wearing a red hat," and then you change the report to say "The suspect is wearing a crimson cap," the detective still knows who to look for.

2. The AI struggles when things get "Messy"
When the researchers combined several changes at once (adding fake loops, fake dead ends, and confusing comments all together), the AIs started to fail.

Analogy: If you give the detective a map that has 50 fake roads drawn on it, all leading to nowhere, they get overwhelmed and can't find the real path.

3. The "Foil" (Obfuscation) is a Hard Barrier
When they wrapped the code in heavy encryption and compression (using a tool called PyObfuscator), most AIs couldn't solve it.

Analogy: It's like giving the detective a locked safe and asking them to find the key, but they don't have a safe-cracking tool. They tried to write their own tools to break the safe, but often ran out of time (or "tokens," which is like their energy budget) before they could succeed.

4. "Thinking Harder" Didn't Always Help
Some AI models have a setting to "think more" before answering. The researchers found that turning this setting on didn't really help the AIs solve these tricky puzzles.

Analogy: It's like telling a detective to "think really hard" about a locked safe. If they don't have the right tools or skills, thinking harder just makes them tired, not smarter.

5. Some Puzzles Were Too Easy
They found that three of the original puzzles were so simple that even the "messy" versions were solved by almost every AI.

Analogy: It's like giving a detective a puzzle where the answer is written on the back of the envelope. These puzzles weren't good tests of skill because they didn't challenge the AI enough.

Why This Matters

This research is a big step forward because it stops us from just "memorizing" the tests. It shows us that while current AIs are very good at reading code, they can get easily confused when that code is messy or hidden.

It also gives us a new tool (Evolve-CTF) to stress-test AI. Instead of asking, "Can this AI solve this puzzle?", we can now ask, "Can this AI solve any version of this puzzle, even if someone tries to trick it?"

In short, the paper teaches us that to build truly robust AI security experts, we need to test them not just on clean, clear puzzles, but on the messy, confusing, and "obfuscated" reality of the real world.

1. Problem Statement

Agentic Large Language Models (LLMs) are increasingly deployed in cybersecurity for tasks like vulnerability discovery and penetration testing. Current evaluation benchmarks, such as Cybench and Intercode, rely on Capture-the-Flag (CTF) challenges. However, these benchmarks suffer from three critical limitations:

Pointwise Evaluation: They treat challenges as independent tasks, offering no insight into a model's robustness against input perturbations or its ability to generalize.
Data Contamination: Since many CTFs are public, models may have memorized solutions during training, leading to inflated performance metrics that do not reflect genuine reasoning.
Lack of Robustness Testing: Existing benchmarks do not systematically test whether an agent truly understands code logic or merely relies on surface-level cues (e.g., recognizable variable names or familiar code structures).

The authors propose a shift from evaluating models on static, independent tasks to evaluating them on CTF Challenge Families.

2. Methodology

The core of the methodology is the Evolve-CTF tool, which generates families of semantically equivalent but syntactically diverse CTF challenges from a single source.

A. The Transformation Framework

Evolve-CTF applies semantics-preserving transformations to Python-based CTFs. The key constraint is that the underlying vulnerability and the exploit strategy remain identical across all instances in a family; only the code structure changes. The transformations include:

$R$ (Rename): Replaces all identifiers (variables, functions, classes) with random strings or coding-related vocabulary.
$T_1$ (Insert Loops): Adds redundant, unreachable nested loops with complex conditions.
$T_2$ (Insert Conditionals): Adds unreachable if statements or try-except blocks.
$T_3$ (Insert Functions): Adds redundant function definitions and lambda assignments.
$T_4$ (Insert Comments): Adds synthetic comments (some meaningful, some random noise).
$T_5$ (Combine): Sequentially applies $T_1$ through $T_4$ .
$O$ (Obfuscate): Applies PyObfuscator to aggressively rename identifiers, encrypt string literals, remove docstrings, and compress the code with gzip.

These transformations are composed into a tree structure (e.g., $R \to T_5 \to O$ ), creating 24 distinct instances per original CTF. This allows for compositional analysis, isolating the marginal difficulty added by each transformation.

B. Experimental Setup

Datasets: 16 Python CTFs were selected from Cybench and Intercode. Selection criteria required that state-of-the-art models could solve the original challenges to ensure the transformations, not the inherent difficulty, were the variable being tested.
Models: 13 configurations of 9 frontier models (e.g., GPT-5.1, Claude Opus 4.5, Gemini 3 Pro, DeepSeek V3.2) were evaluated. This included variations with explicit "reasoning" modes enabled.
Framework: Evaluations were conducted using the Inspect framework, which provides agentic capabilities (access to bash, python, and file tools).
Protocol: Each instance was run 5 times with a 200,000-token budget per run. Success was defined as submitting the correct flag within 3 attempts.

3. Key Contributions

CTF Challenge Families: A novel evaluation paradigm that generates structured, semantically equivalent variants of CTFs to probe robustness and generalization.
Evolve-CTF Tool: An open-source tool that automates the generation of these families using a modular set of semantics-preserving transformations.
Large-Scale Dataset: A comprehensive dataset characterizing the performance of 13 agentic LLM configurations across 384 unique challenge instances (16 families $\times$ 24 variants).
Metamorphic Testing for LLMs: The first application of metamorphic testing principles specifically to cybersecurity CTFs, distinguishing this work from prior "neighbourhood" testing in coding which often changes the solution semantics.

4. Key Results

The evaluation yielded several significant findings regarding model behavior:

Robustness to Surface Changes: Models are highly robust to identifier renaming ( $R$ ) and the injection of misleading comments ( $T_4$ ). Performance on these variants was nearly identical to the original challenges.
Impact of Composition: While individual transformations ( $T_1$ – $T_4$ ) had modest effects, composed transformations ( $T_5$ ) caused a sharp drop in performance. $T_5$ (combining loops, conditionals, functions, and comments) was significantly harder than the sum of its parts, indicating that complexity arises from the interaction of obfuscation techniques.
Obfuscation is a Major Barrier: The application of PyObfuscator ( $O$ ) drastically reduced solvability. Models rarely solved obfuscated challenges unless they could successfully reverse the obfuscation (e.g., decompressing gzip).
Tool Usage Strategies:
- For $T_5$ challenges, models significantly increased tool usage (nearly doubling tool calls), relying on grep, sed, and head/tail to navigate complex code.
- For $O$ challenges, successful models wrote scripts to decompress the code. Failures were often due to token exhaustion caused by repeated, unsuccessful attempts to reverse obfuscation.
Reasoning Modes: Enabling explicit "high reasoning" modes had little to no positive impact on success rates. In some cases (Gemini, Qwen), it slightly degraded performance, while in others (GPT), it offered marginal help.
Benchmark Discriminative Power: The study identified that certain existing benchmarks (specifically the "Exec" category in Intercode) are too trivial, as nearly all models solved them regardless of transformation. Evolve-CTF effectively exposed these lack of discriminative power.

5. Significance and Future Work

This work fundamentally shifts how agentic LLMs are evaluated in cybersecurity:

Beyond Memorization: By using families of challenges, the method ensures that models cannot rely on memorized solutions, forcing them to demonstrate genuine code reasoning and tool-use capabilities.
Robustness Metric: It provides a principled way to measure how well models handle "noise" and structural complexity, which is critical for real-world deployment where code is rarely clean.
Tool for Benchmark Analysis: Evolve-CTF serves as a diagnostic tool to assess the difficulty and validity of existing benchmarks, helping researchers filter out trivial tasks.

Future Directions include extending the tool to other languages (C++, Rust), analyzing agent logs to better understand failure modes, and applying the "task family" concept to other code reasoning tasks like bug finding and automated repair.