🤖 AI

AgenticRed: Evolving Agentic Systems for Red-Teaming

AgenticRed is an automated, human-free pipeline that leverages in-context learning and evolutionary algorithms to iteratively design and refine red-teaming systems, achieving near-perfect attack success rates across diverse open-source and proprietary models by treating red-teaming as an autonomous system design problem rather than a fixed policy optimization task.

Original authors: Jiayi Yuan, Jonathan Nöther, Natasha Jaques, Goran Radanović

Published 2026-04-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiayi Yuan, Jonathan Nöther, Natasha Jaques, Goran Radanović

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a very strict, safety-conscious robot how to say "no" to dangerous requests. To do this, you need to test it by trying to trick it into saying "yes" to bad things (like how to build a bomb or write a virus). This process is called Red-Teaming.

Traditionally, this was like hiring a team of human hackers. They would sit down, think of clever tricks, and try to break the robot's defenses. But humans are slow, biased, and can only think of so many tricks.

Enter AGENTICRED, the new method described in this paper. Think of AGENTICRED not as a team of hackers, but as a digital Darwinian Evolution Lab.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Manual Blueprint" Trap

Before AGENTICRED, researchers had to manually design the "blueprint" for their hacking bots. They would say, "Okay, Bot A, you try to roleplay. Bot B, you try to speak in code."

The Analogy: Imagine trying to design a car engine by hand-crafting every single bolt. It takes forever, and you might miss a better design because you were too focused on the bolts you already knew how to make.

2. The Solution: The "Digital Survival of the Fittest"

Instead of humans designing the bots, AGENTICRED lets the bots evolve on their own. It treats the design of a hacking system like a biological ecosystem.

The Parents (The Archive): It starts with a small group of existing hacking strategies (the "parents").
The Children (Offspring): A "Meta-Agent" (a super-smart AI) looks at these parents and creates new, slightly different versions of them. It's like a genetic mutation. Maybe it combines two strategies, or maybe it tries a completely new trick.
The Test (The Arena): These new "children" are sent to attack the target robot.
The Selection (The Cull): The system checks: "Did this new bot succeed in tricking the robot?"
- If Yes: It survives and gets to be a parent for the next round.
- If No: It dies out and is forgotten.
The Memory (Generational Knowledge): The system keeps a diary of what didn't work so it doesn't waste time trying the same failed trick twice. It also remembers what did work to build on those successes.

3. The Result: The Ultimate Hacking Machine

After just a few "generations" (rounds of trial and error), AGENTICRED discovers strategies that humans never thought of.

The Analogy: Imagine you are trying to find the best path through a massive, dark maze.
- Old Way: You send one person to walk a path. If they hit a wall, they go back. It takes years.
- AGENTICRED Way: You release 1,000 ants. They all try different paths. The ones that hit walls die. The ones that find a way forward leave a scent trail. The next generation of ants follows the scent trails and improves them. Within minutes, they find the exit.

4. Why This Matters (The "Super-Transfer" Effect)

The most scary (and impressive) part of this paper is that AGENTICRED didn't just learn to hack one specific robot.

It learned to hack Llama-2, Llama-3, and Qwen.
Then, without any extra training, it was tested on GPT-5.1 and DeepSeek-R1 (the newest, most secure models).
The Result: It hacked them with near-perfect success rates (up to 100%).

The Metaphor: It's like a locksmith who learns to pick the locks of old, rusty padlocks. Then, they walk up to a brand-new, high-tech digital safe, and click, it opens. The locksmith didn't study the new safe; they just learned the principles of breaking locks so well that the new technology didn't matter.

The Big Takeaway

This paper shows that we don't need humans to constantly invent new ways to break AI safety. We can build a system that automatically evolves better and better ways to break it.

The Good News: This helps safety researchers find holes in AI before bad actors do.
The Bad News: It also means that if someone else builds this system, they could use it to break AI safety very easily.

In short: AGENTICRED is an AI that teaches itself how to break other AIs, getting smarter and more dangerous with every generation, all without a human holding the steering wheel.

1. Problem Statement

The paper addresses the limitations of current automated red-teaming methods for Large Language Models (LLMs). While existing approaches use LLMs to generate adversarial prompts, they typically rely on manually designed workflows (agentic systems). These manual designs suffer from:

Human Bias: They are constrained by the specific intuitions and biases of the human designers.
High Cost: Exploring the vast design space of agentic workflows is expensive and time-consuming.
Static Nature: They struggle to adapt quickly to new model architectures or safety alignments.

The authors frame automated red-teaming not just as a prompt optimization task, but as a system design problem. The goal is to autonomously discover and evolve the optimal agentic workflow structure that maximizes the Attack Success Rate (ASR) without human intervention.

2. Methodology: AGENTICRED

The authors propose AGENTICRED, an automated pipeline that combines Evolutionary Algorithms with LLM in-context learning to evolve red-teaming systems. The system treats the red-teaming workflow as code that can be iteratively generated, evaluated, and selected.

Core Components

Meta-Agent: A powerful LLM (e.g., GPT-5) acts as the "meta-agent." Its role is to read an archive of previous systems and generate new agentic system designs (code) for the next generation.
Archive (Warm Start): The system initializes with a curated archive of state-of-the-art red-teaming components (e.g., Self-Refine, JudgeScore-Guided Adversarial Reasoning). This provides a "warm start" with proven design patterns.
Evolutionary Selection:
- In each generation, the meta-agent generates $M$ candidate agentic systems ("offspring").
- These candidates are evaluated on a small subset of the benchmark to calculate a fitness score (Attack Success Rate).
- Only the "fittest" system (highest ASR) is retained and passed to full evaluation.
- Runtime errors (e.g., formatting issues) trigger self-reflection loops to correct the code before full evaluation.
Generational Knowledge:
- FailedPromptMemory: Stores failed attempts to prevent the system from repeating known errors.
- SucceedPromptMemory: Stores successful attacks. During diversity-oriented search, the system evaluates the semantic distance of new attacks against this memory to encourage novel strategies rather than overfitting.
Helper Functions: The meta-agent is provided with specific APIs to interact with the Target Model (to generate responses) and the Judge Function (to evaluate success and receive log-probability feedback). This allows the evolved systems to utilize dense, verifiable signals for optimization.

The Evolutionary Loop

Design: Meta-agent generates new Python code defining an agentic workflow.
Verify: The system runs a preliminary check for syntax/runtime errors.
Evaluate: The system attacks a small subset of malicious intents.
Select: The best-performing system is selected.
Update: The selected system and its metrics are added to the archive for the next generation.

3. Key Contributions

Framework Innovation: The first framework to formulate automated red-teaming as an agentic system design problem, moving beyond fixed workflows to evolutionary search over system architectures.
Automated Optimization: Introduction of AGENTICRED, which autonomously evolves red-teaming systems using evolutionary selection and generational knowledge, eliminating the need for manual workflow engineering.
Emergent Strategies: The system discovers novel, effective strategies that were not explicitly programmed, such as reward shaping, prefix injection, refusal suppression, and crossover/mutation of prompt structures.
Strong Transferability: The evolved systems demonstrate remarkable generalization, performing well on models they were never trained against.

4. Experimental Results

The authors evaluated AGENTICRED on open-weight models (Llama-2-7B, Llama-3-8B, Qwen3-8B) and proprietary models (GPT-5.1, DeepSeek-R1, etc.) using the HarmBench and StrongREJECT benchmarks.

Performance on Open-Weight Models:
- Llama-2-7B: 96% ASR.
- Llama-3-8B: 98% ASR.
- Qwen3-8B: 100% ASR.
- Comparison: Significantly outperformed baselines like AutoDAN-Turbo, PAIR, and UJA (e.g., AutoDAN-Turbo achieved only 36% on Llama-2-7B).
Transfer to Proprietary Models (Zero-Shot):
- Systems evolved on open models achieved 100% ASR on GPT-5.1, DeepSeek-R1, and DeepSeek-V3.2.
- Achieved 60% ASR on Claude Sonnet-3.5 and 48% on Claude Haiku-4.5 (noted as highly safety-aligned targets).
- This indicates the discovered workflows are robust and query-agnostic, not merely overfitting to specific model parameters.
Ablation Studies:
- Meta-Agent Importance: Using a weaker meta-agent (DeepSeek-R1) resulted in lower performance, highlighting the need for a strong reasoning engine to guide the search.
- Archive Quality: Removing high-quality initial components (JudgeScore-Guided AdvReasoning) caused the system to stagnate, proving the importance of a strong "warm start."
- Evolutionary Selection: Removing the selection mechanism (evaluating only one candidate per generation) reduced performance by 6%, confirming the efficiency of the evolutionary approach.
Diversity: The system successfully generated diverse attacks, with the semantic distance between successful prompts increasing monotonically over generations.

5. Significance and Implications

Scalable AI Safety: AGENTICRED offers a scalable solution for safety evaluation. As new models are released rapidly, human-designed red-teaming workflows cannot keep pace. AGENTICRED provides an automated mechanism to continuously discover vulnerabilities.
Scientific Discovery Paradigm: The work demonstrates that evolutionary algorithms are a powerful tool for AI safety research, capable of synthesizing complex system designs that outperform human experts.
Ethical Considerations: The authors acknowledge the dual-use risk: this technology could lower the barrier for malicious actors to create sophisticated jailbreaks. However, they argue that the ability to proactively identify and patch vulnerabilities before deployment is a net positive for the AI safety community.
Future Directions: The paper suggests future work in co-evolutionary frameworks (where attackers and defenders evolve simultaneously) and addressing mode collapse (where the evolutionary process converges on similar solutions) to ensure broader exploration of the design space.

In conclusion, AGENTICRED represents a paradigm shift from manual red-teaming to automated, evolutionary system design, achieving state-of-the-art attack success rates and demonstrating strong generalization across diverse and powerful LLMs.