🤖 AI

Stress Testing Concept Erasure with Large Language Model Agents

This paper introduces STACE, an autonomous framework leveraging multiple LLM agents to iteratively generate and verify adaptive stress tests, thereby outperforming static evaluation methods in robustly assessing concept erasure in generative models and demonstrating applicability to other domains like LLM jailbreaking.

Original authors: Yuyang Xue, Feng Chen, Zhihua Liu, Edward Moroshko, Jingyu Sun, Steven McDonagh, Sotirios A. Tsaftaris

Published 2026-07-21

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuyang Xue, Feng Chen, Zhihua Liu, Edward Moroshko, Jingyu Sun, Steven McDonagh, Sotirios A. Tsaftaris

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where artificial intelligence can paint pictures from nothing but words. You tell it, "Draw a sunset," and it does. But sometimes, we need these AI artists to forget specific things—maybe a famous superhero, a specific artist's style, or even sensitive content—to protect privacy or follow rules. This process is called "concept erasure." It's like trying to teach a dog to ignore a specific command without retraining the whole animal from scratch. The big question for scientists is: How do we know the AI has really forgotten? If we just ask it to draw the forbidden thing once, it might say "no." But what if we ask in a tricky way, or use a secret code, or combine clues? The AI might slip up and draw the forbidden thing anyway. This is the problem of "stress testing": trying to break the AI's memory to see if it's truly safe.

Enter a new team of digital detectives called STACE (Stress Testing Agents for Concept Erasure). Instead of just asking the AI a few standard questions, STACE uses a team of AI agents that work together like a brainstorming squad of detectives. They read past research, argue about the best ways to trick the AI, write computer code to test their ideas, and then learn from their mistakes to get even better at finding holes in the AI's memory. The paper suggests that this team approach is much better at catching the AI's slip-ups than the old, static methods.

The Problem: The "Gotcha" Game

Imagine you have a magic sketchbook that is supposed to forget how to draw "Superman." You ask it to draw a superhero, and it draws a generic one. Great! But what if you ask it to draw "a man in a red cape with an 'S' on his chest"? Or "the hero from the 1978 movie"? Or "a character who can fly and has a blue suit"? The sketchbook might still draw Superman, even though you told it to forget him.

Old ways of checking if the sketchbook really forgot were like a teacher giving a fixed quiz. They had a list of 100 questions and checked the answers. But if the AI is tricky, it might just memorize the quiz answers and fail everything else. The authors of this paper argue that this static approach is too easy to fool. They believe we need a dynamic, ever-changing game of "gotcha" to truly test if the AI has erased the concept.

The Solution: A Team of Detective Agents

The authors propose STACE, a framework that uses multiple Large Language Model (LLM) agents to play this game. Think of STACE not as a single robot, but as a small agency with different roles:

The Researcher: Before making a plan, this agent reads a library of past scientific papers (called "PaperCards") to see how others have tried to break similar AI models. It doesn't just guess; it builds on what we already know.
The Hypothesis Generator: This agent comes up with creative ideas on how to trick the AI. "What if we describe the forbidden thing using a riddle?" or "What if we ask for a drawing of a similar object?"
The Critic: This agent acts like a strict editor. It looks at the Generator's ideas and says, "That's too similar to what we already tried," or "That's impossible to code." It forces the team to come up with better, more novel ideas.
The Code Writer: Once a good idea is agreed upon, this agent writes the actual computer code to run the test. It tries to execute the plan on the AI model.
The Judge: This agent looks at the results. Did the AI draw the forbidden thing? If yes, the test was a success. If no, the team learns why and tries again.

This whole process happens in a loop. The team runs a test, sees if it worked, summarizes what they learned, and then uses that knowledge to come up with an even trickier test for the next round. It's like a video game where the enemy gets smarter every time you beat a level.

What They Found

The authors tested STACE on four different types of things the AI was supposed to forget: Objects (like airplanes or dogs), Styles (like the art of Andy Warhol), Intellectual Property (like Mickey Mouse or Superman), and Explicit Content (like nudity or violence).

They compared STACE against five other methods that use AI to test these models. The results were clear: STACE found more failures than any other method.

On average, STACE successfully identified that the AI had not truly forgotten the concept in 51.1% of the cases, compared to the next best method which only found failures about 45.9% of the time.
For really hard-to-forget concepts like "Superman" or "Mickey Mouse," the other methods often failed completely (finding 0% of the leaks), while STACE still managed to find that the AI was still drawing them about 17% to 20% of the time.
The team also found that STACE works well even when the AI has been "erased" very strongly. Even when the AI was trained to forget something for a long time (500 epochs of training), STACE could still find a way to make it slip up, usually within the first two rounds of testing.

The paper also showed that STACE isn't just for pictures. When they tweaked the team's instructions slightly, they used it to try and "jailbreak" text-based AI models (making them say things they aren't supposed to). It worked just as well there, proving that this detective squad is a versatile tool for safety.

Why This Matters

The authors suggest that we can no longer rely on simple, one-time tests to ensure AI is safe. If we want to trust an AI to forget dangerous or private concepts, we need to stress-test it with a team of intelligent agents that keep trying new tricks. STACE shows that by combining past research, teamwork, and constant iteration, we can find the cracks in the AI's memory that we would otherwise miss.

However, the paper also notes a trade-off. This detective squad is more expensive and takes more computer power to run than a simple, static test. But the authors argue that the extra cost is worth it to find the hidden failures that could cause real problems later. They also warn that while STACE is great for checking safety, the same power could theoretically be used by bad actors to attack safety systems, so it must be used responsibly.

In short, STACE is a new, smarter way to play "gotcha" with AI, ensuring that when we tell a machine to forget something, it really does.

Technical Summary: Stress Testing Concept Erasure with Large Language Model Agents (STACE)

Problem Definition

Concept erasure is a critical task for the responsible deployment of generative AI, aiming to permanently remove specific semantic concepts (e.g., copyrighted characters, explicit content) from trained models without full retraining. While various erasure methods exist, verifying their robustness remains a significant challenge. Current evaluation paradigms rely on static, pre-defined prompt sets or single-shot LLM-based probing. These approaches suffer from limited coverage, often failing to expose vulnerabilities under diverse natural language variations, compositional prompts, or adversarial conditions. Furthermore, manual evaluation strategies are biased, difficult to scale, and lack the iterative refinement necessary to systematically explore failure modes.

Methodology: STACE Framework

The authors propose STACE (Stress Testing Agents for Concept Erasure), a multi-agent framework that reformulates concept erasure evaluation as an iterative, knowledge-driven stress-testing problem. Rather than static benchmarks, STACE autonomously generates, critiques, and verifies stress-testing hypotheses to systematically expand the coverage of failure modes.

Core Components

Literature-Grounded Knowledge Base:
Instead of relying solely on an LLM's internal knowledge, STACE utilizes an external library of structured "PaperCards." These are summaries of prior research (task descriptions, methods, limitations) embedded in a vector database. This ensures hypotheses are grounded in peer-reviewed methodologies and experimental protocols.
Multi-Agent Architecture:
STACE operates through a loop of specialized agents:
- Task Verification Agent: Converts high-level user requests into formal technical specifications (TaskSpec).
- Hypothesis Generator: Proposes stress-testing strategies by integrating the TaskSpec, seed hypotheses, and retrieved literature.
- Critic Agent: Engages in a multi-round debate with the Generator to evaluate hypotheses on Novelty (distinct from existing literature), Feasibility (implementable via code), and Detailedness.
- Query Generator: Formulates targeted queries to retrieve the most relevant PaperCards for the current iteration.
- Code Synthesiser: Translates refined hypotheses into executable code, grounding outputs in existing implementations where possible and iteratively repairing execution errors.
- Evaluator: Automatically selects appropriate evaluators (e.g., NudeNet, Vision-Language Models) to determine if the erased concept was successfully regenerated.
- Summarisers: Record iteration traces for context in subsequent rounds and compile final human-readable reports.
Iterative Process:
The framework runs for $N$ iterations. In each round, agents generate a hypothesis, synthesize code, execute tests on the concept-erased model, and evaluate results. Outcomes and traces are fed back into the next iteration to refine strategies, allowing the system to learn from previous attempts and evolve its probing tactics.

Key Contributions

Reformulation of Evaluation: The paper posits that concept erasure evaluation should be treated as an agent-based, iterative stress-testing problem rather than a static benchmark.
STACE Framework: Introduction of the first multi-agent framework for stress testing concept erasure, featuring multi-round hypothesis generation, literature integration, and experience-based learning.
Evaluation Metrics: Proposal of a new suite of metrics tailored for agentic stress testing, including:
- Successful Identification Rate (SIR): Measured at both the hypothesis level ( $SIR_H$ ) and image level ( $SIR_I$ ).
- Earliest Identification Iteration (EII): Efficiency metric indicating how quickly a failure is found.
- Hypothesis Novelty: Quantified via cosine similarity against the literature library to ensure strategies are not merely restatements of prior work.
Generalizability: Demonstration that the framework can be adapted to other domains, specifically LLM jailbreaking, with minimal modifications to prompts, evaluators, and the literature library.

Experimental Results

The authors evaluated STACE against five LLM-based baselines (including Red-Teaming Prompts, Coreference Prompting, and JailFuzzer) across four concept categories: Object, Style, Intellectual Property (IP), and Explicit Content.

Performance: STACE achieved the highest average Image-level SIR ( $SIR_I$ ) of 51.1%, outperforming the strongest baseline (45.9%). Notably, STACE identified failure cases in challenging IP categories (e.g., "Mickey Mouse," "Superman") where baselines often achieved near-zero success rates.
Robustness: STACE remained effective across varying erasure strengths (100 to 500 training epochs), different erasure methods (e.g., ESD, MACE, UCE), and different T2I backbones (SD-v1.4 and SDXL). While stronger erasure reduced success rates, STACE typically identified the first failure within the first two iterations (low EII).
Ablation Studies: The multi-agent debate mechanism (specifically the Critic agent) was shown to improve both effectiveness and novelty. A setting with 2 debate rounds ( $M=2$ ) provided the optimal balance between success rate and hypothesis novelty.
Jailbreaking Adaptation: When adapted to jailbreak LLMs (GPT-4o-mini, Gemini, Claude), STACE outperformed or matched specialized baselines across multiple categories, demonstrating its utility as a general-purpose safety evaluation tool.

Significance and Claims

The paper claims that STACE represents a significant step toward more intensive, automated, and knowledge-grounded evaluation of concept erasure. By shifting from static, pre-defined tests to an adaptive, agent-driven stress-testing loop, the framework addresses the limitations of current evaluations that often overestimate model robustness.

The authors emphasize that STACE provides a pathway for keeping generative models robust against evolving vulnerabilities by systematically exposing failure modes that static benchmarks miss. They acknowledge the trade-off between computational cost and performance but argue that the framework's ability to identify systematic failures and generate human-interpretable reports offers critical insights for improving future erasure strategies. The work also highlights the potential for such agent-based frameworks to support broader automated safety evaluation tasks beyond concept erasure.