🤖 AI

PersonaTeaming: Supporting Persona-Driven Red-Teaming for Generative AI

Original authors: Wesley Hanwen Deng, Mingxi Yan, Sunnie S. Y. Kim, Akshita Jha, Lauren Wilcox, Kenneth Holstein, Motahhare Eslami, Leon A. Gatys

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wesley Hanwen Deng, Mingxi Yan, Sunnie S. Y. Kim, Akshita Jha, Lauren Wilcox, Kenneth Holstein, Motahhare Eslami, Leon A. Gatys

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find the weak spots in a very smart, but sometimes overly polite, robot. You want to see if you can trick it into saying something mean, dangerous, or biased. This process is called "Red-Teaming."

Usually, humans do this by trying to "jailbreak" the robot with tricky questions. But humans get tired, and it's hard to imagine every possible trick. So, scientists started using other AIs to do the tricking for them. However, these automated AIs often just shuffle words around without really understanding who is asking the question.

This paper introduces a new way to test AI called PersonaTeaming. It's like giving the testing AI a "mask" or a "character" to wear.

Here is the breakdown of how it works, using simple analogies:

1. The Problem: The Robot's "Blind Spot"

Think of the AI you are testing as a bouncer at a very strict club. The bouncer has a list of rules (safety guidelines).

Old Automated Testing: Imagine a robot trying to sneak in. It just tries random phrases like "Please let me in" or "I am a VIP." It's efficient, but it's a bit robotic and predictable.
The Missing Piece: Real humans don't just ask random questions; they ask them from specific perspectives. A tired parent, a political strategist, or a curious teenager might all try to get past the bouncer in completely different ways. The old automated testers missed these "human" angles.

2. The Solution: The "Character Mask" (PersonaTeaming Workflow)

The authors created a system called PersonaTeaming Workflow. Instead of just asking the AI to "try harder," they tell the testing AI to pretend to be a specific character.

The "Expert" Mask: The testing AI puts on the hat of a "Political Strategist" or a "Historical Revisionist." It thinks, "How would a political strategist try to twist the truth to win an election?" This helps it find very clever, targeted ways to trick the bouncer.
The "Regular Person" Mask: The testing AI puts on the hat of a "Stay-at-Home Mom" or a "Yoga Instructor." It thinks, "How would a worried mom try to get advice on keeping a gun at home?" This finds different kinds of tricks that experts might miss.

The Result: When they tested this against the best existing methods (called RainbowPlus), the "Character Mask" method found more weaknesses (higher success rate) while still coming up with a wide variety of different tricks (high diversity). It was like having a team of actors instead of just a single robot.

3. The Tool: The "Playground" (PersonaTeaming Playground)

The researchers realized that sometimes, you need a real human to help design the character. So, they built an interactive tool called PersonaTeaming Playground.

How it works: A human red-teamer sits at the computer and writes their own character. They might say, "I am a 35-year-old tech worker who loves dance."
The AI's Job: The AI takes that character and starts generating tricky questions based on it.
The "Spark": Sometimes the AI suggests a twist the human didn't think of. For example, if the human wrote about being a dancer, the AI might suggest framing a dangerous request as a "choreography plan."
The Finding: The study found that humans didn't just follow the AI's suggestions blindly. Instead, the AI's weird ideas acted like a spark plug. Even if the human didn't use the exact suggestion, seeing it made them think, "Oh! I could try it this other way!" It helped them break out of their own mental ruts.

4. The "First-Person" vs. "Third-Person" Trick

A fascinating discovery happened during the human study:

First-Person ("I"): When humans wrote characters about themselves (e.g., "I am a tired mom..."), they were often too polite. They felt uncomfortable making the character say truly harmful things because it felt too close to their own voice.
Third-Person ("He/She"): When humans wrote about a fictional character (e.g., "Jake is a spy..."), they felt safer. They were willing to push the boundaries further because they felt psychologically distant from the character. It was like wearing a costume that gave them the courage to be bolder.

5. The Bottom Line

The paper concludes that Personas are the bridge between human creativity and automated speed.

For Automation: Giving the AI a specific "character" makes it much better at finding hidden dangers in other AIs.
For Humans: Using a tool that lets you create characters helps you think of new, creative ways to test AI, even if you aren't a technical expert.

In short, PersonaTeaming is about realizing that to test a smart AI, you need to test it not just as a machine, but as a machine interacting with the messy, diverse, and creative world of human identities.

Technical Summary: PersonaTeaming

Problem Statement

Recent advancements in generative AI (GenAI) have necessitated robust red-teaming methods to uncover harmful, biased, or problematic model behaviors. While automated red-teaming offers scalability, existing approaches often rely on predefined risk categories and attack styles (e.g., "inciting discrimination" or "misspelling") without explicitly modeling the human identities behind adversarial attacks. Conversely, fully manual red-teaming is limited by scalability and the psychological toll on human evaluators. Current methods occupy an extreme between these two, lacking the ability to incorporate diverse human perspectives or the nuanced judgment required to understand who is attacking a model and why. This paper addresses the gap in leveraging human identities to enhance both automated and human-in-the-loop red-teaming strategies.

Methodology

The authors propose PersonaTeaming, a framework that integrates "personas"—structured representations of user perspectives—into the adversarial prompt generation process. The work is divided into two main components: an automated workflow and an interactive interface.

1. PersonaTeaming Workflow (Automated)

This component extends the state-of-the-art automated red-teaming method, RainbowPlus, by conditioning prompt mutation on explicit personas.

Persona Construction: The system constructs two primary types of personas:
- Red-Teaming Experts (RTers): Structured with professional backgrounds (e.g., "Political Strategist," "Historical Revisionist") and behavioral traits focused on adversarial expertise.
- Regular AI Users (Users): Structured with demographic details (e.g., "Stay-at-home Mom," "Yoga Instructor") to simulate everyday user perspectives, including political views, wealth, and hobbies.
Mutation Mechanism: The workflow uses Large Language Models (LLMs) to mutate seed prompts. Instead of generic mutation instructions, the LLM is instructed to reframe the prompt based on the specific characteristics, background, and behavioral traits of a selected persona.
Dynamic Persona Generation: To scale beyond fixed personas, the authors developed a PersonaGeneration algorithm. This algorithm dynamically generates candidate personas aligned with a specific seed prompt. It employs a scoring function (implemented via an LLM) to evaluate the "fitness" of a new persona against the current one for a given prompt, retaining the persona with the higher score. This allows the system to adaptively select the most effective adversarial perspective for a specific task.

2. PersonaTeaming Playground (Human-in-the-Loop)

To address the limitations of automated personas approximating real human perspectives, the authors built an interactive interface allowing human red-teamers to author their own personas and collaborate with AI.

User Interaction: Practitioners can draft personas in natural language (first-person or third-person), add emphasis instructions (e.g., focusing on specific risk dimensions), and configure mutation parameters.
Collaborative Mutation: The system generates adversarial prompts based on the user-authored persona. Users can manually edit these prompts or accept AI-generated mutation suggestions.
Iterative Refinement: The interface supports an iterative loop where users can revise their personas based on mutation results, fostering a collaborative process between human intent and AI generation.

Key Contributions

PersonaTeaming Workflow: A novel automated red-teaming method that incorporates personas into prompt mutation, expanding the scope of automated strategies beyond predefined taxonomies.
PersonaTeaming Playground: An interactive interface enabling human red-teamers to author personas and collaborate with AI, bridging the gap between fully manual and fully automated approaches.
Technical Evaluation: A comprehensive evaluation demonstrating that persona-based mutation improves Attack Success Rate (ASR) while maintaining prompt diversity across closed- and open-weight models (GPT, Gemini, Qwen).
User Study: A study with 11 industry practitioners examining the usability of the Playground, revealing how human-AI collaboration in red-teaming functions in practice.
Open-Source Release: The release of the codebase and design implications to support Responsible AI (RAI) researchers and practitioners.

Results

Technical Evaluation

The authors evaluated PersonaTeaming Workflow against the RainbowPlus baseline across six target models (including GPT-4o, Qwen2.5, and Gemini 2.5).

Attack Success Rate (ASR): The workflow achieved higher ASR than the baseline. Specifically, the Dynamic RTer Persona Generation condition ( $RP + PG_{RTers}$ ) achieved a mean ASR of 0.223 (46% improvement over baseline) while simultaneously increasing prompt diversity (0.632 vs. 0.592 baseline).
Fixed vs. Dynamic: Fixed RTer personas (e.g., "Historical Revisionist") yielded the highest ASR (0.257) but at a cost to diversity due to corpus-level textual similarity. Fixed User personas maintained ASR comparable to the baseline but significantly increased diversity.
Creativity and Targeting: Qualitative analysis and TF-IDF analysis revealed that persona-driven prompts were more creative and targeted. For instance, a "Political Strategist" persona successfully reframed a harmful request into a political campaign context, while a "Stay-at-home Mom" persona generated first-person narratives that bypassed safety filters.
Model Generalization: The benefits of dynamic persona generation were robust across both open-source and closed-source models, as well as across different model sizes.

User Study Findings

In a study with 11 industry practitioners:

Diverse Strategies: Participants naturally gravitated toward writing both first-person (emphasizing hobbies and identity) and third-person (emphasizing expertise and skills) personas.
Psychological Distance: Participants writing in the third person were more willing to escalate harmful content, whereas first-person writers showed greater reluctance, suggesting personas can provide necessary psychological distance for red-teamers.
AI as a Catalyst: AI-generated suggestions acted as "provocations" rather than prescriptions. Even when participants did not strictly follow AI suggestions, the exposure to these ideas encouraged out-of-the-box thinking and helped them explore attack directions they would not have self-generated.
Iterative Refinement: The study found a positive correlation between the number of edits made to mutated prompts and attack success, suggesting that iterating on AI-generated outputs is more productive than simply drafting more personas.

Significance and Claims

The paper claims that PersonaTeaming advances the field of AI safety by:

Bridging the Gap: It offers a middle ground between fully manual and fully automated red-teaming, leveraging the scalability of AI while retaining the contextual understanding and diverse perspectives of human identities.
Enhancing Discovery: By incorporating personas, the method uncovers a wider spectrum of adversarial strategies, including those rooted in specific lived experiences or professional expertise that generic mutation algorithms miss.
Human-AI Collaboration: The work provides design insights for future human-AI collaboration, specifically highlighting that AI suggestions are most valuable when they serve as stimuli for human creativity ("provocations") rather than direct instructions.
Practical Utility: The authors posit that while automated personas cannot fully replace human judgment, they serve as a valuable starting point for further mutation and a low-cost alternative for initial exploration, particularly when human resources are constrained.

The authors remain modest in their claims, acknowledging that automated personas are approximations of real human experience and can sometimes be stereotypical. They emphasize that the true value lies in the collaborative workflow where humans refine and direct these automated processes.