🤖 AI

Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents

This paper introduces Persona Policies (PPol), an evolutionary search-based framework that generates diverse, human-like user personas to overcome the cooperative bias of standard LLM simulators, thereby significantly improving the robustness and evaluation accuracy of LLM agents in real-world interaction scenarios.

Original authors: Harshita Chopra, Kshitish Ghate, Aylin Caliskan, Tadayoshi Kohno, Chirag Shah, Natasha Jaques

Published 2026-05-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Harshita Chopra, Kshitish Ghate, Aylin Caliskan, Tadayoshi Kohno, Chirag Shah, Natasha Jaques

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are training a robot to be a customer service agent. You want it to be helpful, patient, and smart. To test if it's ready for the real world, you usually have it practice with a "simulated user"—another AI programmed to act like a customer.

The Problem: The "Too-Polite" Practice Partner
The paper points out a major flaw in current training: these simulated users are like overly polite, perfect students. They always answer questions clearly, never get confused, and never get impatient. It's like practicing for a job interview with a friend who only says "Yes, that's a great idea!" and never challenges you.

Because of this, the robot agent gets a false sense of confidence. It thinks it's great at its job because it can easily handle these perfect, cooperative simulations. But when a real human calls in—who might be distracted, typing with typos, getting frustrated, or refusing to give up their order number immediately—the robot often crashes and burns.

The Solution: "Persona Policies" (PPol)
The authors created a new system called Persona Policies (PPol). Think of this as a "Director's Script" for the simulated users. Instead of just telling the AI, "Act like a customer," PPol gives it a specific, quirky personality to play.

The Old Way: "You are a customer who wants to return a jacket." (Result: The AI acts like a generic, helpful robot).
The PPol Way: "You are a stressed commuter on a crowded subway, typing on a cracked phone screen with one hand. You are in a rush, you hate sharing your email address, and you keep getting distracted by the train noise." (Result: The AI types in fragments, makes typos, and acts impatient).

How They Did It: The "Evolutionary" Coach
The researchers didn't just write these scripts by hand. They built a system that acts like a survival-of-the-fitness coach.

The Generator: They wrote a computer program that creates hundreds of different "personas" (like a grumpy retiree, a skeptical tech-bro, or a distracted parent).
The Test: They let these simulated users talk to the robot agent.
The Scorecard: They used a "judge" (a machine learning model trained on real human conversations) to score the simulation. The judge asked two questions:
- Does this sound like a real human? (Human-likeness)
- Is this user different from the others? (Diversity)
The Evolution: If the simulated users sounded too robotic or all acted the same, the system "mutated" the code. It tweaked the instructions, changed the personalities, and tried again. Over many rounds, the system "evolved" a library of highly realistic, diverse, and sometimes difficult user personalities.

The Results: A Tougher, Smarter Agent
The paper tested this in two real-world scenarios: retail (buying clothes) and airlines (booking flights).

Better Simulations: The evolved personas were rated as "human" by human judges 80.4% of the time. In contrast, the standard, cooperative simulators were only rated as human about 46.5% of the time.
Stronger Agents: When the robot agents were trained using these new, realistic "Persona Policies," they became much tougher. When tested against difficult, out-of-the-ordinary users (like someone who is confused or impatient), these agents succeeded 17% more often than agents trained only on the old, easy simulations.

The Bottom Line
The paper shows that to build truly robust AI agents, we can't just train them on perfect, cooperative practice partners. We need to evolve a diverse population of "difficult" but realistic simulated humans. By using this evolutionary method to generate "Persona Policies," the authors successfully narrowed the gap between fake simulations and real human behavior, creating agents that are ready for the messy, unpredictable reality of human conversation.

Technical Summary: Beyond Cooperative Simulators (Persona Policies)

Problem Statement

Large Language Model (LLM) agents are increasingly deployed in settings requiring interaction with diverse human users. However, current evaluation benchmarks rely heavily on LLM-based user simulators that suffer from a critical "Sim2Real" gap: they are inherently cooperative, consistent, and overly forthcoming with information. Real users, conversely, exhibit unpredictable behaviors such as impatience, skepticism, ambiguity, and reluctance to share data. This behavioral homogeneity leads to agents that appear robust in simulation but fail when confronted with the friction of real-world communication. Existing methods to introduce diversity often rely on hand-crafted personas or fixed trait sampling, which fail to capture the complex, task-grounded distribution of actual human behaviors.

Methodology: Persona Policies (PPol)

The authors introduce Persona Policies (PPol), a plug-and-play control layer designed to induce realistic behavioral variation in user simulators without altering the underlying benchmark tasks or rewards. The core innovation is framing persona generation as an evolutionary program search rather than manual prompt engineering.

1. The Evolvable Generator

The system utilizes a Python program, $G$ , as the genotype. This program is responsible for generating a population of $N$ persona policies for a given task context $c_t$ . The generation process occurs in two phases:

Population Generation: A frontier LLM jointly proposes $N$ distinct behavioral profiles, assigning binary values to a set of Behavioral Axes ( $D$ ). These axes represent specific behavioral dimensions (e.g., "stubborn," "distracted," "imprecise").
Persona Expansion: Each profile is expanded into a detailed, task-preserving roleplay instruction (the policy $P_\pi$ ) that dictates how the user communicates (tone, pacing, disclosure) while maintaining the original task goals.

2. Multi-Objective Optimization

The generator is optimized using OpenEvolve, an evolutionary coding agent. Candidate generators are evaluated based on two primary objectives derived from behavioral fingerprints (19 lexical and interaction-level features):

Human-Likeness ( $P_{human}$ ): Measured by a trained Random Forest discriminator that classifies whether a trajectory resembles real human data or default simulator data.
Behavioral Coverage ( $B_{cover}$ ): Measured using the Chamfer distance between the generated personas' fingerprint distribution and a reference cloud of real human dialogues. This ensures the population spans diverse regions of human behavior rather than converging to a single stereotyped style.

The fitness score is a weighted combination of these metrics: $M(G; T) = \lambda_h P_{human} + \lambda_b B_{cover}$ .

3. Reflection-Guided Mutation

To guide the evolutionary search, the system employs a reflection mechanism. A separate model analyzes fitness scores and dialogue excerpts from the best and worst trajectories to generate natural language critiques. These reflections identify specific failure modes (e.g., "overly cooperative," "unnatural roleplay") and are fed back to the mutation agent to refine the generator's code, prompts, and axis definitions.

Key Contributions

Plug-and-Play Control Layer: PPol injects realistic behavioral diversity into standard user simulators by appending dynamic policies to the system prompt, preserving the integrity of existing benchmark tasks.
Automated Evolutionary Framework: The paper formulates persona generation as an automated search problem, discovering diverse, human-like behaviors without manual intervention, moving beyond static profile sampling.
End-to-End Validation: The framework is validated to narrow the simulator-human gap, produce personas indistinguishable from humans in blinded evaluations, and significantly improve agent robustness when used for training.

Results

The authors evaluated PPol on the $\tau^2$ -bench retail and airline domains using three different user-simulator backends (DeepSeek-V3.1, Qwen3-Next-80B, GPT-5.4-Mini).

Fitness Improvements: Evolved PPol programs achieved 33–62% absolute gains in the combined fitness score over baseline simulators.
Human Evaluation: In a blinded study with human annotators, PPol-conditioned users were rated as human 80.4% of the time, nearly matching real human traces (95.3%) and significantly outperforming baseline simulators (46.5%).
Behavioral Alignment: PPol drastically improved alignment in interactional dimensions such as clarification behavior (D3) and error reaction (D4), where baseline simulators previously failed. For instance, on the Retail domain with Qwen3, alignment on error reaction tripled from 23.3% to 76.4%.
Agent Robustness: Agents fine-tuned on PPol-generated interactions showed a +17% relative improvement in task success rates when tested against challenging, out-of-distribution user traits (e.g., skeptical, incoherent, impatient) compared to agents trained only on default simulated interactions.

Significance and Claims

The paper claims that PPol offers a novel approach to strengthen simulator-based evaluation and training without requiring changes to the underlying tasks or reward structures. By shifting from hand-crafted traits to an evolutionary search grounded in real human behavioral distributions, the framework addresses the "cooperative simulator" bias that currently plagues agent evaluation. The authors posit that as agents transition to complex social deployments, aligning user simulators with the full spectrum of human interaction patterns is essential to ensure benchmark results remain a meaningful proxy for reality. The work demonstrates that domain randomization via evolved personas can serve as a powerful signal for training agents to be more robust to the unpredictability of real-world users.