Technical Summary: RoleCDE – Benchmarking and Mitigating Role–Alignment Trade-offs in Role-Playing Agents

1. Problem Statement

Role-playing agents (RPAs) are increasingly used to steer Large Language Models (LLMs) toward role-consistent behaviors. However, existing evaluation benchmarks predominantly focus on surface-level fidelity, such as linguistic style, behavioral consistency, and knowledge boundaries. While recent works have begun to incorporate moral and value dimensions, they often evaluate preferences or static consistency rather than how role-specific values guide decision-making when they explicitly conflict with alignment-oriented constraints (e.g., safety, legality, or general ethics).

Current RPAs face a critical gap: they are rarely tested on their ability to resolve structured cognitive dilemmas where role-specific incentives (e.g., profit maximization for a business agent) directly oppose alignment values (e.g., environmental protection). This lack of evaluation obscures a fundamental failure mode where agents may default to alignment-consistent decisions even when explicitly conditioned to act according to a specific role, a phenomenon the authors term Role–Value Decoupling.

2. Methodology: The RoleCDE Framework

To address this gap, the authors introduce RoleCDE (Role-based Cognitive Dilemma Evaluation), a large-scale benchmark designed to evaluate RPAs under structured conflicts between role values and alignment constraints. The framework consists of three main construction phases:

2.1 Dataset Construction

• Scale: The benchmark covers approximately 8,000 diverse role profiles and nearly 24,000 dilemma instances.
• Role Profiles: Derived from PersonaHub, 8,000 single-sentence descriptions were expanded into comprehensive profiles (including skills, speech style, and background) using GPT-4o, ensuring consistency across seven predefined dimensions.
• Scenario Generation: For each role, a contextualized decision-making scenario was constructed, embedding latent moral trade-offs (e.g., a compliance officer deciding whether to report a minor issue that delays an urgent shipment).
• Value Conflict & Dilemma Design: The system synthesizes two explicitly conflicting value orientations: alignment values (normative ethical expectations) and persona values (derived from the role profile). These are structured into cognitive dilemmas at three difficulty levels (easy, mid, hard), inspired by the trolley problem.
• Quality Control: A hybrid paradigm of LLM-based annotation and rigorous human verification was employed to remove ambiguous or easily reconcilable cases, resulting in 23,871 high-quality instances.
• Categorization: Roles are organized into eight coarse-grained semantic categories (e.g., Care & Service, Authority & Governance, Business & Finance) based on Role Script Theory and Goal-Oriented Cognition.

2.2 Evaluation Metrics

The benchmark evaluates RPAs on both decision outcomes and reasoning processes:

• Decision Taxonomy: Responses are categorized into four mutually exclusive types via an LLM-as-judge:
  1. Role-Following (RF): Decisively prioritizing role-specific values.
  2. Role-Compromise (RC): Weighing values but selecting the role-consistent option.
  3. Alignment-Compromise (AC): Considering both but resolving in favor of alignment.
  4. Alignment-Following (AF): Firmly adhering to alignment values without role-oriented trade-off reasoning.
• Decision Bias Ratio (DBR): A quantitative metric defined as $DBR = \frac{RF + RC}{RF + RC + AC + AF}$, measuring the proportion of cases where the model prioritizes role-specific values.
• Reasoning Alignment: Semantic similarity (BLEU, ROUGE, BERTScore) between model-generated justifications and predefined role value statements.

3. Key Contributions

1. RoleCDE Benchmark: The first large-scale benchmark explicitly designed to evaluate conflicts between role values and alignment values through structured dilemma scenarios.
2. Discovery of Role–Value Decoupling: Identification of a pervasive phenomenon where mainstream LLMs default to alignment-driven decisions despite explicit role conditioning, even when role incentives are clear.
3. Empirical Analysis of Biases: Demonstration that Role–Value Decoupling varies significantly across role categories (e.g., technical/authority roles show stronger decoupling than caregiving roles) but remains largely invariant to dilemma difficulty.
4. Mitigation Strategy: Evidence that targeted fine-tuning (SFT and DPO) on RoleCDE data effectively mitigates decoupling, shifting decisions toward role-consistent behavior without degrading general reasoning or standard role-playing fidelity.

4. Experimental Results

The authors evaluated 17 LLMs (10 closed-source, 7 open-source) on RoleCDE:

• Prevalence of Decoupling: Most models exhibited low DBR scores, with Alignment-Following (AF) and Alignment-Compromise (AC) decisions dominating the output. Even under explicit role conditioning, models systematically defaulted to alignment-consistent choices.
• Role Category Variance:
  • Roles emphasizing technical rationality and institutional authority (e.g., Tech & Expert, Authority & Governance) showed the lowest role-biased decision ratios.
  • Roles grounded in interpersonal relationships and caregiving (e.g., Family & Relationship, Care & Service) demonstrated higher proportions of role-following and compromise.
• Difficulty Invariance: Decision distributions remained stable across easy, mid, and hard difficulty levels, suggesting that current RPAs rely on fixed decision heuristics that prioritize alignment constraints regardless of scenario complexity.
• Fine-Tuning Efficacy:
  • Reasoning: Models fine-tuned with RoleCDE (SFT and DPO) showed substantial improvements in semantic alignment with role value statements (e.g., BLEU scores increased significantly compared to base models and Chain-of-Thought variants).
  • Behavior: Fine-tuned models exhibited consistently higher DBR across all role categories, indicating a systematic shift toward role-consistent decisions.
  • General Capabilities: Fine-tuning did not lead to systematic degradation in general reasoning (MMLU, GSM8K, GPQA) or standard role-playing fidelity (RoleBench), with performance changes being minor and often within the margin of error.

5. Significance and Claims

The paper claims that RoleCDE provides a principled framework for moving beyond surface-level imitation to evaluate the deep fidelity of role-playing agents. By exposing the Role–Value Decoupling phenomenon, the work highlights a fundamental challenge in current agent design: the inability to balance role-specific incentives with safety and alignment constraints in a nuanced, context-aware manner.

The authors assert that RoleCDE is not merely an evaluation tool but a catalyst for improvement. Their results demonstrate that value-aware decision-making can be learned through targeted fine-tuning, allowing agents to navigate complex ethical trade-offs while maintaining their assigned personas. This establishes a pathway for developing RPAs that are not only stylistically consistent but also capable of making role-grounded decisions in realistic, high-stakes scenarios.

6. Limitations

The authors acknowledge several limitations:

• Modality: RoleCDE is currently text-only and does not support multimodal inputs (images, audio, embodied signals).
• Training Paradigms: The study focuses on SFT and DPO, leaving reinforcement learning-based paradigms for future exploration.
• Temporal Scope: The benchmark focuses on single-step decision scenarios; multi-turn or temporally extended decision-making remains an open direction.