Technical Summary: EIBench and CTC-GRPO for Emotion Management

Problem Statement

Emotional intelligence (EI) in Large Language Models (LLMs) is currently evaluated primarily through static understanding tasks or single-response dialogue generation. However, effective emotion management is inherently interactive and dynamic; it requires a model to not only recognize a user's emotion but to actively improve the user's emotional and relational state over multiple turns. Existing benchmarks often focus on narrow scopes, such as support-seeking dialogues (e.g., SAGE) or goal completion in social scenarios (e.g., SOTOPIA), failing to cover critical scenarios like maintaining boundaries under pressure, repairing trust after model errors, or proactively building rapport. Furthermore, standard reinforcement learning with verifiable rewards (RLVR) struggles in this domain because subjective tasks lack a single "correct" response, and terminal rewards fail to provide the dense, turn-level feedback necessary to credit specific actions that shift a user's state.

Methodology

1. EIBench: A Simulator-Based Benchmark

The authors introduce EIBench, a benchmark designed to evaluate and train LLMs on multi-turn emotion management.

• Taxonomy: Scenarios are organized by a $2 \times 2$ taxonomy based on the target of regulation (user vs. self) and the source of the situation (user-side event vs. model-side action). This yields four distinct scene types:
  • Support: Soothing user distress caused by external events.
  • Defense: Maintaining boundaries and tone under user pressure.
  • Repair: Rebuilding trust after model mistakes.
  • Charm: Proactively building rapport in model-initiated interactions.
• Dataset: The benchmark contains 2,222 scenarios (2,009 for training, 213 for testing).
• Simulation and Scoring: An LLM simulator plays the user role, updating an emotion–relation state ($a_\tau, t_\tau$) after every turn based on the model's response.
  • $a_\tau$: Negative emotion intensity (0–100).
  • $t_\tau$: Relational state/trust (0–100).
  • Anchors: Each scenario defines start ($s_{start}$), successful ($s_{succ}$), and failed ($s_{fail}$) state anchors. The final score is calculated by measuring the trajectory's progress toward the successful or failed anchor relative to the start, normalized to $[-1, 1]$.
• Dual Utility: The simulator serves as both an evaluation environment (using the final outcome reward) and a training environment (providing per-turn state updates as dense feedback).

2. CTC-GRPO: Centered Turn-Credit GRPO

To optimize LLMs for this task, the authors propose Centered Turn-Credit GRPO (CTC-GRPO), an extension of Group Relative Policy Optimization (GRPO).

• Motivation: Standard GRPO assigns the same trajectory-level credit to all turns based on the final outcome. In multi-turn emotional dialogue, this obscures which specific turns improved or worsened the user state.
• Process Reward: The simulator's per-turn state updates ($\Delta a_\tau, \Delta t_\tau$) are converted into a local process reward ($r^{proc}_\tau$) that rewards immediate improvements in user state, independent of the final outcome.
• Centered Credit Assignment: To prevent the process reward from altering the overall trajectory advantage (which is determined by the final outcome), the method centers the process reward within each rollout:
  $$A^{(k)}_\tau = A^{trace,(k)} + \alpha (r^{proc,(k)}_\tau - \bar{r}^{proc,(k)})$$
  Where $A^{trace,(k)}$ is the standard trajectory advantage, $\alpha$ is a weighting hyperparameter, and the term in parentheses is the deviation of the turn's process reward from the rollout's mean process reward. This ensures the process reward redistributes credit among turns without shifting the outcome-level signal.

Key Contributions

1. EIBench Benchmark: A comprehensive, simulator-based benchmark covering four distinct emotion management scenarios (Support, Defense, Repair, Charm) with a 2,222-scenario dataset and a state-tracking evaluation protocol.
2. Leaderboard and Validation: Evaluation of 15 open- and closed-source LLMs, revealing that current models excel at support and rapport building but struggle significantly with boundary maintenance (Defense). Rankings are validated via a second simulator (DeepSeek-V4-Pro) and a human pairwise arena.
3. CTC-GRPO Algorithm: A novel RL method that leverages simulator-tracked state changes as dense, turn-level process rewards. It successfully optimizes multi-turn dialogue policies without requiring additional judge models for intermediate feedback.

Experimental Results

• Baseline Performance: On the EIBench leaderboard, models generally perform well on Support and Charm scenes but score negatively on Defense (boundary maintenance), indicating a weakness in balancing warmth with constraint under pressure.
• CTC-GRPO Performance:
  • In-Domain: Training Qwen3-8B with CTC-GRPO improved its EIBench score from −22.4 to +22.4. The 32B variant reached +28.7, outperforming all evaluated closed-source systems on the training simulator.
  • Cross-Simulator: The improvements transferred to the unseen DeepSeek-V4-Pro simulator (8B: −23.7 $\to$ +16.6; 32B: +5.8 $\to$ +18.9), though gains were slightly lower on subjective scenes (Charm/Support) compared to concrete ones (Repair/Defense).
  • Out-of-Distribution (OOD): The trained models showed significant gains on external benchmarks:
    • SAGE: +12.4 improvement.
    • EQBench3: +20.9% improvement in rubric score, ranking #6 on the public leaderboard.
• Ablation Studies:
  • Removing the per-turn process reward caused performance to collapse (Avg: +22.4 $\to$ −4.9), confirming the necessity of dense feedback.
  • Removing the centering mechanism reduced performance significantly (+22.4 $\to$ +10.0), validating that centering is crucial to isolate turn-level credit without distorting the outcome signal.
  • Removing the $\sigma_{min}$ floor in GRPO normalization led to unstable training and lower final rewards.
• General Capability: Evaluation on math (AIME 2025), reasoning (GPQA), coding (HumanEval+), and instruction following (IFEval) showed that CTC-GRPO training largely preserved general capabilities, with only minor decreases in difficult math reasoning.

Significance and Claims

The paper claims that simulator-tracked user states provide a viable signal for both the evaluation and training of multi-turn emotion management, bridging the gap between static evaluation and dynamic interaction. The authors demonstrate that:

1. Current LLMs possess uneven EI capabilities, being strong in supportive roles but weak in defensive boundary management.
2. CTC-GRPO effectively utilizes dense, turn-level feedback derived from state transitions to optimize dialogue policies, outperforming standard RL approaches that rely solely on terminal rewards.
3. This approach improves performance not only on the specific benchmark but also generalizes to out-of-distribution social reasoning tasks, suggesting that learning to manage emotional states is a transferable skill.

The authors maintain a modest stance, acknowledging limitations such as the inherent noise in LLM-based simulator scoring and the varying difficulty across scene types (with Defense remaining the most challenging). They note that the current implementation uses a single hyperparameter for all scenes, leaving room for future work on scene-specific optimization strategies.