Technical Summary: PSYCHEPASS

Problem Statement

The evaluation of Large Language Models (LLMs) in mental healthcare faces a fundamental "unanchored defect" that leads to two forms of instability:

1. Process Drift: Current simulation paradigms often rely on unsteered, free-form chats or single-turn QA. Without clinical direction, these interactions wander aimlessly, failing to trigger high-stakes scenarios (e.g., trauma processing) or measure specific therapeutic capabilities, resulting in evaluations that capture only superficial features like generic empathy.
2. Standard Drift: Existing judgment methods predominantly use static pointwise scoring (e.g., Likert scales). This forces evaluators to rely on vague, absolute standards without comparative references, leading to evaluation collapse where scores cluster in narrow ranges with poor discriminability, making accurate ranking and optimization impossible.

Methodology: PSYCHEPASS Framework

PSYCHEPASS introduces a unified framework to calibrate therapeutic competence by anchoring two traceable trajectories: the Interaction Trajectory in simulation and the Battle Trajectory in judgment.

1. Anchoring the Interaction Trajectory (Simulation)

To eliminate process drift, the framework replaces open-ended chats with Scripted Probing grounded in Single-Session Therapy (SST) theory.

• Client Profiles: Simulated clients are initialized from de-identified real-world data and expanded into high-fidelity personas containing core drives, reaction patterns, and specific conflict events.
• Competency Dimensions: The evaluation targets 12 specific dimensions categorized into Alliance Building (Empathy, Discernment, Engagement), Professional Technique (Skill, Suggestion, Reframing, Progression, Trauma), and Reliability Support (Crisis, Ethics, Diversity, Memory).
• Trajectory Control: The simulation is structured into five distinct phases (Assessment, Pattern Awareness, Conflict Evolution, Behavior Construction, Integration). A system prompt dynamically updates the simulated client's behavior at each turn to trigger specific responses targeting the nine local competency dimensions, ensuring a comprehensive "stress test" of the model.

2. Anchoring the Battle Trajectory (Judgment)

To resolve standard drift, the framework abandons static scoring in favor of Dynamic Pairwise Battles.

• Swiss-System Tournament: To manage computational costs ($O(N^2)$ vs. $O(N \log N)$), a four-round Swiss-system tournament is employed. Models are matched against opponents with similar win records, ensuring strong models compete against each other.
• Debiasing: To counter position bias in LLM judges, a stage-slicing debiasing strategy is used. Dialogue segments are presented in an alternating pattern (Model A's segment 1, Model B's segment 1, Model B's segment 2, Model A's segment 2) to cancel out first-mover advantages.
• Elo Rating Estimation: The Bradley-Terry model is used to convert pairwise outcomes into stable scalar Elo ratings via global optimization (Maximum Likelihood Estimation), providing a robust and comparable ranking.

3. Optimization via Trajectory-Based RL

The framework establishes a closed loop where tournament trajectories are transformed into reward signals for On-Policy Reinforcement Learning.

• Reward Modeling: A reward model ($M_{RM}$) is trained on the partial orders of battle outcomes to distinguish therapeutic quality across dimensions.
• Group Relative Policy Optimization (GRPO): A target LLM is optimized using the reward model on the same client profiles used in the tournament. The objective maximizes the advantage of high-reward responses while penalizing distribution shift via KL divergence.
• Generalization Verification: Performance is validated on a held-out set of 100 client profiles to ensure the model learns generalized therapeutic principles rather than memorizing specific interaction patterns.

Key Results

The framework was applied to evaluate 12 LLMs, including domain-specific counseling models and cutting-edge general models.

• Competence Landscape: Proprietary frontier general models (e.g., GPT-5.2, Gemini 3 Pro) significantly outperformed domain-specific models. While domain models showed strength in narrow metrics like empathy, they lacked the reasoning, world knowledge, and instruction-following capabilities required for rigorous, longitudinal counseling scenarios.
• Dimensional Differentiation: Models exhibited distinct profiles. For example, DeepSeek-V3.2 scored exceptionally high in Diversity but lower in other areas, while PsyLLM surpassed GPT-5.2 in Empathy but struggled with Memory and Progression.
• Human Alignment: PSYCHEPASS demonstrated high consistency with human expert judgments (Cohen's $\kappa > 0.7$), achieving near-perfect agreement (1.000) in strong-vs-weak matchups and surpassing inter-expert consistency in some cases.
• RL Effectiveness: After trajectory-based RL training, a target model achieved a win-to-loss ratio of 62:25 against its original counterpart. Significant improvements were observed in Diversity (4.0% $\to$ 92.0%) and Empathy (31.0% $\to$ 82.0%), though some regression occurred in Crisis and Ethics dimensions, likely due to the shift from direct advice-giving to therapeutic engagement.

Significance and Claims

The paper claims PSYCHEPASS addresses the critical gap in evaluating LLM therapeutic competence by establishing a rigorous, discriminative, and traceable paradigm. Its primary contributions are:

1. Calibration Framework: It provides the first unified framework to rank and optimize LLMs based on anchored interaction and battle trajectories, moving beyond unanchored simulations and static scoring.
2. Reward Generation: It demonstrates a viable pathway to obtaining high-fidelity rewards for aligning LLM counseling capabilities via on-policy RL, creating a complete loop from evaluation to optimization.
3. Foundation for Trust: By validating high consistency with human experts and demonstrating effective optimization, PSYCHEPASS aims to lay the foundation for trustworthy, continuously evolving counseling systems that can serve as assistive tools under professional supervision.

The authors explicitly state that PSYCHEPASS is not a clinical standard for human therapists nor a replacement for licensed professionals. Instead, it is a calibration tool designed to ensure AI assistants meet professional standards before integration into clinical workflows as supportive mechanisms for screening, psychoeducation, and engagement.