💻 computer science

Empirical Modeling of Therapist-Client Dynamics in Psychotherapy Using LLM-Based Assessments

This study leverages large language models to analyze nearly 2,000 hours of psychotherapy transcripts, revealing through structural equation modeling that therapist empathy and exploration directly drive client disclosure while rapport primarily reduces internal emotional distress.

Original authors: Angela Chen, Siwei Jin, Canwen Wang, Holly Swartz, Tongshuang Wu, Robert E Kraut, Haiyi Zhu

Published 2026-02-16

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Angela Chen, Siwei Jin, Canwen Wang, Holly Swartz, Tongshuang Wu, Robert E Kraut, Haiyi Zhu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine psychotherapy as a complex, high-stakes dance between two people: the therapist and the client. For decades, researchers have tried to figure out the steps of this dance. They know that when the therapist does something nice (like showing empathy), the client usually responds well. But exactly how, when, and why that happens has been a bit of a mystery, mostly because watching these dances in real-time is incredibly hard, expensive, and slow.

This paper is like a team of scientists building a super-powered, AI-powered camera that can watch thousands of these therapy dances at once, break them down into tiny steps, and tell us exactly which move leads to which reaction.

Here is the breakdown of their journey, using some everyday analogies:

1. The Problem: The "Black Box" of Therapy

Think of a therapy session like a black box. You put a client in, they talk for an hour, and you get a result (they feel better or worse). But what happened inside the box?

Old way: Researchers would ask the client later, "How did you feel?" or have a human expert sit and watch one video, taking notes. This is like trying to understand a whole movie by watching one frame every hour. It's slow, expensive, and misses the fast-paced details.
The Goal: The authors wanted to build a tool that could watch the entire movie, second-by-second, and understand the subtle shifts in the conversation.

2. The Tool: The "AI Translator" (LLMs)

The researchers used Large Language Models (LLMs)—the same kind of smart AI that powers chatbots. But instead of using them to write emails, they trained them to act like super-observant psychology students.

They taught the AI to look at therapy transcripts and score three specific things:

The Therapist's Moves: Did they show Empathy (like a warm hug in words) or Exploration (asking "Tell me more" questions)?
The Relationship (Rapport): Is there a "spark" or a strong bond between them? Think of this as the "chemistry" or the "trust level" in the room.
The Client's Reaction: Did the client open up and share secrets (Self-Disclosure)? Did they express sadness, anger, or fear?

The Test: They checked the AI's work against human experts. The AI was surprisingly good, agreeing with humans about 66% of the time on average. It was like hiring a robot that could grade a thousand essays in the time it takes a human to grade one, and it got most of the grades right.

3. The Discovery: What Actually Works?

Once they had the data from nearly 2,000 hours of therapy, they used a statistical method called Structural Equation Modeling (SEM). Think of this as a traffic map that shows how one car (therapist behavior) causes a traffic jam or a clear road (client reaction) downstream.

Here are the four big surprises they found:

🚦 Surprise #1: The "Push and Pull" of Empathy

The Myth: "If the therapist is nice, the client will just open up and feel happy."
The Reality: When a therapist shows Empathy or asks Exploratory questions, the client does open up more. BUT, they also start expressing more negative emotions (like sadness, anxiety, or fear).
The Analogy: Imagine a therapist is like a therapist opening a pressure valve. When they are kind and curious, the client feels safe enough to let the "steam" out. That steam is the client's pain. It looks like things are getting "worse" (more crying, more anger) in that exact moment, but it's actually a sign that the client is finally letting go of the burden.

🚦 Surprise #2: The "Bond" Doesn't Force Openness

The Myth: "If the client and therapist have a great friendship (Rapport), the client will spill all their secrets immediately."
The Reality: The study found that a strong, pre-existing bond did not make the client talk more about their secrets in the next moment.
The Analogy: Think of Rapport like the comfort of a living room. Just because the living room is cozy doesn't mean you immediately start telling your host your deepest secrets. Sometimes, a cozy room just means you feel safe enough to stop worrying so much, rather than start talking more.

🚦 Surprise #3: The Bond Heals the "Internal" Pain

The Reality: While the bond didn't make them talk more, it did make them feel less internal pain (less sadness, anxiety, depression).
The Analogy: If the therapist-client relationship is like a warm blanket, it doesn't necessarily make you tell a story, but it stops you from shivering. The client feels less distress internally, even if they aren't saying much out loud.

🚦 Surprise #4: Different Kinds of Anger

The researchers noticed that clients express "bad feelings" in two ways:

Inward: Sadness, fear, anxiety (looking at yourself).
Outward: Anger, disgust, contempt (looking at others).

The Finding: Empathy mostly helped with the Inward feelings (making the client feel safe enough to admit they are sad). It didn't really change the Outward feelings (like anger at the world) as much.

4. Why This Matters: The "GPS for Therapists"

So, what do we do with this? The authors suggest building digital tools for therapists that act like a GPS navigation system.

Current Training: A therapist might get feedback once a month: "You did a good job."
Future Training (The AI Idea): Imagine a dashboard that whispers to the therapist during the session: "Hey, you just asked a great exploratory question, and look—the client just opened up about their fear. Keep going!" or "You've been very empathic, and the client is getting very sad. Maybe it's time to pause and just sit with them, rather than digging deeper."

The Big Takeaway

This paper proves that we can use AI to understand the "micro-moments" of therapy. It teaches us that therapy isn't just about being nice; it's about knowing when to be empathic to help a client release pain, and knowing that a strong relationship acts like a safety net that calms internal distress, even if the client isn't talking much.

It's like moving from guessing the weather by looking out the window, to having a satellite that tells you exactly where the rain is falling, so you can help people stay dry.

1. Problem Statement

Psychotherapy is a primary treatment for mental health conditions, but the mechanisms linking therapist behaviors to client outcomes are difficult to measure at scale. Current evaluation methods rely heavily on:

Infrequent self-reports: (e.g., PHQ-9, GAD-7) collected post-session, which lack resolution regarding in-session dynamics.
Manual behavioral coding: Which is labor-intensive, expensive, and difficult to deploy in real-world digital health settings.

There is a critical gap in scalable, reliable, and consistent models that can quantify the moment-to-moment interplay between therapist behaviors (e.g., empathy, exploration), the therapeutic relationship (rapport), and client responses (disclosure, emotional expression) directly from session content.

2. Methodology

The study employs a two-stage computational framework: LLM-based measurement followed by Structural Equation Modeling (SEM).

A. Dataset

Source: Alexander Street (AS) corpus, containing 1,610 unique, anonymized one-on-one therapy sessions.
Scale: ~243,407 utterances from 37 clients.
Privacy: Data is strictly anonymized; the study operates under a non-exclusive license for research only. OpenAI API usage ensures data is not used to train models.

B. LLM-Based Assessment Pipeline

The authors developed automated methods using Large Language Models (specifically GPT-4o-mini) to score specific constructs.

Constructs Measured:
1. Therapist Behaviors: Empathy (broken into Emotional Reaction, Interpretation, Exploration) and Reflection (based on Motivational Interviewing).
2. Client Responses: Self-disclosure (General, Medium, High) and Emotions (9 categories: anger, contempt, disgust, enjoyment, fear, sadness, surprise, anxiety, depression).
3. Relationship Quality: Rapport (modeled as the "Bond" component of the therapeutic alliance).
Prompting Strategy:
- Used Zero-shot and One-shot prompting with clinical definitions (e.g., EPITOME framework, WAI-O-S).
- Implemented Chain-of-Thought (CoT) reasoning: The LLM first generates a justification grounded in specific conversational cues before assigning a rating.
- Context Windows: Tailored per construct (e.g., 2 prior utterances for empathy; 5 prior for emotion; full session segments for rapport).
Validation:
- Validated against human annotations from 27 trained psychology students/counselors.
- Metrics: Intraclass Correlation Coefficient (ICC) and Pearson correlation ( $r$ ).
- Result: LLM scores showed strong alignment with human consensus (Mean Pearson $r = .66$ ). Rapport and Self-Disclosure showed the highest agreement ( $r \approx 0.67-0.75$ ), while internalizing emotions (fear, anxiety) showed lower agreement due to their implicit nature in text.

C. Statistical Modeling (SEM)

Technique: Structural Equation Modeling (SEM) using R's lavaan package.
Data Reduction: Principal Component Analysis (PCA) was used to group correlated variables:
- Emotions: Grouped into Self-Directed Negative Emotions (sadness, fear, anxiety, depression, reverse enjoyment) and Outward-Directed Negative Emotions (anger, contempt, disgust).
- Empathy: Grouped into General Empathy (Reaction, Interpretation, Reflection) and Exploration.
Model Structure: A multivariate SEM regressing client outcomes (disclosure, emotions) on:
- Immediate therapist behaviors (utterance-level).
- Cumulative rapport (session-level, from the previous session).
- Session index (log-transformed).
Hypotheses Tested:
- H1: Therapist empathy/exploration predicts immediate client disclosure.
- H2: Cumulative rapport predicts immediate client disclosure.
- H3: Therapist empathy predicts increased expression of negative emotions.
- H4: Cumulative rapport predicts increased expression of negative emotions.

3. Key Results

A. Validation Performance

Reliability: LLMs achieved "good" to "excellent" agreement with human raters for most constructs.
Discrepancies: The model struggled most with internalizing emotions (fear, sadness, anxiety, depression), where human raters could infer context better than the model relying on lexical cues.

B. Structural Equation Modeling Findings

Hypothesis 1 (Supported): Therapist Empathy and Exploration strongly and positively predict Client Self-Disclosure in the immediate next turn ( $\beta \approx 0.09, p < .001$ ).
Hypothesis 2 (Not Supported): Cumulative Rapport from previous sessions did not predict immediate client disclosure in the current session ( $\beta = 0.00, p = .71$ ).
Hypothesis 3 (Supported): Therapist Empathy significantly increases the expression of Negative Emotions.
- The effect was strongest for Self-Directed Negative Emotions ( $\beta = 0.12$ ) compared to Outward-Directed ( $\beta = 0.03$ ).
- Interpretation: Empathy creates a safe space for clients to acknowledge internal distress.
Hypothesis 4 (Contradicted): Higher Rapport was associated with lower expression of Self-Directed Negative Emotions ( $\beta = -0.02, p = .05$ $β = - 0.02, p = .05$ ), not higher.
- Interpretation: High rapport may indicate that clients feel less internal distress or feel less need to articulate it, rather than simply "opening up" more.

4. Key Contributions

Scalable Measurement Framework: Demonstrated that LLMs can serve as a scalable, reliable layer for measuring complex, clinically grounded therapeutic constructs (empathy, rapport, disclosure) from raw transcripts, achieving alignment with human experts.
Mechanistic Insights: Provided empirical evidence distinguishing the roles of moment-to-moment behaviors vs. cumulative relationship quality:
- Immediate behaviors (empathy/exploration) drive immediate client responses (disclosure/emotion).
- Rapport acts as a contextual moderator that dampens internal distress rather than a direct driver of disclosure volume.
Dual-Route Model: Proposed a model where empathy/exploration facilitate emotional expression, while rapport regulates the intensity of negative affect, suggesting a nuanced approach to therapeutic intervention.
Methodological Advancement: Successfully applied SEM to turn-level, LLM-derived data, bridging the gap between micro-level interaction analysis and macro-level therapeutic outcomes.

5. Significance and Implications

Digital Mental Health: The framework enables the development of real-time decision support systems for clinicians. Instead of generic feedback, tools can visualize how specific therapist moves (e.g., an empathic reflection) immediately influence client emotional states.
Therapist Training: The findings support the creation of AI-driven virtual patients for training. Trainees can receive immediate feedback on how their interventions affect simulated client disclosure and emotion, allowing for the practice of "titrating" empathy and exploration based on the relational climate.
Quality Improvement: Automated analytics can identify specific moments in therapy where rapport or technique needs adjustment, moving beyond aggregate session scores to granular process improvement.
Ethical Considerations: The authors highlight the need for caution regarding cultural bias in LLMs (e.g., misinterpreting cultural norms as disengagement) and the importance of using observer-rated vs. client-rated rapport in future studies.

In summary, this work establishes a computational foundation for understanding the mechanisms of psychotherapy, proving that AI can capture the subtle, dynamic interplay between therapist actions and client responses at a scale previously impossible with manual coding.