💬 NLP

Discourse Diversity in Multi-Turn Empathic Dialogue

This paper reveals that large language models exhibit excessive rigidity in reusing discourse moves during multi-turn empathic conversations, and proposes MINT, a reinforcement learning framework that optimizes for cross-turn tactic novelty to significantly improve both discourse diversity and overall empathy quality.

Original authors: Hongli Zhan, Emma S. Gueorguieva, Javier Hernandez, Jina Suh, Desmond C. Ong, Junyi Jessy Li

Published 2026-04-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hongli Zhan, Emma S. Gueorguieva, Javier Hernandez, Jina Suh, Desmond C. Ong, Junyi Jessy Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Broken Record" Empath

Imagine you go to a friend for comfort because you're having a terrible day.

Turn 1: You say, "I'm so stressed about work."
Friend: "That sounds really hard. Have you tried making a to-do list?"
Turn 2: You say, "I tried, but my boss is still yelling at me."
Friend: "That sounds really hard. Have you tried making a to-do list?"
Turn 3: You say, "I'm just so tired."
Friend: "That sounds really hard. Have you tried making a to-do list?"

Even though the friend is being "nice," they feel robotic and repetitive. They are stuck on a single track.

This is exactly what Large Language Models (LLMs) like ChatGPT do.
Research shows that while AI is great at giving a kind, empathetic answer once, it gets stuck in a loop when you keep talking to it. If it gives advice in the first turn, it gives advice in the second, third, and fourth turns. It's like a DJ who only knows one song and plays it over and over again, even when the crowd wants to dance to something else.

The paper calls this "Tactic Stickiness." It's the tendency of AI to "stick" to the same conversational move (like giving advice or asking a question) repeatedly, even when the human needs something different.

The Discovery: Why Standard Checks Fail

The researchers tried to measure this repetition using standard computer tools.

The "Copy-Paste" Detector: They checked if the AI used the same words. (It didn't always; the AI changed the words).
The "Meaning" Detector: They checked if the AI said the same idea. (The tools said the ideas were different enough).

But the human listeners still felt bored and unheard. Why? Because the function of the conversation was the same.

Analogy: Imagine a chef who makes a burger, then a burger with different toppings, then a burger with different cheese. To a robot scanner, the ingredients are different. But to the hungry customer, they are just getting the same thing over and over. They wanted a salad, then a soup, then a sandwich. The AI kept serving burgers.

The researchers found that AI repeats its "conversational moves" (tactics) at nearly double the rate of human friends.

The Solution: MINT (The "Conversational Chameleon")

To fix this, the team created a new training method called MINT (Multi-turn Inter-tactic Novelty Training).

Think of MINT not as teaching the AI what to say, but teaching it how to change its style as the conversation flows.

How MINT Works (The Analogy):
Imagine the AI is a chameleon sitting on a conversation branch.

The Old Way (Vanilla AI): The chameleon turns blue, stays blue, and stays blue, even if the branch changes color. It's comfortable but boring.
The MINT Way: The AI is trained to look at what it did last time.
- Last turn: "I gave you advice."
- Current turn: "Okay, I need to switch! I will validate your feelings instead of giving advice."
- Next turn: "Okay, I validated. Now I will ask a question to help you think deeper."

MINT rewards the AI with a "digital high-five" (a reward signal) when it successfully switches tactics. It says, "Good job! You didn't just repeat yourself; you adapted to the moment."

The Results: A Better Conversation

When they tested MINT on AI models:

Less Repetition: The AI stopped looping. It became much better at mixing up its moves (asking questions, offering comfort, sharing stories, giving advice) just like a real human would.
More Empathy: Surprisingly, by making the AI less repetitive, the AI actually became more empathetic.
- Why? Because real empathy isn't just saying "I'm sorry." It's knowing when to listen, when to ask, and when to offer a solution. By forcing the AI to vary its approach, it learned to be more responsive to the human's actual needs.

The Big Takeaway

The paper concludes that current AI models aren't lacking in "kindness." They have plenty of kind words. What they lack is variety.

The Final Metaphor:
Think of a conversation as a dance.

Current AI: It's a dancer who learned one perfect step. It does that step beautifully, but then it keeps doing it for 10 minutes straight. It's awkward and stiff.
MINT AI: It's a dancer who learned the whole routine. It knows when to spin, when to dip, and when to hold still. It moves with the music (the human's emotions).

The researchers proved that by teaching the AI to be a better dancer (varying its moves), it becomes a much better partner for people who are hurting.

Where to find it:
The team has released their code and data so others can build these "chameleon" AIs. You can find it on their GitHub page mentioned in the paper.

1. Problem Statement

While Large Language Models (LLMs) have demonstrated high empathy in single-turn settings, they exhibit significant formulaic behavior in multi-turn conversations. Specifically, LLMs tend to reuse the same discourse moves (empathy tactics) repeatedly across turns, failing to adapt their support strategies as a conversation evolves.

The Gap: Existing research has documented lexical and syntactic repetition in LLMs, but few studies have analyzed repetition at the discourse function level (e.g., validating, advising, questioning).
The Phenomenon: The authors introduce "Tactic Stickiness," a metric measuring the probability that a tactic used in turn $t-1$ reappears in turn $t$ .
Key Finding: LLMs exhibit tactic stickiness nearly double that of human supporters (0.50–0.56 for LLMs vs. 0.27 for humans).
Limitation of Current Metrics: Standard similarity metrics (BLEU, BERTScore) fail to detect this repetition because they measure surface-level lexical or semantic overlap, not the functional diversity of the support strategy. This rigidity limits the effectiveness of AI in sustained emotional support, even if individual responses seem empathic.

2. Methodology

A. Empathy Tactic Taxonomy & Tagging

The authors utilize a taxonomy of 10 empathy tactics derived from psychological literature (e.g., Validation, Questioning, Advice, Reappraisal, Self-Disclosure).

Automatic Tagging: They trained a sentence-level tagger using LoRA adapters fine-tuned on Llama-3.1-8B-Instruct. Each of the 10 tactics has a dedicated adapter trained on human-annotated data (3,164 sentences) to classify whether a specific sentence contains a given tactic.
Performance: The taggers achieve an average F1 score of 0.80, with high performance on distinct tactics like Questioning (0.94) and Advice (0.87).

B. Analysis of Multi-Turn Conversations

The authors analyzed 322 multi-turn emotional support conversations (1,719 supporter turns) from WildChat and SENSE-7 datasets, alongside a human-human benchmark.

Observation: LLMs heavily overuse Advice and Information while underusing Questioning.
Stickiness: Once an LLM uses a tactic (e.g., Advice), it is highly likely to use it again in the next turn, creating a "loop" of repetitiveness that correlates negatively with user willingness to re-engage ( $\rho = -0.287$ ).

C. MINT Framework (Multi-turn Inter-tactic Novelty Training)

To address this, the authors propose MINT, the first Reinforcement Learning (RL) framework designed to optimize discourse move diversity across multi-turn dialogues.

Objective: Maximize empathy quality while minimizing cross-turn tactic repetition.
Algorithm: Uses Group Relative Policy Optimization (GRPO).
Reward Function: The total reward $R$ $R$ combines three components:
1. Quality Reward ( $Q$ ): A base reward (using the PsychoCounsel model) ensuring the response is empathic and supportive.
2. Cross-Turn Novelty ( $D_{KL}$ ): A Kullback-Leibler (KL) divergence term measuring the difference between the current turn's tactic distribution ( $Q_t$ ) and the previous turn's distribution ( $P_{t-1}$ ). This penalizes the model for repeating the same mix of tactics.
3. Within-Turn Breadth ( $H$ ): An entropy term encouraging the model to use multiple different tactics within a single response, rather than relying on a single dominant tactic.
Training: The model is trained on 1.7B and 4B parameter models (Qwen3) using the GRPO algorithm with a length penalty to prevent overly long responses.

3. Key Contributions

Discovery of Discourse-Level Repetition: Provided the first multi-turn evidence that LLMs repeat discourse moves at nearly double the human rate, a phenomenon invisible to standard lexical/semantic similarity metrics.
MINT Framework: Introduced the first RL framework to explicitly optimize for tactic diversity conditioned on conversation history, moving beyond single-turn quality optimization.
Evaluation Protocol: Validated a turn-level evaluation protocol for perceived empathy that achieves expert-level agreement ( $\kappa_w = 0.58$ ), allowing for granular assessment of multi-turn empathy.

4. Results

The authors evaluated MINT against several baselines, including Vanilla prompting, Prompt-based diversity (Tactic Prompts), Quality-only RL (PsychoCounsel), and Token-level diversity (R1-Zero-Div).

Empathy Quality: MINT (specifically the $Q + D_{KL}$ variant) improved aggregate empathy scores by 25.3% over the Vanilla baseline across both 1.7B and 4B models.
Tactic Diversity: MINT reduced tactic stickiness by 26.3% on the 4B model, significantly outperforming all baselines.
Comparison with Baselines:
- Token-level diversity (R1-Zero-Div): Failed to improve discourse diversity; models remained "sticky" despite token variation.
- Quality-only RL: Achieved high empathy but high stickiness (repetitive patterns).
- Prompt-based methods: Improved empathy slightly but did not solve the stickiness problem effectively.
Qualitative Shift: MINT models significantly reduced Unsolicited Advice (by ~40-45%) and increased Encouraging Elaboration (by ~200%), shifting the conversation from "fixing" to "exploring."

5. Significance and Implications

Beyond "Warmth": The study demonstrates that effective AI empathy is not just about generating warm individual messages, but about strategically varying support strategies over the arc of a conversation.
Training Objective: It establishes discourse move diversity as a learnable objective. Simply optimizing for token diversity or single-turn quality is insufficient; the model must learn to adapt its functional role (e.g., shifting from validation to questioning) based on conversation history.
Practical Impact: By reducing tactic stickiness, MINT makes AI supporters more credible and engaging for sustained emotional support, addressing a critical limitation in current LLM deployments for mental health applications.
Ethical Note: The authors acknowledge that their evaluation uses fixed human conversation histories (gold context) rather than live interactive chats with real users, citing ethical concerns regarding real-time psychological risk. However, they posit this as a necessary first step to establish the technical feasibility of the approach.

In conclusion, the paper argues that the current limitation of empathic LLMs is not a lack of empathy, but a lack of strategic variation in their discourse moves. MINT successfully bridges this gap, enabling models to provide more dynamic, human-like, and effective emotional support.