💬 NLP

When Less Is More: An Empirical Study of Minimal Responses in Counseling Dialogues and the Behavior of LLMs

This paper empirically demonstrates that while minimal responses are crucial for effective human counseling, they are significantly underrepresented in LLM-generated dialogues and often undervalued by current evaluation frameworks, as models struggle to appropriately deploy them without explicit instruction.

Original authors: Zhiyang Qi

Published 2026-08-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhiyang Qi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the quiet space of a counseling session, the most powerful tool a therapist often wields is not a long explanation or a complex strategy, but a simple sound. A soft "mm-hmm," a brief "I see," or a short "yes" can signal deep attention, validate a client's feelings, and invite them to keep speaking. These are known as minimal responses. They do not interrupt the flow of a person's story; instead, they act as a gentle hand on the shoulder, saying, "I am here, and I am listening." For decades, human counselors have relied on these tiny verbal cues to build trust and encourage clients to explore their deepest thoughts. However, as artificial intelligence begins to step into the role of a virtual counselor, a critical question has emerged: can a computer learn the art of saying less to achieve more?

A researcher at the University of Tokyo set out to investigate this very question. They examined how well current artificial intelligence systems understand the value of brevity in psychological support. The researcher analyzed a vast collection of counseling conversations, comparing those recorded from real human therapists against those generated by large language models, the sophisticated computer programs that power modern chatbots. They found a striking difference. In the real-world recordings, minimal responses were common, appearing frequently as therapists encouraged clients to continue sharing their burdens. In the computer-generated conversations, however, these brief acknowledgments were almost entirely absent. Instead, the artificial intelligence tended to produce long, information-heavy replies that often included questions or advice, even when the situation called for silence and simple presence.

To understand why this gap existed, the researcher developed a careful method to identify these short responses across different languages, including Chinese, Japanese, and English. They looked for counselor utterances that were very short and did not introduce new topics or questions, checking to see if the client continued to speak immediately afterward. Their analysis confirmed that human therapists use these minimal cues in roughly five to twenty-four percent of their turns, depending on the dataset. In contrast, the artificial intelligence models trained on synthetic data—conversations created by computers rather than recorded from humans—produced minimal responses in rates as low as 0.00% and 0.35% in the specific models tested. The models trained on synthetic data seemed to have learned a rigid pattern: listen, then immediately offer a detailed, helpful response. They appeared to miss the subtle, interactional value of simply holding space for the client.

The researcher then tested whether these artificial intelligence systems could be taught to use minimal responses if explicitly asked. They took specific moments from real counseling sessions where a human therapist had used a short, supportive phrase and asked the artificial intelligence to generate a response for the same situation. When given a standard instruction to act as a counselor, the most advanced commercial models produced long, detailed answers. However, when the researcher added a specific instruction to use a short response when the client was still expressing emotions, the models changed their behavior. They successfully generated minimal responses in nearly all cases, proving that the technology is capable of producing them. Yet, without that specific instruction, the models still struggled to recognize the right moment to be brief, often defaulting to their habit of providing too much information.

This tendency has a significant side effect on how we judge the quality of these conversations. The study revealed that the current methods used to evaluate artificial intelligence counseling responses often penalize brevity. When the researcher asked the artificial intelligence to grade the responses based on standard criteria like professionalism and completeness, the short, human-like minimal responses received low scores. The evaluators favored the long, content-rich replies, even though those long replies were more likely to interrupt the client's flow of thought. In the real world, a counselor who speaks too much can shut down a client's willingness to share, but the current evaluation systems seem to value the appearance of expertise over the actual flow of the conversation.

The findings suggest that the path to better artificial counseling systems requires a shift in how these models are trained and evaluated. Simply feeding computers more data is not enough if that data lacks the natural rhythm of human interaction. The study indicates that models trained on synthetic, computer-generated data fail to learn the nuance of minimal responses, while those trained on real human conversations perform better. Furthermore, the researcher warns that relying on automated scoring systems may inadvertently push developers to create chatbots that talk too much, missing the therapeutic power of silence. For artificial intelligence to truly support mental health, it must learn that sometimes, the most professional and empathetic thing it can say is nothing at all, or just a few words that say, "I hear you."

Technical Summary: "When Less Is More: An Empirical Study of Minimal Responses in Counseling Dialogues and the Behavior of LLMs"

Problem Statement
In psychological counseling, effective support often relies on "minimal responses"—brief utterances such as backchannels ("Mm-hmm," "I see") or concise empathic statements. These serve critical interactional functions: signaling attentive listening, expressing empathy, and encouraging clients to continue their narratives without interruption. However, existing Large Language Model (LLM) counseling systems and evaluation frameworks tend to favor explicit, information-rich, and structurally complete responses. This bias risks overlooking the therapeutic value of brevity, potentially leading models to generate verbose or templated replies that disrupt the client's flow. Furthermore, current LLM-based evaluation methods may systematically undervalue minimal responses, penalizing them for lacking content density even when they are interactionally appropriate.

Methodology
The author employs a two-pronged approach involving dataset analysis and generative evaluation across three languages (Chinese, Japanese, and English).

Cross-Lingual Dataset Analysis:
- Datasets: The study analyzes seven counseling dialogue datasets: four LLM-generated (CACTUS, CPsyCounD, PsyDTCorpus, SmileChat) and three human-collected (AnnoMI, PsyDial-D4, KokoroChat).
- Identification Pipeline: A two-stage filtering method is developed to identify minimal responses:
  - Stage 1 (Rule-based): Filters counselor utterances based on length thresholds (e.g., ≤15 characters for Chinese/Japanese, ≤10 words for English) and the length of the subsequent client utterance (to ensure the client continued speaking). It excludes short questions, directives, greetings, and logistical phrases using predefined keyword lists.
  - Stage 2 (LLM-based Verification): Uses GPT-5.4-mini to contextually verify candidates, classifying them into: (1) backchannel-like responses, (2) brief empathic/reflective responses, or (3) other short responses. Only the first two categories are counted as minimal responses.
Generative Evaluation:
- Setup: The author manually curated 284–300 dialogue contexts per language where human counselors used minimal responses.
- Models: Evaluated general-purpose open-source models (Qwen, Llama), a state-of-the-art commercial model (GPT-5.4), and counseling-specific models (SoulChat2.0, MindChat, KokoroChat-Full, PsyDial-Pi4).
- Prompting Strategies:
  - General Prompt: Asks the model to generate the next response based on history (testing natural appropriateness).
  - Instructional Prompt: Explicitly instructs the model to prioritize minimal responses when the client is expressing emotions (testing instruction-following capability).
- Evaluation Metrics: Responses were assessed by two LLMs (GPT-5.4-mini and Gemini-3.1-Flash-Lite) on:
  - Minimal Response Rate (MR): Percentage of generated responses that qualify as minimal.
  - Interruption Risk: A 0–5 score (lower is better) estimating the likelihood of interrupting the client.
  - Response Quality: A composite score (0–2 for comprehensiveness, 0–3 for professionalism/authenticity, 0–1 for safety) based on existing frameworks (Zhang et al., 2024).

Key Contributions

Empirical Evidence of Disparity: The study provides the first systematic cross-lingual analysis showing that minimal responses are substantially more common in human-collected datasets (ranging from ~5% to 24% of counselor turns) than in LLM-generated datasets (often <0.2% or nearly absent).
Model Behavior Analysis: The paper demonstrates that models trained on synthetic data struggle to generate minimal responses, whereas those trained on human data (or retrained on it) perform significantly better.
Evaluation Bias Identification: The author reveals that standard LLM-based quality evaluation metrics favor content-rich responses. When models are forced to generate minimal responses, their "quality" scores drop significantly, even though their "interruption risk" scores improve, indicating a misalignment between current evaluation rubrics and therapeutic appropriateness.

Results

Dataset Distribution: Minimal responses were nearly absent in LLM-generated datasets (e.g., 4 instances or 0.00% in CACTUS, 0.15% in CPsyCounD) but prevalent in human-collected ones (e.g., 24.37% in AnnoMI, 5.12% in KokoroChat).
Generation Capabilities:
- Synthetic-trained models: Models like SoulChat2.0 and MindChat (trained on synthetic data) generated minimal responses at rates of 0.00% and 0.35% respectively under general prompting.
- Human-trained models: Models retrained on human data (PsyDial-Pi4, KokoroChat-Full) achieved MR rates of 12.68% and 82.67% respectively.
- Instruction Following: Strong commercial LLMs (GPT-5.4) could generate minimal responses when explicitly instructed (97.18% in Chinese, 82.00% in Japanese) but failed to do so naturally under general prompting (0.00%).
Evaluation Discrepancy: There is a strong negative correlation between minimal response rates and LLM-based quality scores (Pearson $r = -0.86$ ). Conversely, there is a strong positive correlation between quality scores and interruption risk ( $r = 0.89$ ). This suggests that current evaluation frameworks penalize the very behaviors (brevity) that reduce interruption risk in counseling.

Significance and Claims
The paper claims that the current trajectory of counseling dialogue research—relying heavily on synthetic data and content-heavy evaluation metrics—risks producing systems that are therapeutically ineffective due to an inability to utilize minimal responses. The author argues that:

Data Construction: Counseling datasets must explicitly include and preserve minimal responses to train models that understand interactional flow.
Model Development: Models need to learn not just how to generate minimal responses, but when they are appropriate, a skill currently lacking in general-purpose LLMs without explicit instruction.
Evaluation Reform: Evaluation frameworks must be revised to value interactional appropriateness (e.g., low interruption risk) over mere content comprehensiveness, as current "LLM-as-a-Judge" methods may systematically undervalue essential counseling techniques.

The author concludes that while LLMs can be instructed to produce minimal responses, they do not yet reliably determine their appropriateness, highlighting a critical gap in the development of safe and realistic virtual counseling systems.

Technical Summary: "When Less Is More: An Empirical Study of Minimal Responses in Counseling Dialogues and the Behavior of LLMs"

More like this