💬 NLP

All four leading LLMs talk more than they listen to personality-verified synthetic help-seekers

This paper reveals that four leading large language models fail to distinguish between personality-verified synthetic help-seekers in crisis scenarios, consistently exhibiting excessive verbosity and premature problem-solving rather than effective listening or emotional stabilization.

Original authors: Pablo A. Fonseca, Raquel Rodríguez-Carvajal, Rafael A. Calvo

Published 2026-08-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pablo A. Fonseca, Raquel Rodríguez-Carvajal, Rafael A. Calvo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

When a person faces a sudden, overwhelming crisis—such as learning that a loved one has been diagnosed with dementia—their mind changes in predictable ways. The intense stress narrows their attention, making it difficult to process new information or follow long explanations. In these moments, the most effective human support follows a specific rhythm: listen first to stabilize the person's emotions, and only later offer solutions or advice. Today, many people turn to artificial intelligence chatbots for help during these exact moments of distress, hoping for clarity and comfort. However, these digital assistants are often designed to be helpful by providing information quickly, which can lead them to talk too much and solve problems too soon, potentially overwhelming a user who is already struggling to keep up.

A team of researchers at Imperial College London and the Universidad Autónoma de Madrid set out to test whether the most advanced artificial intelligence models available today can actually adapt to this human need for listening over talking. They built a sophisticated simulation to act as a safe testing ground. Instead of risking real people in crisis, they created seven synthetic "help-seekers," each programmed with a distinct psychological profile. These profiles were not just vague descriptions but were based on established scientific measures of personality, coping styles, and resilience. Each of these digital characters was placed in a scenario where they had just received the devastating news of a relative's dementia diagnosis. They then engaged in a ten-turn conversation with four of the world's leading language models, which acted as the "advisors" trying to offer support.

To ensure the results were objective, the researchers introduced a third role: an independent "auditor." This auditor was another artificial intelligence model that listened to the entire conversation without knowing the psychological profile of the help-seeker. Its job was to judge whether the advisor had successfully listened, whether it had stabilized the person's emotions, and whether it had avoided common pitfalls like giving premature advice or speaking too much. The researchers also checked if the synthetic help-seekers were actually behaving consistently with their assigned personalities, asking if an observer could tell who was who just by reading the chat logs.

The study revealed a striking and consistent pattern across all four artificial intelligence models tested. Regardless of which model was acting as the advisor, they all talked more than they listened. In every single conversation, the advisor spoke more words than the help-seeker, with the ratio of advisor words to user words ranging from nearly two to one up to more than three to one. Furthermore, all four models tended to jump to problem-solving before the user had finished explaining their situation or expressing their feelings. This behavior runs counter to what experts know about crisis support, where the priority is to let the person feel heard and calm down before offering any solutions. The researchers found that this tendency to be verbose and solution-oriented was a shared failure mode, not a quirk of a single model.

Despite this shared flaw, the study did find that the models could be distinguished from one another based on the quality of their support. One model, DeepSeek-V3.2, performed slightly better than the others, showing a greater ability to support the user's sense of autonomy and competence while generating fewer instances of counterproductive behaviors. However, the difference between the top performers was narrow, and none of the models achieved a level of performance that would be considered perfect. The research also confirmed that the synthetic help-seekers were successful: the independent auditors were able to accurately identify the specific psychological traits of the help-seekers just by reading the conversation, proving that these digital characters could reliably portray complex human personalities.

The findings suggest that while current artificial intelligence systems are capable of engaging in complex, multi-turn conversations, they have not yet learned the crucial skill of restraint required for crisis support. They are still too eager to speak and solve, a trait that may be helpful in many contexts but is harmful when a person is in acute distress and has a limited capacity to process information. The researchers concluded that for these tools to be truly safe and effective in high-stakes situations, they need to be recalibrated to prioritize listening and brevity, mirroring the patience of a skilled human counselor rather than the efficiency of a search engine. This work provides a new, safe way to test and improve these systems before they are deployed to help real people in their most vulnerable moments.

Technical Summary: Personality-Verified Synthetic Help-Seekers and LLM Crisis Support

Problem Statement
Large Language Models (LLMs) are increasingly consulted during moments of acute distress, such as receiving a difficult medical diagnosis. However, current evaluation methods rely heavily on single-turn benchmarks that assess factual accuracy but fail to capture the dynamics of sustained, emotionally charged conversations. In crisis scenarios, acute stress narrows attention and reduces working memory, necessitating a support style that prioritizes listening and stabilization over immediate problem-solving. Existing safety benchmarks are often static, rapidly becoming obsolete, and fail to account for how an LLM's behavior evolves across multi-turn interactions or how it adapts to specific user psychological profiles. Furthermore, there is a lack of evidence regarding whether LLMs can effectively support users with diverse dispositional characteristics (e.g., coping styles, resilience, reactance) in high-stakes health contexts.

Methodology: The AAA Framework
The authors introduce the Advisee–Advisor–Auditor (AAA) framework, a role-separated architecture designed to simulate and evaluate crisis support safely at scale.

Advisees: Seven synthetic help-seekers were instantiated, each assigned a specific psychometric profile based on five dimensions critical to caregiver mental health trajectories: NEO-FFI (Big Five traits), CHIP (coping styles), CSES-8 (coping self-efficacy), Hong (psychological reactance), and ResQ-Care (resilience). These profiles were specified as score bands (low, medium, high) rather than descriptive adjectives.
Advisors: Four widely used LLMs acted as advisors: GPT-4.1, Claude-Sonnet-4.5, Mistral-Medium, and DeepSeek-V3.2.
Auditors: Independent LLMs evaluated the conversations. In Study 1, an ensemble of the four models acted as auditors to recover the Advisee's profile from dialogue alone. In Studies 2–4, a single high-capability auditor (gpt-5.4) evaluated outcomes.
Scenario: All interactions occurred in a controlled acute crisis scenario: a family caregiver learning of a relative's dementia diagnosis.
Experimental Design:
- Study 1: Tested whether specified profiles could be consistently reproduced by different instantiating models and recovered by blind auditors.
- Study 2: Measured emotional stabilization (Perceived Stress Scale, PANAS).
- Study 3: Assessed support for basic psychological needs (Autonomy, Competence, Relatedness) using the METUX TENS-Life scale.
- Study 4: Evaluated support quality using the Crisis and Emergency Intervention Skills Scale (CEISS), Motivational Interviewing Treatment Integrity (MITI) code, and a novel Behavioral Counter.

Key Contributions

Personality-Aware Evaluation: The paper demonstrates that synthetic Advisees can be assigned specific, non-lexical psychological profiles (including coping and self-efficacy) that are reliably recovered by independent auditors from dialogue alone. This validates the use of synthetic agents for personality-aware testing.
The Behavioral Counter: A new metric grounded in crisis intervention theory, motivational interviewing, and therapeutic relationship literature. It quantifies specific counterproductive behaviors, such as "premature problem-solving," "false understanding," "premature trust," and "verbosity" (measured by word count and talk-to-listen ratios).
Role Separation: The AAA framework decouples the roles of help-seeker, advisor, and evaluator, allowing for the isolation of variables (e.g., holding the Advisee model constant while varying the Advisor) and the safe simulation of adversarial clinical conditions.

Results

Profile Recovery (Study 1): Auditors successfully recovered specified profiles with high agreement (ICC(2,4) = 0.91). Correlations between specified bands and recovered scores were strong ( $r = 0.78$ ). This held true not only for lexical Big Five traits but also for appraisal-based constructs like reactance and self-efficacy, which were previously thought to be difficult to infer from text. An ensemble of auditors was required to achieve high reliability for internal constructs.
Emotional Stabilization (Study 2): All four models reduced perceived stress and negative affect to some degree, but no statistically significant differences were found between the Advisors on emotional outcomes after correcting for multiplicity. The Advisee's profile accounted for more variance in emotional change than the Advisor model did.
Psychological Needs (Study 3): Significant differences emerged in the support of basic psychological needs. DeepSeek-V3.2 and Claude-Sonnet-4.5 generally outperformed Mistral-Medium, which generated the highest frustration and lowest support across autonomy, competence, and relatedness.
Support Quality and Behavior (Study 4):
- DeepSeek-V3.2 achieved the highest composite score (65.0), followed by GPT-4.1 (59.9) and Claude-Sonnet-4.5 (59.6), with Mistral-Medium ranking last (34.8).
- Shared Failure Modes: Despite ranking differences, all four models exhibited the same critical failure modes:
  1. Verbosity: All Advisors talked more than they listened, with word ratios exceeding 1:1 (ranging from 1.79 to 3.26) and turn lengths averaging 280–590 words.
  2. Premature Problem-Solving: All models moved to solutions before fully exploring the user's situation, violating the "listen-before-solving" sequence required for crisis support.
  3. Lack of Emotional Stabilization: While they reduced stress, they failed to distinguish themselves on emotional outcomes, suggesting a ceiling effect in their ability to manage acute distress.

Significance and Claims
The paper claims that the AAA framework provides a reusable, safe, and personality-aware method for evaluating LLMs in acute crisis contexts. Its primary significance lies in exposing a systemic mismatch between current LLM behaviors and the cognitive constraints of distressed users. The authors argue that while models may pass static safety filters, they consistently fail in sustained interactions by being too verbose and too quick to advise, thereby imposing a processing load on users who have reduced attentional capacity.

The study concludes that response-level benchmarks are necessary but insufficient; multi-turn, personality-aware evaluations are required to detect these behavioral deficits. The findings suggest that calibrating LLMs toward brevity, active listening, and stabilization is a high-value target for improving crisis support systems. The authors emphasize that their results are a proof of concept using synthetic data and call for future work involving human validation and varied cultural contexts.

Technical Summary: Personality-Verified Synthetic Help-Seekers and LLM Crisis Support

More like this