💬 NLP

How sensitive do we want AI to be? Socio-communicative competencies of large language models in healthcare

This study evaluates the socio-communicative competencies of three large language models (GPT-4o, Llama 3, and Command R+) in healthcare dialogues using the IC-MD instrument, finding that while they exhibit non-hostility, they lack the consistent sensitivity, non-intrusiveness, and structuring skills necessary for safe and effective use as healthcare advisors.

Original authors: Dorothee Amelung, Andrew M. Bean, Sabine C. Herpertz, Felix H. Krones, Guy Parsons, Adam Mahdi, Isabella Schneider

Published 2026-08-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Dorothee Amelung, Andrew M. Bean, Sabine C. Herpertz, Felix H. Krones, Guy Parsons, Adam Mahdi, Isabella Schneider

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking into a library where the librarian is an incredibly smart robot. This robot has read every medical textbook ever written and can recite facts about diseases faster than you can blink. But here's the catch: being a good doctor isn't just about knowing facts; it's about how you talk to people when they are scared, confused, or in pain. This field of study is called "socio-communicative competence." It's the art of listening, showing empathy, organizing a messy conversation, and making the other person feel safe. We care about this because if we start using these super-smart robots to help us with our health, we need to know if they can handle the human side of medicine, not just the math side. If a robot gives you the right answer but makes you feel ignored or panicked, is it really helping?

This is exactly what a team of researchers from Heidelberg University and the University of Oxford wanted to find out. They decided to put three popular AI chatbots—GPT-4o, Llama 3, and Command R+—through a "social skills test" using 1,800 real conversations where people asked for medical help. The researchers acted like strict but fair coaches, grading the robots on four specific traits: Non-hostility (being polite and not rude), Sensitivity (showing empathy and understanding feelings), Non-intrusiveness (letting the user make their own choices without being bossy), and Structuring (keeping the conversation organized and guiding the user step-by-step).

The results were a bit of a mixed bag, like a student who aces the math test but forgets to say "please." The robots were excellent at being polite; they were almost never rude or hostile, earning high marks for Non-hostility. They were also generally okay at Non-intrusiveness, meaning they usually let the user drive the conversation. However, they stumbled badly when it came to Sensitivity. While they could say "I'm sorry you're sick," they often missed the emotional cues that required a deeper connection, making the chats feel like a dry exchange of facts rather than a caring conversation.

The biggest problem, however, was Structuring. Imagine trying to solve a puzzle where the robot hands you pieces one by one but never tells you how they fit together, or worse, changes the picture halfway through without explaining why. In the study, the robots often failed to ask the right questions to get a clear picture of the problem. Instead of guiding the user through a logical path to figure out what was wrong, the robots dumped information and left the user to do all the heavy lifting of organizing it. In one case, a robot told a user their symptoms were "likely short-lived," and then later, without explaining the change, called the same symptoms "a cause for concern," leaving the user confused and anxious.

The researchers concluded that while these AI tools are getting better at medical facts, they currently lack the reliable social skills needed to be safe, effective healthcare advisors on their own. They aren't ready to replace a doctor's "bedside manner" because they struggle to build a trusting relationship or guide a conversation with a steady hand. The study suggests that if we want to use these robots in healthcare, we can't just teach them to be more polite; we need to teach them how to be better listeners and organizers, and we need to remember that a robot's job is different from a human doctor's. Until they learn to structure a conversation as well as they structure a sentence, they should be seen as helpful tools, not as the main person you talk to when you're feeling unwell.

Technical Summary: Socio-communicative Competencies of Large Language Models in Healthcare

Problem Statement
Effective clinical practice relies heavily on the socio-communicative skills of medical professionals, particularly sensitivity, empathy, and the ability to structure interactions. While Large Language Models (LLMs) are increasingly proposed for healthcare tasks such as triaging, report drafting, and translating medical jargon, their deployment requires both factual accuracy and social competence. Although LLMs have shown significant improvement in medical accuracy on licensing benchmarks, less attention has been paid to their socio-communicative behaviors. Current research often focuses on scenarios where LLMs directly replace physicians, but there is a need to evaluate LLMs acting as preliminary information-retrieval tools within the healthcare system. This study addresses the gap in understanding how well current public-facing LLMs meet the socio-communicative expectations placed on healthcare advisors.

Methodology
The researchers conducted a qualitative evaluation using a subset of the HELP-Med dataset, which comprises 1,800 conversation transcripts between human participants and three distinct LLMs: GPT-4o, Llama 3 70B, and Command R+.

Data Selection: From the full dataset, the authors selected two specific medical scenarios. For each model-scenario pair, they extracted two conversations: one where the participant correctly identified the next steps and one where they did not, resulting in a total of 12 dialogues for analysis.
Evaluation Instrument: The study utilized an adapted version of the Interactional Competencies – Medical Doctors (IC-MD) instrument. Originally designed to evaluate medical student admissions via video, the IC-MD assesses four global scores: Sensitivity, Structuring, Non-intrusiveness, and Non-hostility.
Adaptation for Text: Since the dataset is text-only, the evaluation criteria were modified to focus on observable textual features. A simplified three-point scale (Negative/-, Neutral/0, Positive/+) was applied to each competency.
Rating Process: Two independent experts, blinded to the specific LLM identity and participant outcomes, rated the dialogues. Initial inter-rater reliability was moderate to high (average Kendall's Tau $r_\tau = .75$ ). Discrepancies, particularly regarding the interpretation of "hostility" in text (e.g., non-responses), were reconciled through discussion to establish detailed criteria for the text-based setting.

Key Contributions

Application of Clinical Frameworks to AI: The study adapts a validated instrument (IC-MD) used for human medical professionals to evaluate the socio-communicative performance of LLMs in a medical information-seeking context.
Granular Competency Analysis: Rather than a general sentiment analysis, the paper breaks down "bedside manner" into four specific, clinically relevant dimensions: sensitivity, structuring, non-intrusiveness, and non-hostility.
Differentiation of Roles: The work highlights the distinction between the social expectations of a human doctor versus an AI tool, suggesting that desirable behaviors may differ based on the agent's functional role (e.g., an AI's "safety-netting" behavior differs from a doctor's).

Results
The analysis of the 12 dialogues revealed distinct performance patterns across the four competencies:

Non-hostility: LLMs performed strongly in this category. They generally maintained polite and respectful language. Negative ratings were rare and only applied when the model failed to engage with relevant questions or repeatedly deflected to other sources without providing actual help, which could be interpreted as indifferent or hostile.
Non-intrusiveness: Results were mixed. Models generally preserved user autonomy by using recommendations rather than commands. However, they struggled with active involvement, a key component of patient-centered care where the provider actively engages the patient to build shared understanding.
Sensitivity: Performance was mixed. Interactions were predominantly polite and fact-based but often lacked affective displays or empathic validation. While models occasionally used phrases like "sorry to hear," they often failed to apply empathetic language at critical moments (e.g., when delivering distressing information) or failed to address the emotional impact of changing their own assessments.
Structuring: This was the area of largest deficit. LLMs rarely demonstrated the ability to organize interactions efficiently. Unlike human doctors who proactively ask targeted questions to assess severity, prioritize information, and build a shared understanding of symptoms and next steps, LLMs often provided disjointed information, leaving the user to organize the conversation and assess urgency. Case studies illustrated instances where models failed to follow up on their own hypotheses or ignored user attempts to structure the dialogue.

Significance and Claims
The paper concludes that current LLMs lack the consistent and reliable socio-communicative skills necessary for safe and effective use as healthcare advisors.

Risk of Overload: The primary risk identified is that the lack of structuring forces medical laypeople, who are often already in distress, to organize disjointed information and regulate their own emotional states simultaneously. This can be overwhelming and may negatively impact patient safety and commitment.
Role-Specific Expectations: The authors argue that while existing frameworks for interactional competencies can guide development, they must be adapted. The "ideal" behavior for an LLM differs from that of a human doctor; for instance, an LLM's refusal to give medical advice is a safety feature, whereas a human doctor's refusal might be considered hostile.
Future Development: The study suggests that rather than training LLMs to produce superficial apologies or pretend emotions (which some users find offensive), development should focus on "sensitive behaviors" such as the appropriate collection and provision of contextual information to balance emotionally impactful statements. Furthermore, LLMs must be trained to proactively ask clarifying questions to build shared understanding, a skill currently missing in public-facing models.

The authors maintain a modest stance, noting that their study uses simulated scenarios with lower stakes than real-world clinical interactions and does not include a direct comparison with practicing physicians performing the same tasks. However, they assert that the identified weaknesses in structuring and sensitivity are critical barriers to the deployment of LLMs as healthcare information tools.

Technical Summary: Socio-communicative Competencies of Large Language Models in Healthcare

More like this