When Avatars Have Personality: Effects on Engagement and Communication in Immersive Medical Training

Problem Statement

While Virtual Reality (VR) has become a powerful tool for simulating physical environments and procedural skills in medical education, it currently lacks the psychological fidelity required to train complex interpersonal competencies. Existing VR simulations often rely on static, scripted interactions that fail to capture the nuances of human communication, such as personality, emotional expressiveness, and social dynamics. This gap is critical in medical training, where a physician's ability to adapt communication strategies to diverse patient personalities (e.g., anxious, resistant, or cooperative) is a core clinical competency. The absence of psychologically plausible virtual humans limits the effectiveness of VR in replicating authentic clinical encounters.

Methodology

The authors propose and evaluate a framework that integrates Large Language Models (LLMs) into immersive VR to create virtual patients with distinct, consistent personalities and medically coherent symptoms.

1. System Architecture
The system utilizes a modular architecture that decouples clinical data from behavioral traits to ensure both medical accuracy and personality consistency:

• VR Environment: Built using Unreal Engine, the environment simulates a high-fidelity outpatient office. It supports real-time speech-to-text (STT), LLM processing, and text-to-speech (TTS) in Brazilian Portuguese, with low-latency lip-syncing for the avatar.
• Virtual Patient Modeling: Each agent is initialized with four distinct components:
  • Patient Identity: Enforces role consistency, first-person perspective, and progressive information disclosure.
  • Backstory: Generated from real patient inquiries, providing demographic and biographical context (e.g., occupation, family history).
  • Personality Profile: Based on the Big Five theory, five distinct profiles were defined (e.g., Introverted/Irritable, Extroverted/Anxious, Introverted/Distrustful). These profiles dictate communicative style, emotional tone, and verbosity.
  • Disease Card: A structured component encoding specific medical conditions (e.g., GERD, Dengue) to ensure symptom disclosure remains medically accurate and consistent with the condition.
• Interaction Loop: The physician's audio is transcribed, processed by the LLM (conditioned on the patient's profile and disease card), and converted back to speech via TTS.

2. Experimental Design
The study employed a mixed-methods, within-subjects design involving four licensed physicians.

• Scenarios: Participants engaged in two simulated consultations: one with an extroverted/anxious patient (GERD) and one with an introverted/calm patient (Dengue). The order was counterbalanced.
• Metrics: Data was collected via pre- and post-interaction questionnaires, including the Simulator Sickness Questionnaire (SSQ), User Engagement Scale (UES-SF), NASA-TLX (workload), and specific Likert-scale ratings for realism, personality perception, and training value.
• Synthetic Data Analysis: To assess robustness at scale, the authors generated 3,000 simulated patient responses (500 consultations × 6 turns) covering five diseases and five personality profiles. These were evaluated using an "LLM-as-judge" approach to score Personality Adherence and Medical Coherence.

Key Results

1. User Study Findings

• Feasibility and Engagement: Physicians found the system feasible and highly rewarding. The "Reward Factor" score was exceptionally high (M=4.83), indicating that interacting with personality-driven agents was perceived as valuable and interesting.
• Cognitive Load: The simulation induced a high mental demand (M=4.0) consistent with clinical reasoning tasks, while physical demand and frustration remained low, suggesting the technology did not interfere with the learning process.
• Personality Perception: The system successfully portrayed foundational traits like introversion (M=4.5) and extroversion (M=3.75). However, high-arousal emotional states like anxiety showed high variance in physician ratings, suggesting difficulty in consistently conveying complex emotions through dialogue alone.
• The "Realism-Verbosity Paradox": A critical finding emerged where the more talkative (extroverted) patient was perceived as more realistic and human-like, while the terse (introverted) patient, despite being role-consistent, was perceived as more robotic and less natural. This suggests that less communicative behavior can paradoxically increase the perception of artificiality.
• Training Value: While the introverted patient posed a greater communication challenge, physicians did not rate this challenge as more instructive or memorable. This indicates that for training to be effective, the difficulty must be perceived as authentic rather than artificial.

2. Synthetic Data Analysis

• Medical Coherence: The framework achieved high consistency (87.5% overall) in maintaining disease-specific facts. Performance was highest for common conditions (Depression, Otitis, GERD) and lower for regionally specific or complex conditions (Dengue, Headache), likely due to training data distribution in the underlying LLM.
• Personality Consistency: The system robustly modeled positive or neutral traits (e.g., Extroverted/Confident). However, it struggled with negative-valence introverted profiles (e.g., Introverted/Irritable, Introverted/Distrustful), often failing to distinguish them from incorrect profiles. This reinforces the "realism-verbosity paradox," where limited dialogue makes it difficult to convey nuanced antagonistic emotions.

Key Contributions

The paper claims three primary contributions:

1. A Novel Framework: A modular architecture for LLM-driven virtual patients that decouples personality from clinical data, enabling controlled manipulation of social variables in VR.
2. Design Principles: The identification of the "realism-verbosity paradox" and the distinction between "authentic" and "artificial" difficulty. The authors argue that effective training simulations must present challenges that feel human-like rather than like system limitations.
3. Empirical Validation: Preliminary evidence that personality diversity significantly enhances the perceived value and effectiveness of medical training simulations compared to uniform, scripted encounters.

Significance and Claims

The paper positions this work as a step toward bridging the gap between physical and psychological fidelity in VR medical training. The authors claim that while perfecting individual agent realism remains a challenge, the integration of personality diversity is a critical advancement for creating impactful training environments.

The study concludes that the approach is technically feasible and perceived as a rewarding enhancement to medical education. However, the authors remain modest regarding the generalizability of the results due to the small sample size (four physicians) and the exploratory nature of the study. They acknowledge that the current system struggles with high-arousal emotions and regionally specific medical knowledge, suggesting that future iterations will require improved prompt engineering, non-verbal animation integration, and fine-tuning with local medical data to achieve robust, scalable believability.