Expressing Social Emotions: Misalignment Between LLMs and Human Cultural Emotion Norms

1. Problem Statement

The paper addresses a critical gap in the deployment of Large Language Models (LLMs) within cross-cultural contexts: the lack of alignment between LLM-generated emotional responses and actual human cultural norms regarding social emotions.

• Social Emotions: Unlike basic emotions (e.g., happiness, anger), social emotions serve specific relational functions. They are categorized as:
  • Engaging Emotions (EN): Foster interdependence and social cohesion (e.g., guilt, friendliness, closeness).
  • Disengaging Emotions (D-EN): Express individuality and independence (e.g., pride, anger, self-esteem).
• Cultural Divergence: Human studies (specifically Salvador et al., 2024) show distinct patterns:
  • Latin Americans (LA): Tend to express engaging emotions more strongly than disengaging ones (collectivist orientation).
  • European Americans (EA): Tend to express disengaging emotions more strongly than engaging ones (individualist orientation).
• The Issue: As LLMs are increasingly used to simulate human behavior in mental health, customer service, and social science research, their failure to replicate these nuanced cultural patterns poses risks. If an LLM acting as a cultural proxy provides advice or emotional responses that contradict a user's cultural norms, it can lead to ineffective interactions or active harm.

2. Methodology

The authors developed a psychologically informed evaluation framework to test six frontier LLMs against human ground truth data.

A. Data and Models

• Human Ground Truth: Based on a study by Salvador et al. [19] involving 598 participants (USA, Mexico, Chile). Participants rated 12 emotions (8 social, 4 basic) across 4 everyday situations on a 1–6 Likert scale.
• LLMs Evaluated: Six state-of-the-art models representing diverse origins and architectures:
  • DeepSeek R1, GPT-4o-mini, Gemini 2.5 Flash, Phi-4, Mistral 7B Instruct v0.3, and Qwen 3 32B.
• Experimental Setup:
  • Personas: Models were assigned cultural personas (White American, Mexican, Chilean) using verbatim descriptions from the human study.
  • Prompting: Prompts were administered in the native language (Spanish for LA, English for EA) to simulate natural interaction.
  • Sampling: Each prompt was sampled $n=190$ times to construct response distributions comparable to human sample sizes, ensuring statistical stability.

B. Evaluation Metrics

Instead of relying solely on mean scores, the study focused on distributional analysis:

1. Directional Alignment: Used the Mann-Whitney U test to determine if the direction of cultural differences (e.g., LA > EA) matched human findings.
2. Magnitude of Difference: Used Wasserstein distance ($W_1$) to quantify the magnitude of divergence between human and model distributions.
3. Diversity Analysis: Calculated the Effective Number of Categories ($N_{eff}$) using Shannon entropy to measure response homogeneity vs. human variability.
4. Ablation Studies:
   • Temperature: Tested if increasing sampling temperature improved diversity and alignment.
   • Language: Tested LA personas with English prompts vs. Spanish prompts.
   • Task Format: Switched from Likert-scale rating to pairwise forced-choice tasks to see if explicit comparison improved alignment.

3. Key Results

A. Systematic Misalignment

All six models exhibited a systematic failure to reproduce human cultural patterns:

• The "Engaging" Bias: Regardless of the assigned persona (EA or LA), all models expressed engaging emotions significantly more strongly than disengaging emotions.
• The EA Anomaly: This was most stark for the European American (EA) persona. Humans express disengaging emotions more than engaging ones, but models reversed this, making EA personas appear highly interdependent (expressing guilt/friendliness) rather than independent. This occurred despite EA culture being heavily represented in training data.
• Cross-Cultural Reversal: Humans show higher overall expressivity for LA than EA. Most models (except Qwen and Mistral) showed the reverse trend, assigning higher intensity to EA personas.

B. Determinism and Homogeneity

• Low Variance: LLM responses were highly concentrated (modal) on specific rating points (often 4 or 5), whereas human responses were smoothly distributed across the 1–6 scale.
• Effective Categories: Humans utilized ~5 effective categories; LLMs utilized only ~2–3.
• Inter-Model Homogeneity: Different model families (e.g., GPT vs. Mistral) converged on similar, incorrect response patterns. The average LLM-LLM correlation (0.72) was much higher than LLM-Human correlation (0.47).

C. Ablation Findings

• Temperature: Increasing temperature slightly increased diversity ($N_{eff}$) for some models (notably Qwen) but did not improve cultural alignment. In some cases, higher temperature made alignment worse.
• Language Paradox: Surprisingly, prompting LA personas in English (rather than Spanish) improved cultural alignment. This suggests that cultural knowledge regarding Latin American norms is better encoded in the English portions of training corpora than in native Spanish corpora.
• Task Format: Pairwise forced-choice tasks improved alignment on overall expressivity (LA > EA) but failed to resolve the fundamental misalignment regarding the Engaging vs. Disengaging distinction for EA personas.

4. Key Contributions

1. Psychologically Grounded Framework: Introduced a rigorous method for evaluating LLMs on social emotions (interdependence vs. independence) rather than just basic emotion classification.
2. Discovery of "Cultural Sycophancy": Identified that models, even when assigned an individualist persona (EA), default to expressing high levels of interdependence (engaging emotions). The authors hypothesize this is due to RLHF tuning that prioritizes "polite" or "agreeable" (engaging) responses.
3. Language-Culture Decoupling: Demonstrated that for LLMs, "multilingual" does not equal "multicultural." Cultural alignment for Latin American personas was actually better in English prompts than in Spanish.
4. Distributional Critique: Highlighted that LLMs fail to simulate the variance of human populations, producing deterministic, homogeneous outputs that lack the natural diversity required for realistic social simulation.

5. Significance and Implications

• High-Stakes Deployment: The findings warn against using current LLMs as proxies for human participants in social science research or as agents in mental health and customer service without significant realignment. A model that misrepresents cultural norms (e.g., telling an individualist user they should feel guilty for asserting independence) can cause psychological harm or erode trust.
• Training Data Bias: The results suggest that current training corpora and RLHF processes encode a "WEIRD" (Western, Educated, Industrialized, Rich, Democratic) bias that overrides even explicit cultural persona instructions.
• Future Directions: The paper calls for "depth-auditing" of models across cultures, moving beyond broad benchmarks to deep, nuanced evaluations of social-emotional reasoning. It emphasizes that cultural alignment cannot be solved merely by adding more languages to the training set; the underlying representational biases regarding social norms must be addressed.

In conclusion, the paper provides empirical evidence that current frontier LLMs are culturally misaligned in their expression of social emotions, exhibiting a universal bias toward interdependence that contradicts human cultural norms, particularly for individualist cultures. This misalignment is robust to temperature changes and task formats, indicating a deep-seated representational error in current AI architectures.