Large language models show fragile cognitive reasoning about human emotions

This paper introduces CoRE, a benchmark based on cognitive appraisal theory, to reveal that while large language models capture systematic relations between cognitive appraisals and emotions, they exhibit misalignment with human judgments and instability across contexts, indicating fragile cognitive reasoning about human emotions.

The Gist

The Big Idea: The "Emotion Detective" vs. The "Emotion Actor"

Imagine you hire a very smart robot detective to solve a mystery about why a person is crying.

• The Old Way: You ask the robot, "Is this person sad?" The robot looks at its training manual, sees a picture of a crying face, and says, "Yes, that's Sadness." It's good at matching pictures to labels.
• The New Way (This Paper): You ask the robot, "Why do you think they are sad?" You want it to explain the reasoning: "They are sad because they lost something they loved, they feel responsible for it, and they can't control the outcome."

This paper asks: Do Large Language Models (LLMs) actually understand the "Why" (the reasoning), or are they just really good at guessing the "What" (the label)?

The answer is a bit scary: They are great at guessing the label, but their internal reasoning is "fragile," inconsistent, and sometimes completely different from how humans think.

---

The Experiment: The "Emotion Gym"

The researchers built a massive gym called CoRE (Cognitive Reasoning for Emotions). They didn't just ask the AI, "What emotion is this?" Instead, they made the AI do a workout with 17 different mental muscles (called cognitive appraisals).

Think of these muscles as questions a human asks themselves when something happens:

• Was this pleasant?
• Did I cause this?
• Was it fair?
• How much effort did I have to use?
• Was I certain about what would happen?

They tested 6 different AI models (like GPT, LLaMA, Gemini) on nearly 70,000 scenarios to see how they used these "muscles" to figure out emotions.

The Findings: Where the Robots Go Wrong

Here are the three main ways the AI's "emotional brain" is different from a human's:

1. The "Effort" Trap (The Over-Exercising Robot)

Humans have a balanced way of thinking. If we feel "Pride," we think, "I worked hard for this." If we feel "Happiness," we think, "This feels good."

• The AI Glitch: The AI models treat Effort as the most important thing for almost every emotion. It's like a robot that thinks every problem in life is solved by "working harder."
• The Metaphor: Imagine a human chef who thinks every dish needs extra salt to taste good. If the dish is sweet, salty, or spicy, the chef just dumps in more salt. The AI does this with "Effort." It overuses this one concept, even when humans wouldn't.

2. The "Secret Identity" Problem (The Lying Detective)

The researchers asked the AI two things:

1. Implicitly: "Show me your work." (We looked at the numbers the AI generated).
2. Explicitly: "Tell me what is most important." (We asked the AI to pick one factor).

• The Glitch: The AI's "work" showed it was heavily relying on Effort and Problems. But when asked directly, it said, "Oh no, I rely on Responsibility and Control."
• The Metaphor: It's like a student who studies the whole textbook (the implicit reasoning) but, when the teacher asks, "What was the most important thing you learned?", they confidently say, "The cover page." The AI doesn't even know what it's actually thinking.

3. The "Cultural Blindness" (The One-Size-Fits-All Suit)

Emotions are different depending on who you are and where you are from. A situation that makes a person in Japan feel "shame" might make a person in the US feel "anger."

• The Glitch: When the researchers told the AI to "act like a person from Japan" or "act like a person from Mexico," the AI's emotional reasoning didn't change at all.
• The Metaphor: Imagine a weather app that says "It's raining" no matter if you are in a desert or a rainforest. The AI has a "universal" emotional suit that fits everyone, but it's too loose to fit any specific culture.
• The Exception: However, if you told the AI to "act like a grumpy person" vs. "act like a happy person" (personality), it did change its reasoning. It understands individual moods, but not cultural backgrounds.

The "Fragility" of the AI Mind

The paper concludes that AI emotional reasoning is fragile.

• Simple is Easy: If you ask the AI to distinguish between "Happy" (good stuff) and "Sad" (bad stuff), it does great. It understands the basic "Good vs. Bad" split.
• Complex is Broken: If you ask it to distinguish between "Shame" and "Guilt" (which are very similar but have different rules), the AI gets confused. It starts mixing them up or using the wrong logic.

Why Does This Matter?

If we use AI to help with mental health, teach kids, or run customer service, we need them to understand why we feel the way we do, not just what we feel.

• The Risk: If an AI thinks "Anger" is always caused by "Unfairness" (which is true for humans), but it doesn't understand that in some cultures, anger is about "Loss of Face," it might give terrible advice.
• The Future: We can't just train AI to say the right words. We need to train them to have a "human-like" internal structure, where they understand the complex web of reasons behind our feelings, not just the surface label.

Summary in One Sentence

Current AI models are like actors who have memorized the script for "Sadness" and "Anger" perfectly, but they don't actually understand the psychology behind the lines, and they often get confused when the scene changes to a different culture or a complex situation.

Technical Summary

1. Problem Statement

While Large Language Models (LLMs) have demonstrated proficiency in recognizing surface-level emotional labels (e.g., classifying text as "happy" or "sad"), it remains unclear whether they possess cognitive reasoning capabilities regarding emotions. Current evaluations often rely on supervised learning with discrete labels, which fails to probe the underlying cognitive structures (appraisals) that humans use to generate emotional responses.

The core problem is that LLMs may be mimicking statistical correlations between stimuli and labels without truly understanding the cognitive appraisal theory—the psychological framework suggesting that emotions arise from how an individual interprets a stimulus along specific dimensions (e.g., goal congruence, responsibility, pleasantness). The authors investigate whether LLMs can reason about emotions through these underlying cognitive dimensions rather than just surface patterns, and whether their internal reasoning aligns with human cognitive structures, remains consistent across contexts, and generalizes across models.

2. Methodology

The CoRE Benchmark

The authors introduce CoRE (Cognitive, Reasoning for Emotions), a large-scale benchmark designed to probe the implicit cognitive structures of LLMs.

• Dataset: Comprises ~74,800 prompts derived from 274 unique emotional scenarios covering 15 discrete emotion categories (e.g., Happiness, Anger, Shame, Pride).
• Dimensions: Each scenario is paired with 17 cognitive appraisal dimensions based on a seminal human study [38], grouped into 8 main categories: Pleasantness, Attentional Activity, Control, Certainty, Goal-Path, Obstacle, Legitimacy, Responsibility, Anticipated Effort, and Understanding.
• Models Evaluated: Six frontier LLMs were tested, including proprietary models (GPT-o4-mini, Gemini 2.5 Flash) and open-source models (DeepSeek R1, LLaMA 3 8B, Phi 4 14B, QwQ 32B).

Experimental Design

1. Implicit Appraisal Analysis: Models were prompted to rate the 17 dimensions for each scenario on a Likert scale. The authors analyzed the resulting distributions using:
   • Principal Component Analysis (PCA): To identify latent cognitive axes and compare variance explained against human baselines.
   • Wasserstein Distance: To measure the similarity of emotion distributions within and across models.
   • Logistic Regression & SynthTree: To determine which appraisal dimensions predict specific emotions and to analyze hierarchical decision structures.
2. Explicit Value Probing: A choice-based task where models were asked to select the single most important dimension for a scenario. This tested the consistency between a model's stated priorities and its implicit behavior.
3. Contextual Robustness: The study introduced in-context personas to test sensitivity to:
   • Culture: Nationality (USA, Mexico, Nigeria, Denmark, Japan).
   • Personality: Big Five traits (Agreeableness, Conscientiousness, Extraversion, Neuroticism, Openness) grouped into "positive" and "negative" personas.

3. Key Contributions

• First Large-Scale Self-Appraisal Benchmark: Unlike prior work focusing on "other-appraisal" (predicting others' emotions), CoRE focuses on cognitive self-appraisal, isolating the model's internal reasoning from social stereotypes or demographic assumptions.
• Systematic Analysis of Cognitive Structures: The paper provides the first systematic comparison of LLMs' implicit appraisal structures against human cognitive appraisal theory across diverse model families.
• Contextual Sensitivity Study: It presents the first analysis of how cultural and personality-based contexts influence LLM emotional reasoning, revealing a stark contrast between personality adaptation and cultural rigidity.
• Identification of "Value-Action Gaps": The study uncovers significant discrepancies between what models claim is important (explicit) and what actually drives their reasoning (implicit).

4. Key Results

A. Misalignment in Cognitive Dimensions

• Effort vs. Fairness: While humans treat "Effort" as a distinct, lower-variance factor, LLMs (except QwQ) treat Effort as highly correlated with "Problem" and "Pleasantness," suggesting an unnatural clustering. Conversely, "Perceived Fairness," a major driver in human anger, explains almost no variance in most LLMs' ratings.
• Variance Explained: Proprietary models (e.g., GPT-o4) show coherent, low-dimensional structures (80% variance explained by top 6 PCs), closely matching humans. Open-source models (e.g., LLaMA 3) exhibit diffuse, unstructured reasoning, requiring 9+ components to match human explanatory power.

B. The Value-Action Gap

• Explicit vs. Implicit: When asked explicitly which dimension is most important, models consistently prioritize Responsibility and Control. However, their implicit ratings show that Effort and Problem are actually the dominant drivers of their downstream reasoning. This indicates a lack of introspective consistency.

C. Distributional Fragility

• Valence Dominance: LLMs primarily separate emotions based on Valence (Positive vs. Negative). They lack the nuanced separation seen in humans for complex emotions like "Challenge" and "Surprise," which often blur together in model representations.
• Cross-Model Inconsistency: While models agree on simple emotions (Happiness, Fear), they diverge significantly on complex ones (Anger, Frustration, Shame). LLaMA 3, in particular, shows high divergence from all other models.

D. Predictive Performance

• Task Difficulty: Models predict "Challenge" and "Boredom" well but fail to reliably distinguish "Sadness," "Disgust," and "Shame."
• Granularity: Performance drops significantly as emotion labels become finer-grained. When emotions are coarsened into 4 Valence-Arousal quadrants, predictive stability and interpretability improve drastically, suggesting LLMs struggle with fine-grained emotional distinctions.

E. Contextual Robustness

• Culture: LLMs showed no statistically significant variation in appraisal distributions across different cultural personas (USA, Japan, etc.), indicating a failure to capture cultural nuance in emotional reasoning.
• Personality: LLMs showed significant, psychologically plausible shifts based on personality personas. "Positive" personas increased self-control and certainty, while "Negative" personas increased perceived obstacles and external control.

5. Significance and Implications

• Fragility of Emotional AI: The study concludes that LLM emotional reasoning is structured yet noisy. While they can mimic human patterns superficially, their internal cognitive structures are fragile, inconsistent, and often misaligned with human psychology.
• Limitations for High-Stakes Applications: The inability to capture cultural nuance and the inconsistency in reasoning about complex emotions (like anger or shame) pose risks for applications in mental health triage, conflict resolution, or personalized therapy.
• Future Directions: The authors argue that current supervised learning paradigms are insufficient for building robust emotional world models. They suggest exploring evolutionary or developmental learning and latent space alignment to foster genuine cognitive reasoning rather than surface-level pattern matching.

In summary, the paper demonstrates that while LLMs can process emotional data, they currently lack the robust, coherent, and culturally adaptive cognitive reasoning required for true emotional intelligence, relying instead on brittle heuristics that often diverge from human psychological reality.