Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMs

This study employs clinical vignettes devoid of explicit emotion keywords to demonstrate that large language models possess a dissociable dual mechanism where early-layer circuits reliably detect general emotional significance (affect reception) regardless of keywords, while specific emotion categorization remains partially dependent on lexical cues.

The Gist

Imagine you are trying to teach a robot to understand human feelings. For a long time, researchers have been checking if the robot has "emotion circuits" by showing it sentences like, "I am devastated," or "She was furious." When the robot correctly identifies these as sad or angry, the researchers say, "Great! It understands emotions!"

But this paper asks a tricky question: Is the robot actually understanding the feeling, or is it just spotting the words?

It's like a student who passes a math test only because they memorized the word "plus" means addition, but if you give them a problem written in a different language or without the word "plus," they fail. Did they learn math, or just the vocabulary?

The Experiment: The "Empty Kitchen Table" Test

To find the answer, the author (a researcher who is also a clinical psychologist) designed a special test. Instead of using sentences with emotional words, they used clinical vignettes—short stories that describe a situation without ever using an emotion word.

The Test Case:

"A kitchen table set for two, as usual. One plate untouched, the coffee cold. Across from her seat, his photo and a small urn."

There are no words like "sad," "grief," or "lonely." There are no keywords. Just a cold cup of coffee and an empty chair. A human reads this and instantly feels the sadness. The question was: Does the AI feel it too, or is it blind without the word "sad"?

They tested this on six different AI models (ranging from small to medium-sized) using four different scientific methods to peek inside the AI's "brain."

The Big Discovery: Two Different Brains in One

The researchers found that AI doesn't have just one "emotion detector." It actually has two separate systems working together, like a security guard and a detective.

1. The Security Guard (Affect Reception)

• What it does: It answers the question, "Is something emotional happening here?"
• How it works: This system is incredibly powerful. It works perfectly (100% accuracy) even when there are no emotion words.
• The Analogy: Imagine a smoke detector. It doesn't need to know why the smoke is there (a candle? a fire? a burnt toast?) to know that something is burning. It just knows, "Alert! Something significant is happening!"
• The Result: The AI can look at that cold coffee and empty chair and immediately know, "This is emotionally heavy." It doesn't need the word "sad" to trigger this alarm. This happens very early in the AI's processing, almost instantly.

2. The Detective (Emotion Categorization)

• What it does: It answers the question, "Exactly what emotion is this? Is it grief? Anger? Fear?"
• How it works: This system is a bit more fragile. It works well, but it likes having keywords. When the AI has to guess the specific emotion without words like "devastated" or "furious," it gets slightly less accurate.
• The Analogy: This is the detective trying to solve the crime. The smoke detector (Security Guard) says, "There's smoke!" The Detective says, "Okay, but was it arson or a burnt toast?" If the Detective has no clues (keywords), they might guess wrong.
• The Result: Without keywords, the AI is still pretty good at guessing the specific emotion, but it's not perfect. It relies a bit more on the words to be sure.

The "Size Matters" Surprise

The study also looked at how the size of the AI model changes things.

• Small Models (The 1B parameter models): They are like a detective who relies heavily on a checklist. If you take away the keywords, they get confused. They can still sense something is wrong (Security Guard), but they struggle to name it.
• Larger Models (The 8B and 9B parameter models): These are like seasoned detectives. They have so much experience that they can figure out the specific emotion even without the keywords. As the models get bigger, they become better at understanding the context rather than just the words.

Why This Matters

This paper changes how we think about AI safety and understanding.

1. AI is not just a "Keyword Spotter": We used to worry that AI was just matching words to feelings. This proves that AI can actually understand the situation and the meaning behind a story, even without the "trigger words."
2. Safety Implications: If you are building an AI to help people in crisis (like a suicide prevention chatbot), you might think, "If the user doesn't say 'I am sad,' the AI won't help." This paper says: Wrong. The AI's "Security Guard" will still detect the distress from the context (the cold coffee, the empty chair) and can trigger a helpful response, even if the user is too scared to use the specific words.
3. Better Testing: The paper argues that to truly test if AI understands emotions, we shouldn't just use word-heavy tests. We need to use these "clinical vignettes" (stories without emotion words) to see if the AI really gets the meaning.

The Bottom Line

The AI is smarter than we thought, but also more complex. It has a super-sensitive alarm system that detects emotional situations instantly, without needing any specific words. Then, it has a labeling system that tries to name the feeling, which works best when it has some words to help it out.

The next time you see a story about a cold cup of coffee and an empty chair, remember: the AI sees that picture, feels the weight of the silence, and knows something is wrong—even if you never told it the word "sad."

Technical Summary

1. Problem Statement

Recent mechanistic interpretability studies have claimed that Large Language Models (LLMs) possess structured internal "emotion circuits" capable of detecting and categorizing emotions. However, a critical methodological flaw plagues this literature: all existing studies use stimuli where emotional content is signaled by explicit emotion keywords (e.g., "devastated," "furious").

This creates a keyword confound: it is unclear whether these circuits detect genuine emotional meaning derived from situational context or merely act as lexical feature detectors for specific emotion words. The paper asks: Do LLMs detect emotional significance from context alone, or do they rely on the presence of emotion vocabulary?

2. Methodology

The authors introduce a rigorous clinical validity test using stimuli grounded in clinical psychology methodology, contrasting them with standard keyword-rich datasets.

A. Stimulus Design

• Set A (Keyword-Rich Baseline): 80 stimuli from existing datasets (e.g., crowd-enVENT) containing explicit emotion vocabulary. Used for replication.
• Set B (Clinical Vignettes): 96 novel, keyword-free stimuli designed by a clinical psychologist.
  • Construction: Emotions are evoked solely through situational and behavioral cues (e.g., an empty kitchen table with a cold coffee and an urn) without emotion words, sentiment phrases, or internal state descriptions.
  • Control: Cross-topic design (3 distinct domains per emotion) to ensure the model encodes emotion rather than topic.
  • Validation: Set B stimuli were confirmed to be "invisible" to lexicon-based sentiment analyzers (zero keyword hits).
• Models Tested: Six models across two families (Llama-3.2-1B, Llama-3-8B, Gemma-2-9B), including both Base and Instruct variants.

B. Mechanistic Interpretability Methods

The study employs four convergent methods to probe internal activations:

1. Linear Probing: Correlational analysis to decode emotion from residual streams, attention outputs, and FFN outputs.
2. Causal Activation Patching: Copying activations from a source example to a target example to test if the model's prediction shifts (causal confirmation).
3. Knockout Experiments: Zeroing out specific layers to test computational necessity and fragility.
4. Representational Geometry: Analyzing clustering (PCA, Silhouette scores) to determine if representations organize by emotion or by stimulus type (keyword presence).

3. Key Contributions

1. First Clinical Validity Test: The first study to test LLM emotion circuits using keyword-free, clinically designed stimuli, establishing a new external standard for emotion processing claims.
2. Discovery of a Mechanistic Dissociation: Identification of two distinct, separable mechanisms: Affect Reception and Emotion Categorization.
3. Convergent Evidence: The first study to apply all four major interpretability methods (probing, patching, knockout, geometry) to the same stimulus contrast.
4. Open Resource: Release of 96 clinical vignettes, extraction pipelines, and analysis code for community replication.

4. Key Results

A. Affect Reception (Binary Detection)

• Definition: The detection of whether content is emotionally significant (Emotional vs. Neutral).
• Findings:
  • Keyword Independence: Models achieve near-perfect accuracy (AUROC 1.000) on keyword-free clinical vignettes.
  • Early Saturation: This signal saturates in the early layers (9–38% of network depth), well before categorical classification peaks.
  • Robustness: This capability exists in Base models (pre-alignment) and scales from 1B to 9B parameters.
  • Causal Confirmation: Cross-set activation patching (injecting keyword-rich activations into keyword-free passes) transfers an "affective salience" signal that boosts categorization, proving the signal is causal and distinct from specific emotion categories.

B. Emotion Categorization (8-Class Mapping)

• Definition: The mapping of detected affect to specific emotion labels (e.g., Grief vs. Rage).
• Findings:
  • Partial Keyword Dependence: Performance drops by 1–7% on keyword-free stimuli compared to keyword-rich stimuli.
  • Scale Dependence: The drop is significant in small models (1B: ~6.6% drop) but minimal in larger models (8B/9B: ~1.1–1.9% drop).
  • Architectural Fragility: In small models (1B), keyword-free categorization is fragile, relying on a concentrated "bottleneck" (e.g., Layer 9 in Llama-1B). In larger models, the process becomes distributed across many layers, with no single point of failure.
  • Geometry Shift: Instruction tuning (Instruct models) reorganizes the representational geometry. While Base models cluster by stimulus type (keyword vs. no-keyword), Instruct models (specifically Llama-8B) cluster by emotion category across stimulus types, indicating a shift toward abstract, stimulus-invariant representations.

C. The "Keyword Cost"

The study quantifies the cost of removing keywords:

• Binary Detection: No cost (AUROC remains ~1.0).
• Categorization: A cost of 1–7% AUROC, which diminishes as model scale increases.
• Transfer Asymmetry: Probes trained on clinical (keyword-free) data generalize better to keyword-rich data than vice versa, suggesting keyword-free representations capture a purer, more generalizable emotion signal.

5. Significance and Implications

A. Theoretical Impact

• Falsifies the Keyword-Spotting Hypothesis: The study proves that LLMs do not merely detect emotion words; they genuinely encode emotional significance from situational context.
• Refines the "Emotion Circuit" Concept: Emotion processing is not a monolithic circuit but a dissociable two-stage process:
  1. Affect Reception: Rapid, robust, early-layer, keyword-independent detection of salience.
  2. Emotion Categorization: Slower, keyword-assisted, scale-dependent, and distributed mapping to specific labels.
• Alignment vs. Pre-training: Emotional encoding is a pre-training phenomenon. Instruction tuning (RLHF) does not create the ability to detect emotion but reorganizes the geometry of existing representations to make them more accessible and category-aligned.

B. AI Safety and Deployment

• Crisis Detection: Systems can detect emotional distress (e.g., self-harm risk) with high reliability even if users avoid explicit keywords, as the "affect reception" mechanism reads the situation.
• Adversarial Robustness: Users cannot bypass emotional safety filters simply by removing emotion words; the situational context still triggers the affect reception mechanism.
• System Design: The dissociation suggests a deployment architecture where Affect Reception (for triage/safety) can run on smaller models, while Emotion Categorization (for nuanced therapeutic response) benefits from larger, aligned models.

C. Methodological Shift

The paper advocates for the adoption of clinical stimulus methodology (keyword-free, situationally grounded vignettes) as a standard validity check in NLP, moving the field beyond crowd-sourced, keyword-heavy datasets to rigorously test genuine emotional understanding.