💬 NLP

Beyond Sentiment Classification: A Generative Framework for Emotion Intensity Evaluation in Text

This paper proposes a novel generative framework that shifts emotion analysis from discrete classification to continuous intensity evaluation (0–100), demonstrating superior performance and generalization in domains like finance where the degree of emotional content is critical for decision-making.

Original authors: Francesco A. Fabozzi, Dasol Kim, William N. Goetzmann

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Francesco A. Fabozzi, Dasol Kim, William N. Goetzmann

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand the mood of a crowd. For years, the standard way to do this with computers was like playing a game of "Spot the Emotion." You'd show the computer a sentence, and it would have to shout out a single label: "That's Anger!" or "That's Joy!"

The problem with this approach, according to this paper, is that it's too black-and-white. It's like a light switch that is either ON or OFF. But human emotions aren't light switches; they are dimmer switches. A sentence can be a little bit angry, or furious, or somewhere in between. In fields like finance, knowing that someone is scared isn't as useful as knowing how terrified they are.

The authors, researchers from Yale and the Office of Financial Research, propose a new way to teach computers: instead of asking "What emotion is this?", ask "How strong is this emotion?"

Here is how they did it, explained through a few simple analogies:

1. The New "Volume Knob" Dataset

To teach the computer this new skill, the researchers didn't just give it labels. They created a special training manual (a dataset) of 1,177 short phrases.

Imagine two expert judges sitting in a room. For every phrase, they didn't just pick a label. Instead, they turned a volume knob for different emotions.

Anger: They turned the knob to 15 (low) or 90 (high).
Sadness: They turned it to 0 (none) or 100 (maximum).
Valence (Good vs. Bad): They used a slider from -100 (very bad) to +100 (very good).
Arousal (Calm vs. Excited): They used a slider from 0 (sleeping) to 100 (jumping up and down).

This created a rich map of human feeling, not just a list of categories.

2. The "Smart Student" vs. The "Flashcard Student"

The researchers tested two types of AI "students" to see who could learn this new skill better.

The Flashcard Student (Old Models): These are the traditional AI models (like RoBERTa). They were trained like a student memorizing flashcards: "If you see word X, the answer is Anger." The paper found these students struggled when asked to give a specific number. They were good at spotting the emotion, but bad at measuring its intensity. It's like a student who knows the word "fear" but can't tell you if the person is slightly nervous or screaming in terror.
The Smart Student (Generative LLMs): These are the newer, larger AI models (Mistral-7B and Mistral-24B). Instead of memorizing flashcards, these models were "fine-tuned" using the volume knob dataset. The researchers taught them to write a report that says: "Anger is at 45, Sadness is at 10, and Arousal is at 80."

The Result: The "Smart Student" was much better. It didn't just guess the emotion; it accurately measured the strength of the feeling. Even better, the larger version (Mistral-24B) was the best at this task, outperforming even the most advanced AI models that hadn't been trained on this specific data yet.

3. The "Magic Transfer" Trick

Here is the most surprising part of the paper, which the authors call a "transfer effect."

Imagine you teach a student how to measure the volume of a drum (Anger) and a volume of a violin (Sadness). You never explicitly teach them how to measure the temperature of the room (Valence) or how fast the music is playing (Arousal).

However, because the student learned to understand the intensity of the specific instruments so well, they accidentally became very good at guessing the temperature and speed of the music too!

The paper found that when they trained the AI only on the specific emotions (Anger, Fear, etc.), the model automatically got really good at predicting Valence (how positive/negative the text is) and Arousal (how excited/calm it is), even though it never saw those specific labels during training. It seems the AI learned the underlying "physics" of emotion, allowing it to figure out the other dimensions on its own.

4. The "Blind Test"

To prove the AI wasn't just memorizing the answers, the researchers did a "blind test." They took a specific emotion (like "Anxiety") and removed all examples of it from the training data. They then asked the AI to rate "Anxiety" in new sentences.

The AI still did a great job. It was like a chef who had never cooked a specific dish before but, having mastered the basics of spices and heat, could still make it taste delicious. This suggests the AI learned a general rule for "how to measure emotion" rather than just memorizing a list of words.

The Bottom Line

The paper argues that for real-world applications (especially in finance, where the degree of emotion matters for decision-making), we need to stop treating emotions like a multiple-choice test.

Instead, we should treat them like a sound mixing board. By using large, generative AI models trained to output specific numbers (0 to 100) for how strong an emotion is, we get a much clearer, more useful picture of human feeling. This approach captures the nuance that the old "Yes/No" classification methods miss.

Technical Summary: Beyond Sentiment Classification: A Generative Framework for Emotion Intensity Evaluation in Text

1. Problem Statement

Current affective Natural Language Processing (NLP) predominantly treats emotion modeling as a discrete classification problem, aiming to identify the presence or absence of categorical emotions (e.g., anger, joy) within a text. While effective for detection, this paradigm obscures the intensity of emotional expression—a critical variable in applied domains such as finance, behavioral science, and psychology where the magnitude of emotion, rather than its mere existence, drives interpretation and decision-making.

Furthermore, existing benchmarks and models often fail to jointly model the circumplex dimensions of affect: valence (positive-negative evaluation) and arousal (intensity of activation). Most encoder-only architectures (e.g., BERT, RoBERTa) are fine-tuned for classification objectives and lack the representational flexibility to output continuous, graded intensity scores across multiple emotions and dimensions simultaneously.

2. Methodology

Dataset Construction

The authors constructed a novel dataset of 1,177 short conversational phrases to support fine-grained intensity modeling.

Annotation: Two native English speakers independently rated each phrase on a continuous scale.
Dimensions: The dataset includes scores for eight emotion categories (anger, anxiety, fear, sadness, disgust, optimism, excitement, surprise), valence (–100 to +100), and arousal (0 to 100).
Scale: Emotions are scored 0–100 (0 = absence, 100 = maximum intensity).
Quality Control: Final scores are the average of two independent annotations, with inter-annotator agreement verified.

Model Architecture and Training

The study proposes a shift from classification to structured text generation for intensity evaluation.

Models: Two open-weight, instruction-tuned generative Large Language Models (LLMs) were utilized: Mistral-7B and Mistral-24B.
Fine-tuning: Both models were adapted using Low-Rank Adaptation (LoRA) (rank $r=16$ , $\alpha=32$ ) applied to query and value projection layers.
Task Formulation: The models are trained to map input text $x$ to a JSON output containing real-valued predictions for all eight emotions, valence, and arousal.
Training Setup: Data was split into training (706), validation (176), and test (295) sets. Training utilized 4-bit quantization and the AdamW optimizer for 10 epochs.
Baselines: Performance was compared against:
- Zero-shot pretrained LLMs (GPT-4, GPT-4o, Mistral-7B, Mistral-24B).
- Fine-tuned RoBERTa-based regressors (trained per emotion with MSE loss).
- The widely used samLowe/roberta-base-go_emotions classification model.

Evaluation Metrics

To address the limitations of standard regression metrics, the authors employed:

Concordance Correlation Coefficient (CCC): Measures both ranking consistency and accuracy in scale, penalizing systematic errors in mean and variance.
Zero-Match F1-Score: A binary classification metric evaluating the model's ability to correctly identify the complete absence of an emotion (score = 0), crucial for minimizing false positives in downstream applications.

3. Key Results

Performance Comparison

Generative vs. Encoder Models: Fine-tuned generative models significantly outperformed both zero-shot LLMs and RoBERTa-based baselines.
- Mistral-24B-FT achieved the highest CCC in 8 out of 10 categories, surpassing GPT-4 across all emotions.
- RoBERTa Limitations: Even fine-tuned RoBERTa regressors struggled to capture graded nuance, with most CCC values remaining below 50%. The samLowe classification model showed inconsistent alignment and failed to predict valence/arousal.
Intensity Gains: Fine-tuning yielded substantial improvements, particularly for difficult categories like Fear (CCC increased from 58.0% to 80.5%) and Surprise (41.6% to 62.3%).

Generalization to Unseen Emotions

In a Leave-One-Out (LOO) experiment, models were trained on $k-1$ emotions and tested on the held-out emotion.

Mistral-24B demonstrated strong generalization, with LOO scores (e.g., 70.6% for Fear) approaching fully supervised performance (80.5%) and far exceeding zero-shot baselines (58.0%).
Mistral-7B showed more variability, occasionally failing to structure outputs correctly for unseen emotions, suggesting that model scale is a critical factor in generalization robustness.

Transfer to Affective Dimensions

The study tested whether training only on discrete emotion intensities (excluding valence and arousal from supervision) would transfer to these dimensions.

Mistral-24B achieved remarkable transfer: without direct supervision, it reached 95.2% CCC on Valence and 67.5% on Arousal, nearly matching the fully supervised model.
This indicates that affective dimensions are learned as latent representations through correlations with specific emotion labels.

4. Contributions and Significance

The paper argues for a paradigm shift in affective NLP from discrete classification to continuous intensity evaluation. Its primary contributions include:

A Unified Framework: The introduction of a generative framework that simultaneously outputs calibrated intensity scores for multiple emotions and affective dimensions (valence/arousal) in a single inference step.
Superior Expressiveness: Demonstrating that fine-tuned generative models capture emotional magnitude and nuance more effectively than traditional encoder-only classifiers, which often fail in continuous regression tasks.
Generalization and Transfer: Showing that models trained on specific emotion intensities can generalize to unseen emotion categories and implicitly learn abstract affective dimensions (valence/arousal) without direct supervision.
Domain Alignment: Positioning this approach as better aligned with the needs of applied fields (e.g., finance) where the degree of emotional content is central to interpretation, rather than just the binary presence of an emotion.

The authors conclude that while challenges remain for subtle emotions like "Surprise" and "Disgust," and dataset subjectivity is a factor, this generative approach offers a more flexible, interpretable, and scientifically robust foundation for affective analysis in interdisciplinary contexts.