💬 NLP

Fine-Grained Perspectives: Modeling Explanations with Annotator-Specific Rationales

This paper proposes a framework that jointly models annotator-specific label predictions and explanations by leveraging a User Passport mechanism and two distinct explainer architectures, demonstrating that integrating fine-grained rationales significantly improves predictive performance and provides a richer representation of perspective-based disagreement.

Original authors: Olufunke O. Sarumi, Charles Welch, Daniel Braun

Published 2026-04-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Olufunke O. Sarumi, Charles Welch, Daniel Braun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand why a group of people disagree about a movie.

In the old days of Artificial Intelligence (AI), if 10 people watched a movie and 6 said "It's good" while 4 said "It's bad," the AI would just pick the majority vote and say, "The movie is good." It would ignore the 4 people who disagreed.

But in the real world, those 4 people aren't "wrong." They just have different backgrounds, experiences, or ways of thinking. This paper is about teaching AI to respect those differences and, more importantly, to understand why they disagree.

Here is a simple breakdown of what the researchers did, using some everyday analogies.

1. The Problem: The "Group Vote" vs. The "Individual Voice"

Most AI systems act like a strict teacher who only accepts the "correct" answer. But in tasks like reading a story and deciding if two sentences match (called Natural Language Inference), people often have valid reasons for different answers.

The Old Way: The AI looks at the text and guesses the "average" answer. It treats the explanation (the "why") as an afterthought, like a student trying to justify a wrong answer after the fact.
The New Way: The researchers wanted the AI to act like a mediator. Instead of just picking a winner, the AI should say, "Okay, Person A thinks this is true because of X, and Person B thinks it's false because of Y."

2. The Secret Sauce: The "User Passport"

To do this, the AI needs to know who is talking. The researchers gave the AI a User Passport for every person it interacts with.

Think of this like a VIP wristband at a music festival.

The wristband doesn't just say "You are here."
It says, "You are a 22-year-old student from China," or "You are a 33-year-old professor from Germany."
When the AI reads a sentence, it looks at the wristband first. It realizes, "Ah, this person has a specific background, so they might interpret this joke differently than someone else."

By giving the AI this "passport," it learns to predict what a specific person would think, rather than what the "average" person thinks.

3. The Two Methods: The "Post-It Note" vs. The "Direct Wire"

The researchers tried two different ways to teach the AI to explain why a specific person thinks a certain way.

Method A: The "Post-It Note" (Post-hoc Explainer)

Imagine you ask a friend to guess the answer to a riddle. After they guess, you ask them to write a note explaining their logic.

How it works: The AI guesses the answer first. Then, it looks at the text and the "User Passport" and writes an explanation.
The Flaw: The explanation is written after the guess. It's like trying to justify a decision you already made. The AI might write a very fluent, grammatically perfect explanation that sounds good but doesn't perfectly match the deep logic it used to make the guess. It's like a lawyer writing a defense speech after the verdict is already in.

Method B: The "Direct Wire" (Prefixed Bridge Explainer)

Now, imagine the friend has a direct wire connecting their brain to the microphone.

How it works: As the AI is thinking about the answer, it sends those raw thoughts (the electrical signals in its brain) directly to the part that writes the explanation.
The Benefit: The explanation isn't written later; it's generated from the actual thinking process. It's like the friend is speaking their thoughts out loud as they happen.
The Result: The researchers found this method was much better. The explanations were more honest, more consistent with the answer, and captured the true "flavor" of that specific person's perspective.

4. The Results: Why It Matters

The researchers tested this on a dataset where 4 different people annotated the same 500 sentences.

The "Direct Wire" (Prefixed Bridge) won. It was better at predicting what a specific person would say and, crucially, it generated explanations that felt more "faithful" to that person's actual reasoning.
The "Post-It Note" (Post-hoc) was good at sounding like the original text (using similar words), but it sometimes missed the deeper meaning.

The Big Picture Takeaway

This paper is a step toward Empathetic AI.

Instead of building a robot that tries to be "neutral" by averaging out human opinions, this research builds a robot that understands human diversity. It acknowledges that:

Two people can look at the same fact and see two different truths.
Both truths are valid based on who that person is.
To truly understand human language, AI needs to listen to the reasoning, not just the answer.

In short: They taught the AI to stop acting like a referee who just counts votes, and start acting like a translator who understands the unique dialect and logic of every single person in the room.

1. Problem Statement

Perspectivist Natural Language Processing (NLP) argues that annotations should reflect individual annotator judgments rather than a single consensus label. While current approaches model these perspectives using disaggregated labels and metadata (e.g., user IDs, demographics), they often treat explanations (rationales) as secondary or post-hoc justifications.

The Gap: Existing methods rarely integrate annotator-specific rationales directly into the modeling process. Consequently, the "why" behind a label remains abstracted, and models fail to capture the fine-grained reasoning diversity that leads to disagreement.
The Goal: To develop a framework that jointly models label prediction and explanation generation, conditioned explicitly on the specific annotator's identity and reasoning, treating explanations as primary expressions of perspective rather than just interpretability tools.

2. Methodology

The authors propose a framework utilizing the VariErrNLI dataset, which contains disaggregated Natural Language Inference (NLI) labels and annotator-provided explanations. The framework consists of two main components:

A. Annotator-Aware Classification (The "User Passport")

Instead of modifying input text, the model incorporates annotator identity at the representation level:

Backbone: DeBERTa-v3-base.
Fusion Mechanism: The model creates a fused representation ( $z_{ij}$ $z_{ij}$ ) by concatenating:
1. The pooled contextual representation of the input ( $h$ ).
2. A learnable embedding vector for the specific annotator ( $u_j$ ).
3. A projected vector of the annotator's demographic metadata ( $m_j$ ).
Objective: Predict the specific label set ( $y_a$ ) for each annotator $a$ , handling missing annotations via masking.

B. Annotator-Conditioned Explanation Modeling

The paper compares two distinct architectures for generating rationales ( $r_a$ ):

Post-hoc Prompt-Based Explainer:
- Architecture: Flan-T5 (Encoder-Decoder).
- Mechanism: Generates explanations based on a text-only prompt containing the context, statement, gold labels, and an "annotator control token" (a special token linked to the annotator ID).
- Limitation: There is no differentiable connection between the classifier and the explainer; the explainer relies solely on surface-level textual prompts.
Prefixed Bridge Explainer (Novel Contribution):
- Architecture: A hybrid of the classifier and Flan-T5.
- Mechanism:
  - The classifier produces a fused internal representation ( $z_{ij}$ ).
  - A small neural network (Prefixed Bridge, a 2-layer MLP) projects this vector into a sequence of prefix embeddings.
  - These embeddings are prepended to the T5 encoder's input before generation.
- Advantage: This creates a tight, differentiable coupling where the explanation generation is directly conditioned on the classifier's internal, annotator-specific reasoning signals, rather than just text prompts.

3. Key Contributions

Joint Modeling Framework: The first approach to explicitly condition explanation generation on annotator-specific internal representations, moving beyond treating explanations as post-hoc rationalizations.
Prefixed Bridge Architecture: Introduces a novel method to transfer classifier representations directly into a generative model via prefix embeddings, enabling the model to leverage deep reasoning signals for generation.
Perspectivist Explanation: Reframes explanations as expressions of fine-grained perspective, demonstrating that modeling them improves the fidelity of disagreement representation.
Empirical Validation: Provides a comprehensive evaluation on the VariErrNLI dataset, analyzing performance across different annotator demographics and reasoning styles.

4. Results

Experiments were conducted on the VariErrNLI dataset (4 annotators, ~1,900 label-explanation pairs).

Predictive Performance:
- Incorporating explanation modeling significantly boosted performance over the baseline "User Passport" classifier (Macro-F1: 70.5).
- Prefixed Bridge Explainer: Achieved the highest Macro-F1 (93.9) and Exact Match (92.4), indicating superior label alignment.
- Post-hoc Explainer: Achieved Macro-F1 of 92.3, slightly lower than the bridge model.
Explanation Quality:
- Lexical Similarity (ROUGE-L): The Post-hoc approach yielded slightly higher scores, suggesting better surface-level overlap with reference texts.
- Semantic Similarity: The Prefixed Bridge approach achieved higher semantic similarity scores, indicating its generated explanations were more meaningfully aligned with the gold rationale, even if the wording differed.
Annotator-Specific Analysis:
- The Prefixed Bridge model showed more stable performance across diverse annotators (e.g., Ann1 and Ann4), particularly when reasoning patterns varied.
- The Post-hoc model performed exceptionally well for Ann2 (a Postdoc with high education), likely due to that annotator's highly structured and text-aligned reasoning.
- Faithfulness: Qualitative analysis showed the Prefixed Bridge produced more concise, inference-grounded explanations, whereas the Post-hoc model tended to be more verbose and occasionally introduced information not present in the context.

5. Significance and Conclusion

Theoretical Impact: The study validates that modeling explanations as expressions of perspective provides a richer representation of human disagreement than label-only approaches. It demonstrates that "why" an annotator chose a label is as critical as the label itself.
Methodological Advance: The Prefixed Bridge architecture proves that transferring internal classifier representations to generative models yields more semantically consistent and faithful outputs than prompt-based methods alone.
Future Directions: The authors note limitations regarding the small number of annotators and the dataset's original design for error detection rather than controlled demographic variation. Future work aims to expand datasets to include bilingual/native speaker comparisons and explore ensemble methods combining both explainer types.

In summary, this paper advances perspectivist NLP by demonstrating that integrating annotator-specific rationales into both predictive and generative components leads to more robust, transparent, and faithful models of human diversity in NLP tasks.