💬 NLP

Learning Moral Diversity: Modelling Individual Perspectives in Moral Classification of Texts

This paper proposes a method for modeling individual annotator perspectives in moral text classification by extending pretrained language models with annotator-specific features, demonstrating that accounting for subjective disagreement yields more accurate predictions and deeper insights than traditional approaches that aggregate labels into a single ground truth.

Original authors: Yi Ren, Lewis Mitchell, Matthew Roughan

Published 2026-06-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yi Ren, Lewis Mitchell, Matthew Roughan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand how people feel about a specific news story or a tweet. You ask 23 different friends to read it and tell you if it's "good," "bad," or "neutral" based on their personal moral compass.

In the past, researchers would take all 23 answers, mash them together, and say, "Okay, the majority says it's 'bad,' so that's the truth." They treated the disagreement between friends as just "noise" or mistakes to be ignored.

This paper argues that ignoring the disagreement is the real mistake.

Here is a simple breakdown of what the researchers did, using some everyday analogies:

1. The Problem: The "Average" Friend Doesn't Exist

The researchers looked at a huge collection of tweets where many people had labeled them with moral values (like "Fairness," "Loyalty," or "Purity"). They noticed that people often disagreed wildly. One person might see a tweet as a violation of "Authority," while another sees it as a champion of "Fairness."

Standard AI models tried to find a single "average" answer. The researchers say this is like trying to describe a complex painting by only looking at the average color of all the pixels. You lose the details, the texture, and the specific intent of the artist. By forcing a single "ground truth," the AI misses the rich, messy reality of how humans actually think.

2. The Solution: Giving the AI "Personalities"

The team built a new type of AI model. Think of a standard AI model as a generic translator that speaks one universal language.

Their new model is like a translator who has 23 different "personalities" or "masks."

They took a powerful, pre-trained AI (called BERT) that is already good at reading text.
They added a special "Annotator Layer" on top of it.
This layer acts like a personal filter for each of the 23 friends. It learns that "Friend #5" tends to be very strict about "Purity," while "Friend #12" is very sensitive to "Care."

Instead of asking, "What is the true moral value of this tweet?", the model asks, "What would Friend #5 say about this tweet?" and "What would Friend #12 say?"

3. The Results: Better Guesses and New Insights

When they tested this new model:

It got better at guessing individual opinions: The model became significantly more accurate at predicting what a specific person would label a tweet as, compared to the old "average" model. It improved accuracy by about 10%.
It revealed hidden patterns: Because the model learned the "personalities" of the annotators, the researchers could look at the data and see things like: "Oh, this group of friends always sees tweets as 'moral violations,' while that group is very cautious and rarely labels anything as moral."
The "Aggregated" Trap: When they tried to force their new, highly personalized model to give a single "average" answer (to mimic the old way), its performance actually dropped. This proved that the old way of averaging labels was hiding the fact that people simply see the world differently. The "average" answer wasn't necessarily the "best" answer; it was just a compromise that lost the nuance.

4. The Big Takeaway

The paper concludes that disagreement isn't a bug; it's a feature.

In the world of moral judgment, people don't just make mistakes; they have different perspectives. By building AI that respects and learns from these individual perspectives, we get a much clearer picture of human morality. The researchers suggest that future AI shouldn't just try to find the "one right answer," but should instead learn to understand the diverse ways people interpret the world.

In short: They taught an AI to stop pretending everyone thinks the same way, and instead, to learn how each specific person thinks. The result was a smarter, more honest model that understands the messy reality of human opinion.

Technical Summary: Learning Moral Diversity

Problem Statement

The paper addresses the challenge of moral value classification in text, specifically within the framework of Moral Foundations Theory (MFT). While supervised Natural Language Processing (NLP) models trained on crowdsourced data have achieved strong performance, standard approaches typically aggregate multiple annotators' labels into a single "ground truth." The authors argue that this practice overlooks the inherent subjectivity of moral judgment. In tasks involving short social media texts (e.g., tweets), disagreements between annotators are not merely unstructured noise but reflect genuine variations in personal viewpoints, cultural contexts, and the ambiguity of the text itself. By forcing a consensus, existing models may hide meaningful variation and provide a misleading impression of performance, failing to capture the nuances of how different individuals interpret moral expressions.

Methodology

The authors propose a modeling approach that explicitly captures individual annotator perspectives rather than aggregating them. The core methodology involves extending a fine-tuned BERT classifier with a specialized Annotator Layer.

Data Structure: The study utilizes the Moral Foundations Twitter Corpus (MFTC), containing 35,108 tweets annotated by 23 individuals. Each tweet is labeled for five moral foundations (Authority, Care, Fairness, Loyalty, Purity) and their polarity (virtue, vice, or absent). The data exhibits significant sparsity and low inter-annotator agreement (Krippendorff's $\alpha$ ranges from 0.25 to 0.40), with no single annotator labeling the entire dataset.
Model Architecture: The model retains a pre-trained BERT encoder to generate contextual embeddings ( $h_i$ $h_{i}$ ) for the input text. A projection layer maps these embeddings to a lower-dimensional space. Crucially, an Annotator Layer is introduced to adjust the logits based on the specific annotator ( $a_j$ $a_{j}$ ) providing the label. The authors implement two variants:
- Bias-only Annotator Layer (BO-AL): Applies a learned bias matrix ( $B$ ) where each row represents an annotator's specific bias toward the three classes (virtue, vice, absent). This variant aims for interpretability with fewer parameters.
- Linear Transformation Annotator Layer (LT-AL): Employs a linear transformation for each annotator, mapping the projected text features to the output logits via a unique weight matrix and bias vector. This variant offers greater representational power to model complex individual patterns.
Training Objective: The models are trained using cross-entropy loss on individual annotations (tuples of text, annotator, and label). Regularization includes L2 weight decay and a centered bias penalty for the BO-AL variant to ensure the average annotator has zero bias.

Key Contributions

The paper makes three primary contributions:

Modeling Individual Perspectives: It introduces a framework that treats annotator-specific features as learnable signals rather than noise, explicitly accounting for task subjectivity in moral classification.
Demonstrating Learnability of Bias: The authors show that annotator biases and tendencies are learnable features. Their model achieves an average 10.2% improvement in macro F1 score (across five foundations) when predicting individual annotations compared to a standard fine-tuned BERT baseline.
Critique of Aggregation: The work highlights the limitations of aggregating annotations into a single ground truth. It demonstrates that models trained on aggregated labels may appear highly accurate while masking the underlying inconsistencies and subjectivity inherent in the data.

Results

Performance on Raw Annotations: Both Annotator Layer variants (BO-AL and LT-AL) outperformed the baseline FT-BERT model in predicting individual annotations. The LT-AL variant yielded the most significant gains, particularly for the "binding" foundations (Authority, Loyalty, Purity), which exhibited higher ambiguity and lower inter-annotator agreement.
Class-Specific Improvements: The most substantial classification improvements occurred in the moral classes (virtue and vice) rather than the "absent" class. For example, the LT-AL model improved the F1 score for the "Purity" virtue class by 0.265 over the baseline.
Interpretability: The bias matrix extracted from the BO-AL model revealed distinct "annotator types." Some annotators consistently favored virtue or vice labels, while others were more cautious, frequently selecting the "absent" label. These patterns varied across different foundations, suggesting that annotator bias is not universal but context-dependent.
The Aggregation Paradox: When the authors evaluated the LT-AL model on aggregated labels (by averaging predictions across all annotators), performance decreased significantly compared to the baseline. This indicates that a model optimized to capture individual perspectives does not necessarily align with a consensus label, and that high performance on aggregated labels may obscure the model's inability to capture the true diversity of human judgment.

Significance and Claims

The paper claims that disagreement in moral classification tasks is a learnable feature that reflects the inherent subjectivity of the domain. By modeling individual perspectives, the authors demonstrate that:

Standard aggregation methods can lead to a "smoothed out" view of reality that misses inherent conflicts and subtleties of human experience.
Annotator biases are not just noise to be removed but contain meaningful information about how different individuals interpret moral content.
Future research should move beyond training universal classifiers for a single "ground truth" and instead develop models that reflect the diversity of moral judgment.

The authors modestly note limitations, specifically that they could not correlate learned annotator features with external metadata (e.g., political ideology) due to data unavailability, and that their current approach does not yet provide a robust mechanism for generating high-quality aggregated predictions for downstream applications that require a single label. They suggest that principled aggregation methods leveraging these learned features remain a vital area for future work.