Technical Summary: Ensemble Diversity Optimization for Subjective Supervision

Problem Statement

Many Natural Language Processing (NLP) tasks, such as content moderation, hate speech detection, and sentiment analysis, are characterized by systematic annotator disagreement. In these subjective domains, divergent annotations often stem from semantic ambiguity, contextual dependence, or varying annotator expertise rather than simple annotation errors. Consequently, the "ground truth" is not a single latent label but a distribution over plausible human interpretations.

Standard supervised learning approaches typically aggregate these annotations into a single target (e.g., via majority vote), discarding distributional information and inducing overfitting to dominant interpretations. While soft-label supervision preserves annotator distributions, existing methods often optimize a single predictive model, lacking explicit mechanisms to regulate internal predictive variability or distinguish between genuine epistemic uncertainty (subjectivity) and structural noise (e.g., class imbalance or sparse annotator coverage). Furthermore, traditional ensemble methods often treat diversity as an emergent property of fixed architectures rather than an explicit optimization objective.

Methodology: Ensemble Diversity Optimization (EDO)

The authors propose Ensemble Diversity Optimization (EDO), a prediction-space framework designed to learn under subjective and imbalanced supervision. EDO operates entirely in the prediction space, allowing it to be model-agnostic and compatible with any set of pre-trained probabilistic classifiers. The framework freezes backbone parameters and focuses on learning ensemble weights and structure end-to-end.

Core Components

1. Differentiable Ensemble Structure Learning:
   Instead of fixing the ensemble size, EDO learns the effective cardinality ($K$) and the reliability weights ($w$) of ensemble members jointly. This is achieved using Gumbel-Softmax relaxation, which allows for gradient-based optimization of discrete structural choices (ensemble size) within a continuous framework.

2. Signed, Reliability-Weighted Diversity Regularizer:
   A central innovation is the introduction of a signed diversity objective ($L^{(s)}_{Div}$) where $s \in \{-1, +1\}$.

   • $s = -1$ (Disagreement Preservation): Encourages intra-ensemble disagreement to preserve epistemic uncertainty when variation reflects genuine subjectivity.
   • $s = +1$ (Disagreement Suppression): Suppresses divergence when the observed variation is driven by structural artifacts, such as extreme class imbalance or sparse annotator coverage.
     This regularizer is reliability-weighted, meaning the contribution of pairwise disagreement is modulated by the learned weights of the ensemble members.

3. Multi-Objective Optimization:
   EDO formulates learning as a joint optimization over three competing objectives:

   • Predictive Utility: Approximated via a differentiable soft micro-F1 loss ($L_{F1}$).
   • Calibration: Enforced via class-weighted cross-entropy ($L_{CE}$) to handle imbalanced soft labels and align with annotator distributions.
   • Internal Diversity: Regulated by the signed diversity loss ($L^{(s)}_{Div}$).
     The total objective is: $\min_{w,K} \lambda_{F1}L_{F1} + \lambda_{CE}L_{CE} + \lambda_{Div}L^{(s)}_{Div} + \lambda_{Reg}L_{Reg}$.

4. Supervision Strategies:
   The framework supports two strategies:

   • Random Select: Samples a single annotator's label per instance for training, providing an unbiased stochastic estimator.
   • Per-Annotator: Pairs specific ensemble members with specific annotators, preserving individual biases and requiring access to annotator identity.

Key Contributions

• Unified Framework: EDO integrates stochastic annotator supervision with explicit, reliability-aware optimization of ensemble diversity, moving beyond fixed architectures or single-model soft-label approaches.
• Directional Diversity Control: Unlike previous methods that treat diversity as a static heuristic, EDO introduces a signed regularizer that steers optimization along the utility-calibration Pareto frontier, allowing the model to either preserve or suppress disagreement based on the nature of the data.
• End-to-End Structural Learning: By utilizing Gumbel-Softmax, EDO learns the optimal ensemble size and composition dynamically, adapting complexity to the specific patterns of annotator disagreement in a dataset.
• Model Agnosticism: The framework operates solely in prediction space, making it applicable to any pre-trained backbone without requiring retraining of the underlying model or access to annotator metadata (in the Random Select variant).

Experimental Results

The authors evaluated EDO on four subjective text-classification benchmarks from the LeWiDi benchmark: ArMIS (Arabic misogyny), ConvAbuse (English dialogue abuse), HS-Brexit (English Brexit-related hate speech), and MD-Agreement (English offensiveness).

• Calibration Improvements: EDO substantially improved probabilistic calibration compared to baselines (Soft-CE, Soft-MD, Top-5 Voting, and Weak Ensemble Learning [WEL]).
  • It reduced Cross-Entropy (CE) by 40–78% relative to baselines depending on the dataset and baseline.
  • It achieved the lowest Soft Brier Scores across all four datasets, indicating robustness against overconfident miscalibration.
• Utility Maintenance: Despite the focus on calibration and uncertainty, EDO maintained competitive Micro-F1 scores, demonstrating that improved uncertainty representation does not necessarily come at the cost of predictive utility.
• Alignment with Annotators: EDO showed stronger alignment with empirical annotator distributions (measured by Manhattan Distance) than single-model approaches.
• Ablation Insights:
  • The signed diversity regularizer was most effective when combined with the joint multi-objective loss, acting as a complementary mechanism to reshape the utility-calibration trade-off.
  • Reliability-weighted ensemble aggregation consistently reduced CE compared to uniform averaging.
  • The learned ensemble size ($K$) adapted dynamically, expanding when additional members improved calibration and contracting when unnecessary.

Significance and Claims

The paper claims that EDO provides an efficient, model-agnostic approach for modeling human subjectivity in supervised learning. By treating annotator disagreement as a structured signal rather than noise, and by jointly optimizing ensemble structure with a directional diversity regularizer, EDO enables controlled navigation of the utility-calibration trade-off.

The authors emphasize that their approach does not require hand-designed ensemble architectures or precomputed uncertainty scores. Instead, reliability weights and ensemble size emerge implicitly from the gradient-based balancing of utility, calibration, and diversity objectives. The results suggest that jointly optimizing ensemble structure allows for a more principled representation of epistemic uncertainty in settings with inherently variable human judgments.

The paper concludes that while EDO offers a flexible framework, its effectiveness depends on the structure of the disagreement (e.g., distinguishing genuine subjectivity from structural artifacts), and future work could explore dynamic weighting strategies and richer representations of annotator behavior.