Density-Ratio Losses for Post-Hoc Learning to Defer

Problem Statement
Learning to Defer (L2D) addresses the challenge of deploying machine learning (ML) models in high-stakes domains (e.g., healthcare, finance) where reliability is paramount. The goal is to enable a model to abstain from making a prediction and instead defer to an expert (potentially human) when the model's output is deemed unreliable. While Learning to Reject (L2R) allows models to abstain based on their own confidence, L2D explicitly incorporates an expert into the training process to learn when the expert is likely to outperform the model.

A critical constraint in modern ML is the "post-hoc" setting, where a pre-trained base model $h$ is fixed, and only the deferral mechanism (rejector) $r$ is learned. Retraining large models is often prohibitively expensive. Existing post-hoc approaches often rely on simple heuristics (e.g., confidence thresholds) or attempt to estimate pointwise expert-comparison targets (predicting when the expert is correct and the model is not). However, these methods can fail under distribution shifts or when experts are specialized. Furthermore, many existing methods require retraining to adjust the deferral rate, limiting their adaptability.

Methodology
The paper proposes a novel framework for post-hoc L2D based on ideal distributions and density-ratio estimation.

1. Ideal Distributions: The authors define an "ideal distribution" as a divergence-regularized reweighting of the true data distribution $P$ under which a specific classifier-loss pair attains low loss. They introduce two types:

   • Marginal Ideal ($Q_m$): Optimized over the observation space $X$.
   • Joint Ideal ($Q_j$): Optimized over the joint space $X \times Y$.
     For the Kullback-Leibler (KL) divergence, these ideals are shown to be multiplicative reweightings of the data distribution, where the weights depend on the expected loss of the classifier (or expert) under the Bayes class-posterior.

2. Density-Ratio Deferral Rule: Instead of learning a binary rejector directly, the authors define deferral by comparing the ideal distributions of the model $h$ and the expert $h^{(e)}$. The deferral decision is made by thresholding the density ratio $\rho(x) = \frac{dQ_i(x)}{dQ^{(e)}_i(x)}$.

   • If the observation $x$ is better explained by the expert's ideal distribution (i.e., the ratio is low), the model defers.
   • This approach naturally separates the scorer (the density ratio estimator) from the deferral threshold, allowing the deferral rate to be adjusted post-training without retraining the scorer.

3. Theoretical Connections:

   • Chow's Rule: For marginal KL-ideals, the derived rule recovers the classic Chow's rule under the original data distribution.
   • Expert-Tilted Posterior: For joint KL-ideals, the rule corresponds to a Chow-style decision under an "expert-tilted" distribution. This distribution reweights labels based on the expert's loss, effectively incorporating the expert's comparative advantage into the deferral decision.
   • Hypothesis Testing: The deferral rule is framed as a Neyman-Pearson hypothesis test distinguishing between the model's ideal distribution and the expert's ideal distribution.

4. DR CPE Losses: Since the ideal distributions depend on the unknown Bayes posterior and require pointwise expert access (which is unavailable at test time), the authors cannot use closed-form solutions. Instead, they reduce Density-Ratio Estimation (DRE) to Class-Probability Estimation (CPE).

   • They derive a family of DR CPE losses ($L_m$ and $L_j$) that allow learning the density ratio between the model's and expert's ideal distributions via a binary classification task.
   • The Joint DR CPE loss ($L_j$) is shown to be practically estimable and serves as an upper bound to the marginal loss ($L_m$).
   • The paper establishes connections between this approach and existing expert-comparison estimation methods (e.g., regressing on the difference in zero-one accuracy) and anomaly detection.

Key Contributions

• Distributional Formulation: The paper introduces a post-hoc L2D framework based on comparing ideal distributions of the model and expert, rather than direct pointwise performance estimation.
• Marginal vs. Joint Ideals: It analyzes the distinction between marginal and joint ideal distributions, showing that the joint rule corresponds to deferral under an expert-tilted distribution, which is practically meaningful as it accounts for the expert's specific strengths.
• DR CPE Losses: The authors derive a new family of loss functions (DR CPE) that enable learning deferral rules via standard class-probability estimation, bridging the gap between theoretical ideal distributions and practical learning.
• Adaptability: The method decouples the scorer from the threshold, enabling the adjustment of deferral rates without retraining.

Experimental Results
The authors evaluate their DR CPE approach on CIFAR-100 and MedMNIST datasets under various corruption settings (label noise, long-tail distributions, and specialist subsets).

• Competitiveness: The DR CPE approach performs competitively against common baselines, including confidence-based heuristics, two-stage surrogate loss methods, and expert-comparison estimators.
• Robustness: The method demonstrates superior robustness across different dataset corruption combinations compared to baselines. For instance, while other methods struggle significantly on specialist subsets (e.g., DermaMNIST specialist), DR CPE maintains stable performance.
• Ablation: Experiments show that the choice of loss function (e.g., Generalized Cross-Entropy vs. Probabilistic Zero-One) and temperature parameter $\gamma$ affects performance, but the joint DR CPE approach remains robust.

Significance and Claims
The paper claims to cast post-hoc L2D as a problem of density-ratio learning between ideal distributions. This perspective:

• Bridges the gap between classical Chow-style rules and modern expert-comparison methods.
• Elucidates connections to related learning settings such as anomaly detection.
• Provides a theoretically grounded method that is practically robust and adaptable, addressing the limitations of retraining large models and the instability of simple heuristics in high-stakes, shifted environments.

The authors note that while their approach does not currently provide consistency guarantees for surrogate losses (a theoretical limitation compared to some end-to-end methods), it offers a practical, robust solution for post-hoc deferral that effectively leverages expert information without requiring full model retraining.