Beyond Augmented-Action Surrogates for Multi-Expert Learning-to-Defer

This paper introduces "Learning-to-Defer with advice," a framework that extends traditional learning-to-defer by dynamically selecting both an expert and additional information for them, proving that standard separated surrogates are inconsistent while proposing a novel augmented surrogate with theoretical guarantees that outperforms existing methods across various tasks.

The Gist

Imagine you are a General Manager running a busy office. You have a team of Specialist Experts (like a tax guru, a coding wizard, and a legal eagle) and you also have your own General Knowledge to handle everyday tasks.

Your goal is simple: For every problem that comes in, you must decide:

1. Solve it yourself (using your general knowledge).
2. Pass it to a specific expert who is best at that type of problem.

This is called "Learning to Defer." The tricky part is teaching the computer (the General Manager) how to make this decision perfectly.

The Old Way: The "One Big Scoreboard"

For a long time, researchers tried to teach the computer using a method called the "Augmented-Action Surrogate."

Think of this like a giant scoreboard with $K$ slots for your own answers and $J$ slots for the experts. The computer tries to predict which single slot on this giant board is the "winner."

The Problem with the Old Way:
The paper argues that this scoreboard has three major flaws, like a broken game show:

1. The "Crowd Effect" (Amplification): If you have 10 experts and 9 of them happen to be right about a problem, the scoreboard gets super excited. It thinks, "Wow, 9 people agree! This must be the most important problem ever!" It over-weights these easy problems and ignores the hard, tricky ones where the decision actually matters.
2. The "Winner Takes All" (Starvation): If two experts are both right, the old method forces them to fight. It picks the one with the slightly higher score and tells the other one, "You're wrong, go sit down." This is bad because it suppresses rare specialists who are right but just happened to have a slightly lower score than the popular generalist.
3. The "Leaky Bucket" (Coupling): The scoreboard mixes your general knowledge and the experts' skills into one big soup. If the experts are confused, it messes up your own ability to think clearly. It's like trying to listen to a radio station while someone is shouting in your ear; the noise from the experts ruins your own signal.

The New Solution: The "Decoupled Surrogate"

The authors propose a new way called the Decoupled Surrogate. Instead of one giant scoreboard, they build two separate, independent systems that talk to each other only at the very end.

1. System A (The General Manager): Uses a Softmax (a standard probability calculator) to figure out: "How confident am I in my own answer?"
2. System B (The Expert Team): Uses Independent Sigmoids (separate confidence meters) for each expert. Expert 1 has their own meter. Expert 2 has their own meter. They don't fight each other; they just report their own confidence.

How it works in practice:
At the end, the computer simply compares the two numbers:

• "Is my confidence (System A) higher than the best expert's confidence (System B)?"
• Yes? I'll do it myself.
• No? I'll pass it to that specific expert.

Why This is a Game Changer (The Metaphors)

• No More Crowd Hype: If 20 experts are right, the new system doesn't get hyped up. It just sees, "Okay, Expert 1 is 90% sure, Expert 2 is 90% sure." It treats them fairly, regardless of how many are right.
• No More Bullying: If a rare specialist is right, the system doesn't punish them just because a popular generalist is also right. Both get credit for being correct.
• Clean Signals: Because the systems are separate, the experts' confusion doesn't mess up the General Manager's brain. The General Manager stays sharp and accurate, even when the experts are struggling.

The Results

The paper tested this new method on:

• Synthetic games (where they could rig the rules to break the old methods).
• Real photos (CIFAR-10).
• Real human annotators (people labeling images).
• Real data models (different AI models acting as experts).

The verdict?
The old methods (the "One Big Scoreboard") started to fail as they added more experts. They got confused, started deferring too much, or stopped learning how to classify things themselves.

The Decoupled Surrogate was the only method that:

1. Got better as they added more experts.
2. Never forgot how to do the job itself.
3. Always found the right expert, even the rare ones.

In a Nutshell

The old way tried to jam everything into one messy bucket, causing chaos when the team got big. The new way gives everyone their own clear, independent voice and lets a simple rule decide who speaks. It's a smarter, fairer, and more scalable way to build AI that knows when to ask for help.

Technical Summary

1. Problem Statement

Learning-to-Defer (L2D) is a framework where a classifier can choose to either predict an output directly or defer the decision to an external expert. In the Multi-Expert setting, the system must decide not only whether to defer but which of $J$ available experts to select.

The optimal decision rule (Bayes rule) is simple: compare the maximum class posterior probability $\max_k \eta_k(x)$ against the maximum expert utility $\max_j \alpha_j(x)$. If the best class probability exceeds the best expert utility, the model predicts; otherwise, it defers to the best expert.

The Challenge: Existing methods rely on Augmented-Action Surrogates, which cast the problem as a single classification task over an enlarged action space of size $K+J$ (where $K$ is the number of classes and $J$ is the number of experts). The paper argues that this shared geometric approach leads to fundamental failures as the number of experts grows:

1. Statistical Target Distortion: The surrogate learns a normalized version of the true probabilities rather than the probabilities themselves.
2. Optimization Pathologies:
   • Amplification: Samples with many correct experts receive disproportionately large gradients, biasing the optimizer toward "easy" high-agreement regions.
   • Starvation: In "winner-take-all" variants, correct experts that do not win the internal competition are actively suppressed (pushed down by gradients), preventing the learning of rare specialists.
   • Coupling: Errors in expert estimation leak into the class estimation (and vice versa), degrading the classifier's performance as the expert pool grows.

2. Methodology: The Decoupled Surrogate

The authors propose a Decoupled Surrogate that abandons the augmented-action geometry entirely. Instead of a single score vector over $K+J$ actions, the model uses two separate heads:

• Class Head: Uses a Softmax activation to estimate the class posterior $p(x) \in \Delta^K$.
• Expert Heads: Uses $J$ independent Sigmoids to estimate each expert's utility $u_j(x) \in (0, 1)$.

The Loss Function:
The total loss $\Phi_{dec}$ is the sum of the standard multiclass cross-entropy for the classifier and the average of $J$ independent Bernoulli cross-entropies for the experts:
$$\Phi_{dec} = -\log p_y(x) - \frac{\lambda}{J} \sum_{j=1}^J \left[ t_j \log u_j(x) + (1-t_j) \log(1-u_j(x)) \right]$$
Where $t_j = 1$ if expert $j$ is correct, and $\lambda$ is a weighting hyperparameter.

Prediction Rule:
At inference, the system compares the maximum class probability directly with the maximum expert probability:
$$\text{Defer to expert } j^* \text{ if } \max_j u_j(x) > \max_k p_k(x); \text{ otherwise predict } \arg\max_k p_k(x).$$

3. Key Contributions

A. Theoretical Analysis of Existing Surrogates

The paper provides a rigorous two-axis analysis of five existing augmented-action surrogates (Additive CE, PiCCE, Mao25, A-SM, OvA):

1. Axis (I) Statistical Target: Does the surrogate recover the true Bayes quantities $(\eta, \alpha)$?
2. Axis (II) Optimization Geometry: How does the surrogate distribute gradient mass during training?

• Finding: Every existing surrogate trades a fix on one axis for a failure on the other. For example, PiCCE fixes gradient amplification but introduces starvation; A-SM fixes the statistical target but retains gradient coupling.

B. Theoretical Guarantees for the Decoupled Surrogate

• Correct Target: The conditional minimizer of the decoupled surrogate is exactly $(\eta, \alpha)$.
• Gradient Stability: The gradients are fully decoupled. The gradient for an expert depends only on that expert's prediction and target, eliminating amplification and starvation.
• H-Consistency Bound: The authors derive an $H$-consistency bound where the calibration constant is independent of the number of experts $J$ (for a fixed per-expert weight $\beta = \lambda/J$). In contrast, existing bounds scale with $O(\sqrt{J})$ or $O(J)$.

C. Empirical Validation

The decoupled surrogate is validated on:

• Synthetic Benchmarks: Specifically designed to isolate pathologies (redundant experts, rare specialists, shared acceptability).
• CIFAR-10 (Synthetic Experts): Shows that as $J$ increases, baselines degrade significantly, while the decoupled method maintains performance.
• CIFAR-10H (Real Human Annotators): Demonstrates robustness with noisy, real-world human labels.
• Covertype (Model Experts): Uses a pool of diverse pre-trained ML models as experts.

4. Results

• Avoidance of Amplification: In synthetic tests with redundant experts ($J=24$), the decoupled surrogate achieved near-Bayes-optimal regret (0.0002), while the best baseline suffered high regret (0.2383) due to gradient amplification.
• Preservation of Rare Specialists: In tasks requiring the selection of a rare specialist expert, PiCCE failed completely (0% selection rate) due to starvation, whereas the decoupled surrogate achieved 99.4% selection.
• Classifier Integrity: On CIFAR-10 and CIFAR-10H, existing methods (like A-SM) saw their standalone classifier accuracy collapse (e.g., dropping from 83% to 68% as $J$ grew) due to gradient coupling. The decoupled surrogate maintained classifier accuracy comparable to a standalone classifier.
• System Accuracy: The decoupled surrogate was the only method that consistently improved system accuracy over a standalone classifier across all datasets and expert pool sizes. All other methods degraded below the baseline as the expert pool grew.

5. Significance

This paper fundamentally challenges the prevailing "augmented-action" paradigm in Learning-to-Defer. It demonstrates that treating classes and experts as a single unified action space introduces unavoidable optimization pathologies that scale poorly with the number of experts.

The Decoupled Surrogate offers a structurally superior alternative by:

1. Respecting Statistical Types: Using Softmax for categorical distributions and Sigmoids for independent probabilities.
2. Ensuring Optimization Stability: Eliminating gradient coupling and amplification, making the method scalable to large expert pools.
3. Providing Stronger Theoretical Bounds: Offering $H$-consistency guarantees that do not degrade with the number of experts.

The work suggests that for multi-expert systems, separation of concerns (decoupling the estimation of class posteriors and expert utilities) is not just a heuristic but a theoretical necessity for robust, scalable learning.