Technical Summary: Avoiding Structural Failure Modes in Tabular Fair SSL

1. Problem Statement

The paper addresses a critical structural conflict in Fair Semi-Supervised Learning (Fair SSL) applied to high-stakes tabular domains (e.g., medical decision support, credit scoring, recidivism prediction). In these settings, labeled data is scarce, necessitating SSL, while regulatory compliance demands statistical fairness guarantees across demographic groups.

The core issue arises from the interaction between confidence-gated pseudo-labeling (a standard SSL mechanism where pseudo-labels are retained only if prediction confidence exceeds a threshold $\tau$) and moment-matching fairness regularizers (which enforce group parity). The authors identify that strong fairness pressure can systematically suppress prediction confidence, causing two distinct structural failure modes:

1. Masking Collapse (Type I): Fairness constraints drive predictions toward the decision boundary ($\hat{y} \approx 0.5$) to minimize group disparity. This erodes confidence, causing the confidence gate to reject most pseudo-labels ($q_t \to 0$). The result is gradient starvation, where the model cannot learn from unlabeled data.
2. Trivial Saturation (Type II): The optimization drifts toward constant predictors (e.g., predicting the mean) to satisfy fairness constraints with zero variance. This yields near-zero fairness violation but results in utility collapse (degraded predictive performance).

The authors argue that static weighting of fairness penalties is brittle because the coupling between pseudo-label selection and constraint evaluation is endogenously non-stationary.

2. Methodology: Online Primal-Dual Allocation (OPDA)

To resolve this structural deadlock, the authors propose OPDA, an online controller that adaptively schedules fairness and stability penalties without per-dataset tuning.

Core Architecture

OPDA employs a bilevel budget-allocation parameterization:

• Total Budget ($B_t$): Controls the overall enforcement intensity.
• Allocation Ratio ($\pi_t$): Determines the split between the fairness penalty ($\lambda_v$) and the entropy-based stability penalty ($\lambda_h$).
  • $\lambda_v^{(t)} = B_t \cdot \pi_t$
  • $\lambda_h^{(t)} = B_t \cdot (1 - \pi_t)$

Signal-Driven Dynamics

OPDA updates $B_t$ and $\pi_t$ epoch-by-epoch using a suite of online observables:

1. Fairness Violation ($v_t$): The current DP-style moment matching penalty.
2. Risk Proxy ($r_t$): A bounded scalar ($1 - \text{MacroF1}_{val}$) tracking utility degradation.
3. Pseudo-Label Health Signals:
   • $q_t$: Pseudo-label pass ratio (coverage).
   • $p_t$: Proxy accuracy.
   • $ESSt$: Effective sample size.
4. Gradient Alignment: Cosine similarity between fairness gradients and base SSL gradients to detect conflict.

Control Mechanisms

• Knee-Seeking Budget Dynamics: The total budget $B_t$ is updated via a projected online gradient descent in the log-domain. It increases only when fairness improves beyond a noise margin without causing risk deterioration beyond a noise margin ($G_t - C_t > 0$). This prevents aggressive over-enforcement that leads to collapse.
• Conflict-Aware Allocation: The ratio $\pi_t$ is determined by minimizing a convex surrogate of urgency. It balances the need to reduce fairness violation ($d_{fair}$) against the need to maintain pseudo-label health ($d_{stab}$). Crucially, it uses a sigmoid gate based on gradient alignment to suppress urgency if constraints are anti-aligned with the base objective.
• Anti-Starvation Guarantee: A dynamic floor $\pi_{min}(t)$ ensures that neither the fairness nor the stability penalty is ever fully turned off, preventing the system from entering a degenerate regime where learning halts.

3. Key Contributions

1. Mechanism Diagnosis: The paper formally identifies and proves the existence of two failure modes in confidence-gated Fair SSL:
   • Masking Collapse: Proven to create vanishing-gradient neighborhoods where $L_{unsup} = 0$ (Proposition 1).
   • Trivial Saturation: Proven to arise from a sign conflict in logit space between moment-matching (pushing toward the mean) and entropy minimization (pushing toward boundaries) under imbalanced groups (Proposition 2).
2. Pseudo-Label-Health-Aware Controller: OPDA is the first controller to explicitly monitor pseudo-label health signals ($q_t, p_t, ESSt$) alongside fairness violations. It schedules dual weights to avoid the "starvation" of the SSL mechanism itself.
3. Calibration-Free Operation: OPDA operates with a single default configuration across diverse datasets (Adult, ACSIncome, COMPAS) and backbone architectures, avoiding the need for manual grid search of fairness weights ($\lambda$).

4. Experimental Results

The authors evaluated OPDA on three tabular benchmarks using a diagnostic stress-test setting (multi-label tasks to isolate failure mechanisms).

• Failure Mode Mitigation: Under static high fairness weights ($\lambda=100$), standard SSL exhibited Masking Collapse (coverage $q_t \to 0.02$) or Trivial Saturation. OPDA successfully avoided these degenerate regimes.
• Performance Trade-offs:
  • Adult & COMPAS: OPDA achieved non-degenerate operating points competitive with the empirical Pareto frontier of static $\lambda$ tuning, balancing utility and fairness effectively.
  • ACSIncome: OPDA preserved utility better than static baselines while exhibiting a wider fairness-utility spread, demonstrating robustness in complex, multi-group settings ($K=9$).
• Comparison to Baselines:
  • Compared to single-signal adaptive baselines (e.g., PI, EMA-P), OPDA showed superior robustness. Single-signal methods often failed to distinguish between capacity issues and pseudo-label starvation, leading to oscillation or saturation.
  • Compared to gradient-based Multi-Objective Optimization (MOO) methods (PCGrad, CAGrad), OPDA achieved lower fairness violations on Adult and COMPAS while maintaining comparable utility. On COMPAS, gradient-only methods exhibited non-zero saturation rates, whereas OPDA maintained 0% saturation.
• Ablation: Removing the full multi-signal logic (OPDA-lite) resulted in lower utility preservation on ACSIncome, confirming the value of the health-aware budget dynamics.

5. Significance and Claims

The paper positions OPDA not as a universal guarantee of uniform dominance across all metrics, but as a calibration-free controller for managing the specific failure regimes inherent to confidence-gated Fair SSL.

• Theoretical Insight: The work highlights that fairness constraints in SSL are not static; they interact dynamically with the data selection mechanism. Static weighting fails because it cannot react to the endogenous non-stationarity of the pseudo-label mask.
• Practical Utility: OPDA provides a pathway to obtain controlled operating points in label-scarce, high-stakes tabular environments without the computational cost of per-dataset hyperparameter tuning.
• Scope Limitations: The authors modestly note that while the identified pathologies are fundamental to the SSL mechanism, the empirical validation is currently limited to tabular data. The extension to high-dimensional image or text data remains an open question. Furthermore, the controller requires pre-specifying internal constants (e.g., warmup length), though it demonstrates robustness to moderate perturbations.

In summary, the paper argues that avoiding structural failure in Fair SSL requires moving beyond static weighting or simple gradient balancing, instead adopting an online control strategy that actively monitors the "health" of the pseudo-labeling process to prevent the system from collapsing into trivial solutions.