From Local Geometry to Global Pseudo Labeling for Robust Positive Unlabeled Learning under Covariate Shift

Technical Summary: From Local Geometry to Global Pseudo-Labeling for Robust Positive–Unlabeled Learning under Covariate Shift

Problem Statement
Detecting covariate shift—where the input distribution $P(X)$ changes while the conditional labeling function $P(Y|X)$ remains constant—is critical for deploying reliable vision systems. While existing research focuses heavily on domain adaptation or generalization, the explicit detection of such shifts remains underexplored. Current detection methods are predominantly fully supervised, requiring labeled examples from both the original (in-distribution) and shifted distributions. This requirement is often impractical, particularly when negative (shifted) samples are unavailable or expensive to curate.

Furthermore, applying standard Positive–Unlabeled (PU) learning directly to covariate shift detection is problematic. Classical PU methods rely on unbiased risk estimators that assume sufficient separability between positive and negative distributions. However, under covariate shift, the in-distribution and shifted data often overlap significantly in the feature space. This overlap results in a small total variation distance and high intrinsic Bayes risk, causing global risk estimation to become unstable and sensitive to noise.

Methodology: S-PUNA
To address these challenges, the authors propose Spectral PU Neighborhood Annotation (S-PUNA), a geometry-aware framework that leverages the local manifold structure of visual features to progressively discover shifted data without requiring global supervision.

1. Local Manifold Expansion:
   Unlike classical k-NN-PU approaches that only propagate positive labels, S-PUNA symmetrically constructs both positive and negative anchor sets. Starting from a pre-trained Vision Transformer (ViT) feature space (specifically the 6th block, chosen for its sensitivity to geometric nuances), the method initializes pseudo-labeled sets based on distance scores from the positive domain. It then iteratively expands these sets by identifying samples that are statistically inconsistent with the current positive support but consistent with the emerging negative manifold.

2. Symmetric Propagation:
   In each iteration, the algorithm computes mean distances for candidate samples to their $k$-nearest neighbors in both the current positive and negative banks. A scoring mechanism ranks candidates to expand the positive set (samples close to positives) and the negative set (samples far from positives but forming a coherent cluster). This iterative process refines the decision boundary by progressively uncovering the negative manifold.

3. Spectral Entropy Stopping Criterion:
   A critical challenge in iterative pseudo-labeling is "semantic drift," where the algorithm mistakenly incorporates in-distribution samples into the negative set. S-PUNA introduces a novel stopping criterion based on spectral entropy. By monitoring the eigenvalue distribution of the covariance matrix of the discovered negative set, the method measures its intrinsic dimensionality and structural complexity.

   • Theoretical Basis: The authors prove (Lemma 1) that as the negative set becomes contaminated with positive samples (semantic drift), the spectral entropy strictly decreases.
   • Mechanism: The expansion process halts when the spectral entropy stabilizes or decreases, indicating that the negative manifold has been sufficiently captured without invading overlapping regions.

4. Knowledge Distillation:
   Once the iterative expansion converges, the non-parametric neighborhood structure is distilled into a parametric Deep Neural Network (DNN) classifier. This step smooths the decision boundary, reducing variance while preserving the local geometric separation discovered during the PU phase.

Key Contributions

• Novel Problem Formulation: The paper is the first to systematically investigate covariate shift detection through the lens of Positive–Unlabeled (PU) learning, providing theoretical insights into why standard PU risk minimization fails under significant distributional overlap.
• S-PUNA Framework: Introduction of a geometry-aware pseudo-labeling framework that progressively uncovers shifted distributions via local manifold expansion rather than global risk estimation.
• Spectral Entropy Criterion: A principled stopping mechanism that detects early contamination and prevents pseudo-labeling drift, theoretically justified by the relationship between spectral entropy and mixture contamination.
• Benchmarks and Performance: The authors establish new covariate shift benchmarks and demonstrate that S-PUNA achieves state-of-the-art performance in PU settings, remarkably matching the performance of fully supervised methods.

Experimental Results
The authors evaluated S-PUNA across three primary benchmarks: ImageNet-1K, TinyImageNet, and EuroSAT, covering both "near" shifts (subtle distributional changes, e.g., ImageNet-V2, GenImage) and "far" shifts (severe corruptions or stylization, e.g., ImageNet-C, ImageNet-R).

• Performance: S-PUNA consistently outperformed classical PU baselines (e.g., Dist-PU, DC-PU, saPU, LaGAM). On ImageNet-1K near-shift scenarios, S-PUNA achieved 97.89% AUROC, surpassing the strongest baseline (Dist-PU at 84.48%) by over 13% and reducing the False Positive Rate at 95% True Positive Rate (FPR95) from 35.05% to 9.69%.
• Comparison to Supervised Methods: In several settings, S-PUNA approached the performance of fully supervised classifiers, which serve as an upper bound. For instance, on ImageNet-C, S-PUNA achieved 98.51% AUROC, nearly matching the supervised baseline.
• Robustness: The method demonstrated strong generalization across different shift types and dataset sizes (TinyImageNet, EuroSAT), maintaining high stability where other PU methods exhibited significant performance gaps between near and far shifts.
• Unsupervised Baselines: The paper also highlights that purely unsupervised Out-of-Distribution (OOD) and anomaly detection methods (e.g., ReAct, PatchCore) struggle significantly with near covariate shifts, often performing close to random, whereas S-PUNA effectively leverages weak supervision to overcome this.

Significance and Claims
The paper argues that strictly supervised signals are not a prerequisite for reliable covariate shift detection. By effectively leveraging the intrinsic geometric structure of feature representations, weakly supervised PU learning can achieve highly robust, state-of-the-art performance. The work bridges the gap between the theoretical limitations of global PU risk estimation under overlap and practical detection needs, offering a scalable solution for real-world applications where labeled shifted data is unavailable. The authors emphasize that their approach is particularly effective in "near-shift" regimes, which are typically the most challenging for existing unsupervised detectors.