OOD-SEG: Exploiting out-of-distribution detection techniques for learning image segmentation from sparse multi-class positive-only annotations

1. Problem Statement

The paper addresses two critical challenges in medical and surgical image segmentation using deep neural networks:

1. Scarcity of Dense Annotations: Acquiring pixel-level ground truth labels for medical images is time-consuming, expensive, and requires high-level domain expertise. Consequently, many datasets contain only sparse, positive-only annotations (annotating specific regions of interest while leaving the rest unlabelled).
2. Lack of Out-of-Distribution (OOD) Detection: Standard segmentation models assume all training and testing data come from the same distribution (In-Distribution or ID). In safety-critical clinical environments, models often fail to detect pixels belonging to unseen classes (e.g., new surgical instruments, unexpected tissue types) or background, leading to spurious, overconfident predictions.

The Core Challenge: Existing Weakly Supervised Learning (WSL) methods often treat unlabelled pixels as background (negative class), which introduces label noise if those pixels actually belong to unannotated positive classes. Furthermore, there is a lack of established protocols and benchmarks for evaluating OOD detection specifically in multi-class medical segmentation tasks.

2. Methodology: OOD-SEG Framework

The authors propose OOD-SEG, a framework that reframes sparse multi-class positive-only segmentation as a Positive-Unlabelled (PU) learning problem, leveraging tools from OOD detection.

A. Problem Formulation

• Positive-Only Learning: The model is trained only on annotated pixels belonging to specific foreground classes (ID classes). No assumptions are made about unannotated pixels; they are not treated as background.
• PU Learning Perspective: Unlabelled pixels are treated as a mixture of positive (unseen foreground) and negative (background/OOD) samples.
• OOD Detection Mechanism: The framework establishes distinct decision boundaries for each positive class. Pixels falling outside these boundaries are flagged as OOD (background or unseen classes) rather than being forced into a known class.

B. Architecture and Training

• Backbone: A standard U-Net architecture with an EfficientNet-B4 encoder.
• Training Strategy: The network is trained using Cross-Entropy Loss computed only on the sparse, positive-only annotated pixels. The output layer has $C$ channels (where $C$ is the number of annotated classes), meaning the model does not explicitly predict a "background" class during training.
• Inference & Scoring:
  • During inference, a pixel-wise scoring function $S_c$ is computed for each class.
  • A confidence threshold $\tau$ is applied. If $\max(S_c) > \tau$, the pixel is assigned to the class with the highest score. If $\max(S_c) \leq \tau$, the pixel is classified as OOD (0).
  • Scoring Methods Evaluated:
    1. Baseline: Standard Softmax output.
    2. ODIN: Uses temperature scaling ($T=10$) on logits to improve calibration.
    3. Mahalanobis: Computes the negative Mahalanobis distance between pixel features and class-conditional Gaussian distributions.
    4. GODIN (Generalized ODIN): A novel adaptation for segmentation where the temperature scaling and similarity functions are implemented via convolutional layers (3x3 kernels) to capture spatial context, rather than just pixel-wise linear layers.

C. Evaluation Strategy: Two-Level Cross-Validation

To address the lack of external OOD datasets, the authors propose a novel Two-Level Cross-Validation strategy within a single dataset:

• Subject Partitions (SP): Ensures no patient data overlap between folds.
• Class Partitions (CP): In each fold, a subset of annotated classes is held out from training.
• Mechanism: The held-out classes serve as the ground truth for OOD during testing. This simulates a scenario where the model encounters classes it has never seen, allowing for rigorous evaluation of OOD detection capabilities without needing a separate dataset.
• Metrics:
  • Threshold-Independent: AUROC (Area Under ROC) and AUPR (Area Under Precision-Recall), calculated by treating held-out classes as the negative/OOD class and annotated classes as positive.
  • Threshold-Dependent: TPR (True Positive Rate) for ID classes and FPR (False Positive Rate) for OOD classes.

3. Key Contributions

1. Novel Framework: The first work to formulate multi-class medical image segmentation with sparse positive-only annotations as a pixel-wise PU learning problem solved via OOD detection techniques.
2. Evaluation Protocol: A dedicated two-level cross-validation strategy (Subject + Class partitions) that eliminates the need for external OOD datasets, enabling systematic assessment of background/OOD performance within a single dataset.
3. Methodological Adaptation: The introduction of GODIN adapted for segmentation using convolutional layers to handle spatial context, and the demonstration that standard OOD classification tools (ODIN, Mahalanobis) can be effectively repurposed for segmentation.
4. Performance: The framework achieves state-of-the-art OOD detection performance without compromising the classification accuracy of the known (ID) classes.

4. Experimental Results

The framework was validated on three datasets:

• Heiporspectral: Hyperspectral imaging (HSI) of porcine organs (20 classes).
• ODSI-DB: HSI of human dental tissues (9 classes).
• DSAD: RGB laparoscopic surgical images (11 classes).

Key Findings:

• Best Performer: The ODIN method (using temperature scaling) consistently yielded the best AUROC and AUPR scores across all datasets. This highlights the importance of well-calibrated confidence scores for OOD detection.
• OOD Detection: The framework successfully identified OOD pixels (held-out classes) with high accuracy (e.g., AUROC up to 0.96 on Heiporspectral for ODIN), significantly outperforming the baseline.
• ID Preservation: Applying the OOD threshold ($\tau_m$) did not degrade the classification accuracy of the known ID classes compared to a baseline with no threshold ($\tau=0$).
• Qualitative Analysis: Visualizations confirmed that the model effectively suppresses predictions on unlabelled regions and held-out classes, whereas baseline models often forced these pixels into incorrect known classes.

5. Significance and Impact

• Clinical Safety: By enabling reliable detection of unknown or anomalous tissues without requiring dense background annotations, OOD-SEG enhances the safety and robustness of automated surgical assistance systems.
• Annotation Efficiency: It validates the feasibility of training robust segmentation models using only sparse, positive-only annotations, drastically reducing the burden on clinical experts.
• Benchmarking: The proposed evaluation protocol provides a scalable, reproducible standard for future research in medical image segmentation, particularly for PU learning and OOD detection, removing the dependency on hard-to-acquire external OOD datasets.

Limitations: The current framework does not utilize unlabelled data during training (only positive annotations), and it does not employ adversarial perturbations (due to computational cost and minimal gains observed). Future work aims to address the trade-off between ID accuracy and OOD detection sensitivity.