Technical Summary: Divide and Conquer: Object Co-occurrence Helps Mitigate Simplicity Bias in OOD Detection

Problem Statement

Deep learning models, while achieving remarkable success on in-distribution (ID) data, struggle with reliability when encountering distribution shifts, particularly in Out-of-Distribution (OOD) detection. Existing methods predominantly rely on entangled latent representations, logit outputs, or a combination thereof. While effective for "far-OOD" scenarios with drastic distribution shifts, these approaches often fail in "near-OOD" settings where subtle semantic deviations exist.

The core limitation identified is the simplicity bias inherent in deep networks. When processing entangled representations, models tend to prioritize easily learnable local cues (e.g., specific object textures or shapes) while overlooking complex semantic relationships and contextual incongruities crucial for scene understanding. For instance, a model might confidently classify an image containing a dog in an ocean as "dog" (ID) because it focuses on the dog's features, ignoring the semantic incompatibility of a dog co-occurring with a shark (OOD) in that setting. This leads to overconfident predictions on anomalous samples.

Methodology: OCO Framework

The authors propose OCO (Object-Centric OOD detection), a framework designed to mitigate simplicity bias by leveraging Object Co-occurrence (OCO) patterns. The method operates on the premise that human visual systems utilize object relationships within scenes to understand context, and machines should similarly decompose images into distinct object compositions.

1. Object-Centric Representation via Slot Attention

The framework utilizes Slot Attention (SA) to transform entangled feature maps into a set of $K$ disentangled slots. Each slot is intended to capture a distinct object or object part.

• Training: A classifier is trained on these slots to predict class labels. To ensure the slots capture meaningful object features rather than arbitrary fragments, an auxiliary reconstruction loss ($L_{aux}$) is introduced alongside the standard classification loss ($L_{ce}$). This forces the slots to reconstruct the original entangled features, ensuring robust object localization.
• Pattern Discovery: During training, the system aggregates slot-wise predictions to identify co-occurrence configurations. It distinguishes between single-category configurations (all slots agree on one class) and multi-category configurations (multiple distinct classes detected). A collection of valid multi-category frequency sets ($F_{train}$) is built from the ID training data to represent typical scene contexts.

2. Divide-and-Conquer Strategy

At test time, the OCO framework analyzes the co-occurrence pattern of a test sample ($F_t$) and adaptively categorizes it into one of three scenarios, applying a specific detection score for each:

• Scenario 1 (S1): Single Patterns

  • Condition: All slots predict the same class ($|F_t| = 1$).
  • Challenge: High risk of overconfidence (simplicity bias).
  • Solution: Dual-Confidence Calibration. The score ($S_{single}$) combines scene-level confidence (aggregated prediction) with object-level confidence (maximum agreement among slots) to mitigate overestimation.

• Scenario 2 (S2): Typical Patterns

  • Condition: Multi-object patterns match those observed in training ($|F_t| \ge 2$ and $F_t \in F_{train}$).
  • Challenge: High ambiguity, especially in near-OOD cases where objects co-occur in known ways but the specific combination might be anomalous.
  • Solution: Uncertainty-Aware Scoring (Dempster-Shafer Theory). The method calculates belief contradictions between the dominant category and other predicted categories. It distinguishes between "Pattern Likelihood" (expected co-occurrence) and "Ambiguous Evidence" (uncertainty about which specific ID class applies), effectively modeling conflicting evidence to detect subtle deviations.

• Scenario 3 (S3): Atypical Patterns

  • Condition: Multi-object patterns do not match training observations ($|F_t| \ge 2$ and $F_t \notin F_{train}$).
  • Challenge: Unseen, implausible combinations (e.g., a penguin and a camel).
  • Solution: Object-Level Confidence. Since the pattern itself is anomalous, the method relies on the highest slot prediction confidence ($S_{unrel}$) to quantify the anomaly directly.

Key Contributions

1. Novel Framework: Introduction of OCO, an object-centric framework that leverages object co-occurrence patterns to learn complex scene-object semantic correlations, directly addressing the simplicity bias of traditional models.
2. Adaptive Divide-and-Conquer: Unlike methods using a unified OOD score, OCO adaptively categorizes test samples into three distinct groups based on their co-occurrence patterns and assigns tailored detection scores to each, acknowledging that different types of anomalies require different detection strategies.
3. Robust Performance: Demonstration that OCO achieves competitive performance against state-of-the-art baselines on both challenging near-OOD benchmarks and Full-spectrum OOD (FS-OOD) settings, effectively handling both semantic shifts (near-OOD) and covariate shifts.

Experimental Results

The authors evaluated OCO on the OpenOOD benchmark using ImageNet-1k as the ID dataset.

• Baselines: Compared against 9 baselines including Energy, MaxLogit, SHE, OODD, and others.
• Backbones: Experiments were conducted using both supervised ViT and self-supervised DINOv2 backbones.
• Performance:
  • OOD Detection: OCO consistently outperformed baselines. On the challenging SSB-hard (near-OOD) dataset, it achieved significant improvements in AUROC compared to the runner-up (OODD).
  • FS-OOD Detection: In Full-spectrum evaluations (including covariate shifts like ImageNet-v2/c/r), OCO showed superior generalization, particularly on the SSB-hard dataset where other methods struggled.
  • Ablation Studies: Results confirmed the necessity of the reconstruction loss ($L_{aux}$) for effective slot disentanglement and the superiority of the scenario-specific scoring over a unified score.
  • Slot Count: Performance peaked at $K=6$ slots; fewer slots led to under-segmentation, while excessive slots caused over-segmentation and performance degradation.

Significance and Claims

The paper claims that the gap between human and machine perception in OOD detection stems from the machine's tendency to focus on simple, local features. By mimicking human reliance on object relationships, OCO successfully mitigates this bias.

The authors assert that their approach provides a new paradigm for OOD detection that moves beyond simple feature discrimination to semantic contextual understanding. The "divide-and-conquer" strategy allows the model to handle the spectrum of OOD challenges—from subtle near-OOD semantic shifts to obvious far-OOD anomalies—more effectively than unified approaches. The method is described as simple, extensible, and promising for real-world applications where reliability under distribution shifts is critical. The authors also note that the statistical alignment between predicted co-occurrence patterns and WordNet hierarchies validates the model's ability to learn meaningful semantic relationships.