OASIC: Occlusion-Agnostic and Severity-Informed Classification

1. Problem Statement

The paper addresses the challenge of fine-grained image classification under severe occlusion (up to 90% occlusion, leaving only 10% of the object visible). The authors identify two root causes for performance degradation in such scenarios:

1. Loss of Visible Information: Critical details required to distinguish between similar classes are hidden.
2. Distracting Patterns: The occluders themselves (e.g., foliage, smoke, rubble) introduce complex textures that mislead the model's attention away from the object of interest.

Existing solutions often rely on specific occlusion types or fail to generalize to unpredictable, real-world occlusions. Furthermore, standard training on clean images or simple data augmentation does not adequately prepare models for the specific visual noise introduced by textured occluders.

2. Methodology: OASIC

The proposed framework, OASIC (Occlusion-Agnostic Severity-Informed Classification), operates in two main stages: Occlusion Handling (Masking) and Severity-Informed Model Selection.

A. Occlusion-Agnostic Segmentation and Masking

Instead of requiring prior knowledge of the occluder type (e.g., "this is smoke"), OASIC treats occlusion as a visual anomaly relative to the clean object.

• Anomaly Detection: The method utilizes AnomalyDINO to generate pixel-level anomaly likelihood maps. These maps highlight regions that deviate from the expected appearance of the object.
• Adaptive Thresholding: To convert the anomaly map into a binary mask, the authors employ Otsu's method rather than a fixed threshold. This dynamically calculates the optimal threshold ($\tau^*$) for each image based on the histogram of anomaly scores, balancing false positives and negatives.
• Gray Masking: Once the occluded regions are segmented, they are replaced with a uniform mid-level gray tone ($g=127$). This removes the high-frequency, distracting textures of the occluder (like leaves or smoke) while preserving the visible parts of the object.

B. Occlusion Severity Estimation

The system quantifies the severity of the occlusion ($\hat{s}$) by calculating the mean of the pixel-wise anomaly scores across the image. This provides a continuous value representing the fraction of the image affected by occlusion.

C. Severity-Informed Model Selection

The authors discovered a critical insight: models optimized for a specific range of occlusion severities perform best when tested on similar levels of occlusion.

• Model Pool: A pool of classification models ($\mathcal{F}$) is created by fine-tuning a base model (DINOv2 ViT-B/14) on datasets with varying maximum occlusion levels ($p \in \{0, 10, \dots, 100\}$). Each model $f_{[0,p]}$ is trained on synthetic occlusions ranging from 0% to $p$%.
• Dynamic Selection: At inference time, the estimated severity $\hat{s}$ is used to select the most suitable model from the pool. The system selects the model trained on the range $[0, p^*]$ where $p^*$ is the closest upper bound to the estimated severity:
  $$p^* = \arg \min_{p \in P} |\hat{s} - p|$$
  This ensures the model processing the image is specifically tuned for the current level of visibility.

3. Key Contributions

1. Unified Framework (OASIC): A novel approach that simultaneously addresses the loss of information (via severity-informed selection) and visual distraction (via anomaly-based masking).
2. Occlusion-Agnostic Segmentation: The ability to localize and mask arbitrary occluders (vegetation, smoke, rubble) without explicit supervision or prior knowledge of the occluder type, by leveraging visual anomaly detection.
3. Severity-Informed Strategy: The discovery that a single model cannot optimally handle all occlusion levels. The proposed adaptive selection strategy dynamically routes images to the model best suited for their specific occlusion severity.

4. Experimental Results

The method was evaluated on the Stanford Cars dataset (196 categories) using synthetic occlusions (Perlin noise masks filled with real textures like foliage, smoke, and rubble).

• Performance Metrics: The primary metric is $AUC_{occ}$ (Area Under the Curve of accuracy vs. occlusion severity).
• Comparative Gains:
  • OASIC improved $AUC_{occ}$ by +18.5 compared to models trained directly on occluded images.
  • It improved $AUC_{occ}$ by +23.7 compared to models fine-tuned on unoccluded (clean) images.
• Component Analysis:
  • Gray Masking: Removing distracting textures significantly boosts performance, especially against complex textures like vegetation and rubble, which degrade accuracy more than uniform gray occlusions.
  • Model Selection: Using a fixed model trained on a broad range (0–90%) is inferior to dynamically selecting a model trained on a specific range matching the test image's severity.
• Segmentation Accuracy: The occlusion-agnostic AnomalyDINO approach significantly outperformed prompt-based segmentation baselines (like OVSeg) across all occlusion types, achieving high AUROC and Average Precision without needing text prompts.

5. Significance and Conclusion

OASIC represents a significant step forward in robust computer vision for real-world deployment, where objects are rarely fully visible.

• Robustness: It demonstrates that treating occlusion as an anomaly allows for generalization across diverse, unpredictable occluders.
• Adaptability: The severity-informed selection mechanism acknowledges that different levels of visibility require different feature representations, moving beyond the "one-size-fits-all" model paradigm.
• Future Impact: The framework is applicable beyond fine-grained classification to tasks like autonomous navigation, surveillance, and medical imaging, where partial visibility is the norm. The authors suggest future work could integrate occlusion estimation and classification into a single end-to-end architecture.