Multi-view Patch Inference with Frozen DINOv3 for Industrial Anomaly Detection

Technical Summary: Multi-view Patch Inference with Frozen DINOv3 for Industrial Anomaly Detection

Problem Statement
Industrial Anomaly Detection (IAD) aims to identify subtle defects in automated visual inspection systems without relying on large-scale anomaly annotations. While pretrained Vision Transformers (ViTs) like DINO have shown strong feature representation capabilities, existing methods typically rely on single-grid patch partitioning during inference. This approach suffers from two primary limitations:

1. Boundary Information Loss: Defects located at patch boundaries are often missed or weakened because the single-grid partitioning fails to capture the full context of the anomaly.
2. Limited Local Context Modeling: Single-view inference struggles with small-scale, irregular defects embedded in noisy backgrounds, leading to degraded performance in robust defect localization.
   Furthermore, many existing reconstruction-based frameworks require complex architectures (e.g., heavy encoder-decoders or masked image modeling) or memory banks that incur high storage costs and computational overhead.

Methodology
The authors propose a lightweight framework built upon a frozen DINOv3 backbone that integrates two core innovations: Multi-view Patch Inference (MVPI) and a Multi-scale Reconstruction Fusion Transformer (MRFT).

1. Frozen DINOv3 Backbone:

   • The model utilizes a pretrained DINOv3 (7B-scale) as a frozen feature extractor, leveraging its "Register Tokens" for global semantic anchoring and its high-capacity representation for fine-grained textures.
   • Features are extracted from three specific layers to capture hierarchical information:
     • Layer 2: Low-level features (edge structures, fine texture gradients).
     • Layer 5: Mid-level features (part-level structure, local geometric consistency).
     • Layer 8: High-level features (semantic consistency, object-level structure).

2. Multi-view Patch Inference (MVPI):

   • To mitigate boundary information loss, the input image is processed through multiple overlapping spatial views generated via a sliding-window mechanism.
   • Each view is independently encoded by the frozen DINOv3 backbone.
   • Features from overlapping regions are aggregated using weighted averaging, where weights are implicitly determined by spatial coverage frequency. This creates an implicit ensemble that enhances boundary consistency and reduces spatial aliasing.

3. Multi-scale Reconstruction Fusion Transformer (MRFT):

   • Instead of complex decoders, a lightweight transformer module performs hierarchical feature reconstruction.
   • It processes features through three stages (shallow, intermediate, deep) to refine texture, structure, and semantic information, respectively.
   • The outputs of these stages are fused to produce a stable reconstruction of normal feature distributions.
   • Jitter Regularization: Gaussian noise is added to input features during training to prevent overfitting to exact normal patterns.

4. Anomaly Scoring:

   • The anomaly score is computed based on the cosine distance between the input features and the reconstructed features ($S = 1 - \cos(z, \hat{z})$).
   • Image-level Aggregation: To handle noise and outliers, the method applies Gaussian smoothing followed by Top-K pooling (selecting the top 100 extreme values) rather than global max or average pooling. This balances noise suppression with the retention of subtle defect signals.

Key Contributions

• Multi-view Patch Inference (MVPI): A strategy that constructs overlapping spatial views to eliminate boundary information loss, significantly improving the detection of small and boundary-located defects.
• Multi-scale Reconstruction Fusion Transformer (MRFT): A lightweight module that fuses hierarchical features from multiple transformer layers to model normal distributions more stably than single-scale approaches, without requiring heavy decoder structures.
• Efficient Frozen Backbone Framework: A system that achieves state-of-the-art performance using a frozen DINOv3 backbone, avoiding the computational overhead of memory banks or complex distillation architectures while maintaining high deployment efficiency.

Experimental Results
The method was evaluated on three benchmark datasets: MVTec AD, BTAD, and the challenging UltraChip dataset (focusing on micro-defects in semiconductors).

• UltraChip Dataset: The proposed method achieved a mean Image-AUC of 81.9%, outperforming recent models like URD and SuperSimpleNet. Qualitative results showed precise defect highlighting without the widespread false-positive background noise common in other methods.
• MVTec AD: The method secured the highest scores across all metrics among compared baselines, achieving 99.5% I-AUC, 99.2% P-AUC, and 96.0% PRO. It demonstrated superior performance over SOTA methods like Dinomaly and Subspace.
• BTAD: The framework maintained a consistent advantage with an average I-AUC of 97.3%, demonstrating robustness against varying lighting and complex topologies.
• Ablation Studies:
  • The combination of layers {2, 5, 8} was identified as optimal, balancing fine-grained texture capture with semantic stability.
  • Top-K Pooling (K=100) was shown to be superior to Global Max-pooling (vulnerable to noise) and Global Average-pooling (over-smoothing), providing the best trade-off for image-level scoring.

Significance and Claims
The paper claims that combining multi-view inference with frozen vision transformer representations offers a robust solution for industrial anomaly detection. The significance lies in:

1. Accuracy vs. Efficiency: The framework achieves state-of-the-art detection accuracy while drastically reducing GPU memory usage and computational overhead compared to heavy reconstruction or memory-bank-based methods.
2. Robustness to Small Defects: The MVPI strategy specifically addresses the failure modes of single-grid methods, enabling the detection of subtle defects (e.g., tiny scratches on screw tips) that are often missed by conventional approaches.
3. Minimalist Design: The approach adheres to a "minimalist pure reconstruction philosophy," operating entirely in semantic feature space and bypassing the need for complex pseudo-anomaly generation or identity mapping issues.

The authors note that while the frozen backbone introduces some inference latency, the method demonstrates promising zero-shot generalization capabilities, particularly on the UltraChip dataset, suggesting potential for future expansion into zero-shot or few-shot scenarios. The current framework operates under a single-class training paradigm, which the authors identify as a limitation for future unified multi-class development.