Technical Summary: Multi-AD for Cross-Domain Unsupervised Anomaly Detection

Problem Statement

Anomaly detection (AD) is critical for early disease diagnosis in medicine and defect detection in industrial manufacturing. However, a primary bottleneck is the scarcity of labeled anomaly data. Anomalous cases are inherently rare, and acquiring labeled data—particularly in medical imaging—is labor-intensive, expensive, and subject to inter-observer variability. While self-supervised learning (SSL) has emerged as a solution by training on unlabeled "normal" data, its performance often degrades in scenarios with limited training data or high intra-class variability (e.g., diverse anatomical structures or complex industrial environments). Furthermore, existing Knowledge Distillation (KD) approaches are frequently limited to single-domain applications and lack robustness in cross-modality or cross-domain transfer.

Methodology: Multi-AD Framework

The authors propose Multi-AD, a convolutional neural network (CNN) framework designed for robust unsupervised anomaly detection across both medical and industrial domains. The method integrates domain-agnostic feature learning with domain-specific adaptation using a Teacher-Student (T-S) architecture enhanced by three core components:

1. Enhanced Backbone with Squeeze-and-Excitation (SE)

The backbone is a modified WideResNet-50 architecture incorporating:

• Dilated Convolutions: Applied across four residual stages with progressively increasing dilation rates ($r=1, 2, 4, 8$) to expand the receptive field while preserving spatial resolution. This allows the model to capture both local details (e.g., microcracks) and global context (e.g., structural distortions).
• Squeeze-and-Excitation (SE) Blocks: Integrated into each residual stage to perform channel-wise feature recalibration. This mechanism computes channel-wise statistics via global average pooling and uses fully connected layers to generate scaling factors, enabling the network to emphasize informative features and suppress irrelevant ones.
• Multi-scale Feature Fusion: A convolutional layer fuses features from lower and upper layers to combine fine-grained details with high-level semantic information.

2. Knowledge Distillation (KD) with Feature Alignment

The framework employs a T-S architecture where a pre-trained Teacher (T) model (frozen) transfers knowledge to a trainable Student (S) model.

• Training Regime: The model is trained exclusively on normal (anomaly-free) data.
• Feature Alignment: The S model is trained to mimic the T model's feature representations. Features from critical layers of both models are L2-normalized.
• Loss Function: A feature generator loss ($\mathcal{L}_G$) minimizes the Euclidean distance (via cosine similarity) between the normalized activation vectors of the T and S models across multiple layers. This ensures the S model learns the intrinsic distribution of normal data.

3. Discriminator (D) Network and Adversarial Training

To further refine the distinction between normal and anomalous data, a Discriminator (D) network is introduced:

• Architecture: A convolutional network that processes feature maps to output a probability score indicating whether the input is "real" (T model features) or "fake" (S model features).
• Adversarial Objective: The D network is optimized to distinguish T features from S features using Binary Cross-Entropy (BCE) loss. Conversely, the S model is trained with an adversarial term ($\mathcal{L}_{adv}$) to fool the D network, forcing the S model to produce feature maps indistinguishable from the T model's normal distribution.
• Total Loss: The S model's total loss combines feature alignment and adversarial terms: $\mathcal{L}_S = \mathcal{L}_G + \lambda \mathcal{L}_{adv}$.

4. Convolution-Enhanced Multi-Scale Feature Fusion (MFF)

During inference, anomaly maps are generated by computing pixel-wise cosine dissimilarity between T and S features across multiple layers ($M_1$ to $M_4$).

• Refinement: Unlike simple summation, the proposed MFF applies a 1×1 convolution, Batch Normalization, and ReLU to each scale's anomaly map before fusion. This step refines feature maps to address scale variations and spatial misalignment.
• Aggregation: The refined maps are combined via bilinear interpolation and smoothed with a Gaussian filter to produce the final anomaly map. The image-level anomaly score is derived from the maximum response of this map.

Key Contributions

The paper outlines four primary contributions:

1. Cross-Domain Framework: A unified AD framework that disentangles domain-agnostic (shared) and domain-specific (adaptive) feature learning, demonstrating generalization across medical (MRI, CT, OCT) and industrial datasets.
2. Convolution-Enhanced MFF: A novel module that improves the localization of both small and large anomalies by refining multi-scale features prior to fusion.
3. Adaptive Discriminator: The integration of D networks with adaptive attention mechanisms to enable precise anomaly localization across diverse imaging modalities.
4. Precise Localization: The provision of accurate anomaly maps for localizing diseases and product defects, validated through extensive experiments.

Experimental Results

Multi-AD was evaluated on three medical datasets (BraTS2021 for brain MRI, RESC for retinal OCT, and BTCV/LiTs for liver CT) and the MVTec AD industrial benchmark.

• Medical Performance: Multi-AD achieved an average Image-level AUROC of 81.4% and Pixel-level AUROC of 97.0%, outperforming state-of-the-art models including PaDiM, PatchCore, SimpleNet, STFPM, RD4AD, and MKD.
• Industrial Performance: On MVTec AD, the model achieved an average Image-level AUROC of 99.6% and Pixel-level AUROC of 98.4%, consistently surpassing clustering-based and reconstruction-based methods.
• Ablation Studies:
  • Removing the SE Block reduced pixel-level AUROC, confirming its role in focusing on informative features.
  • Removing the Discriminator (D) significantly degraded performance (e.g., medical pixel-level AUROC dropped from 97.0% to 93.2%), highlighting the importance of adversarial refinement.
  • Removing MFF led to a notable decrease in localization accuracy, validating the necessity of convolutional refinement for multi-scale fusion.

Significance and Claims

The authors claim that Multi-AD offers a robust solution for real-world applications where labeled anomaly data is scarce. By leveraging KD and adversarial training, the model effectively learns the distribution of normal data and detects subtle deviations without requiring anomaly labels during training.

The paper emphasizes that Multi-AD's ability to generate sharp, focused anomaly heatmaps with minimal false positives makes it superior to clustering-based methods (which often produce broad, unfocused maps) and reconstruction-based methods (which struggle with complex textures). The framework's demonstrated cross-domain generalization suggests it can be deployed across diverse medical and industrial settings without extensive retraining, addressing a critical need for scalable and reliable anomaly detection systems. However, the authors modestly note that the integration of MFF and adversarial networks increases computational complexity, which may pose challenges for real-time implementation in resource-constrained environments.