Explainable Visual Anomaly Detection via Concept Bottleneck Models

1. Problem Statement

Visual Anomaly Detection (VAD) aims to identify defects in images using only normal samples during training, a common requirement in industrial settings where defective data is scarce. While state-of-the-art VAD models can generate pixel-level anomaly heatmaps, they suffer from a critical limitation: lack of semantic interpretability.

• The Gap: Current models highlight where a defect is but cannot explain what the defect is in human-understandable terms (e.g., "scratch," "discoloration," "hole").
• The Challenge: Existing Concept Bottleneck Models (CBMs), which provide human-interpretable explanations via intermediate concepts, are typically designed for multiclass classification and require large, manually annotated concept datasets. No such datasets exist for VAD, and adapting CBMs to the binary (normal/anomaly) and fine-grained defect detection setting is non-trivial.

2. Methodology: CONVAD

The authors propose CONVAD (Concept-Aware Visual Anomaly Detection), a dual-branch architecture that bridges semantic interpretability with spatial localization.

A. Concept Dataset Pipeline (Automated Annotation)

Since no concept-annotated VAD datasets exist, the authors developed an automated pipeline using Vision-Language Models (VLMs) to create a "Concept Dataset":

1. Context-Aware List Creation: A VLM (Gemma3) analyzes a small subset of images (5%) using Chain-of-Thought prompting to generate a list of domain-specific concepts (e.g., "uneven tone," "visible ink").
2. Grouping and Filtering: Concepts are merged semantically and filtered using CLIP embeddings to remove duplicates, resulting in a concise vocabulary (<50 concepts per category).
3. Automatic Annotation: The VLM is queried to assign binary labels (presence/absence) for each concept to every image in the dataset.

B. Dual-Branch Architecture

CONVAD combines two distinct modules:

1. CBM Branch (Semantic):
   • Uses a Concept Bottleneck Model structure: $Input \xrightarrow{g} Concepts \xrightarrow{f} Prediction$.
   • The concept extractor ($g$) predicts the presence of specific defects.
   • The label predictor ($f$) uses these concepts to determine if the image is anomalous.
   • Key Feature: Supports Test-Time Intervention. If a human expert corrects a mispredicted concept (e.g., changing "scratch" from false to true), the model instantly updates its final prediction without retraining.
2. Visual Branch (Spatial):
   • Addresses the lack of pixel-level localization in standard CBMs.
   • Uses a Student-Teacher paradigm (similar to PatchCore/STFPM).
   • A teacher network (fine-tuned on the CBM task) and a student network (trained only on normal data) process the image.
   • An anomaly heatmap is generated based on the feature discrepancy between the two networks.

C. Training Regimes & Synthetic Anomaly Generation (SAG)

The paper evaluates various supervision levels to balance performance and labeling effort:

• Fully Supervised: Training on all real normal and anomalous images.
• Weakly Supervised: Training on only 1 or 3 real anomalous images per defect type.
• Synthetic (SAG): Generating anomalies from normal images using text-to-image editing (Nano Banana) with specific prompts for defect type, object, and pose.
• Hybrid: Combining few real anomalies with synthetic ones.

3. Key Contributions

1. First Adaptation of CBMs to VAD: Successfully extends CBMs to the anomaly detection domain, providing the first framework that offers both pixel-level localization and concept-level explanations.
2. Automated Concept Dataset Creation: Developed and released a pipeline to generate concept-annotated datasets for three major industrial benchmarks (MVTec AD, VisA, Real-IAD) using VLMs, overcoming the data scarcity bottleneck.
3. Dual-Branch Interpretability: Introduced an architecture that unifies semantic reasoning (CBM) with spatial precision (Visual Branch), offering a comprehensive view of anomalies.
4. Comprehensive Supervision Analysis: Systematically benchmarked the trade-off between annotation effort and performance, demonstrating that Weakly+SAG (few real anomalies + synthetic data) offers a practical, cost-effective alternative to full supervision.

4. Experimental Results

The method was evaluated on MVTec AD, VisA, and Real-IAD.

• Performance:
  • In the Fully Supervised setting, CONVAD achieves Image-level AUROC (I-AUC) comparable to state-of-the-art unsupervised methods (e.g., PatchCore, STFPM) despite using a much smaller backbone (MobileNet-v2, ~2.25M params vs. ResNet/WideResNet).
  • Weakly+SAG scenarios show significant gains over purely unsupervised baselines, particularly for categories with small, hard-to-detect defects (e.g., "screw," "grid").
• Concept Accuracy: The model achieves high Concept-level AUROC (C-AUC), proving that the extracted concepts are predictive and meaningful.
• Intervention Efficacy: Correcting just a few predicted concepts during inference leads to substantial improvements in detection accuracy, validating the utility of human-machine collaboration.
• Efficiency: CONVAD is highly efficient (15ms inference on CPU), making it suitable for edge deployment, unlike VLM-based approaches that require massive computational resources.

5. Significance and Future Work

• Trust and Explainability: CONVAD moves VAD beyond "black box" heatmaps, allowing operators to understand why a defect was flagged (e.g., "detected due to 'surface discontinuity'"), which is crucial for high-stakes industrial and medical applications.
• Human-in-the-Loop: The ability to intervene on concept activations allows domain experts to guide the model in real-time, correcting errors without retraining.
• Cost Reduction: The findings suggest that collecting massive amounts of real defective data is unnecessary; a small set of real anomalies combined with high-quality synthetic data can yield robust performance.
• Future Directions: The authors plan to improve the quality of synthetic anomaly generation to fully match real distributions and leverage the visual module for detecting novel defect types that the concept vocabulary might miss.

In summary, CONVAD represents a paradigm shift in VAD, successfully integrating the interpretability of Concept Bottleneck Models with the spatial precision required for industrial defect detection, all while reducing the dependency on expensive, large-scale annotated anomaly datasets.