A Structured Benchmark for Text-Guided Anomaly Detection: When Language Stops Conditioning the Decision

Problem Statement
Industrial anomaly detection (AD) has historically been a unimodal task, relying on visual inputs to learn representations of normality and flag deviations. While recent vision-language models (LVLMs) have introduced text-guided, zero-shot, and few-shot inspection capabilities, their evaluation protocols remain inherited from unimodal benchmarks like MVTec AD. These standard protocols hold textual conditions effectively constant across test samples, treating prompts as static class labels rather than controllable instructions. Consequently, it remains unclear whether reported performance gains stem from genuine text-guided decision-making or merely from strong pretrained visual features augmented by a generic linguistic prior. The core problem addressed is the lack of a protocol capable of measuring whether language actually conditions the anomaly detection decision.

Methodology: The TGAD Benchmark
The authors introduce Text-Guided Anomaly Detection (TGAD), a structured benchmark designed to progressively increase the functional role of language in the inspection decision across three scenarios:

1. Scenario 1: Controlled Prompt-Sensitivity (MVTec AD): This scenario isolates whether predictions vary as a function of language. The visual input and ground truth are fixed while the textual input is systematically varied. Variants range from full descriptions to minimal queries, specifically testing the removal of the object noun, normality assertions, and defect lists.
2. Scenario 2: Per-Part Anomaly Detection (MVTec AD Extension): This scenario instructs the model to inspect a specific component of an object, treating defects in other components as normal. This forces the model to selectively follow textual constraints rather than reporting any visually evident defect. The authors release a component-tagged extension of MVTec AD with specific evaluation settings (EV1 and EV2) to test this.
3. Scenario 3: Industrial Robustness (Assembled Panel Dataset - APD): A new, realistic industrial dataset of electronic panels featuring interacting components. This scenario requires simultaneous knowledge of defect type and component location, representing the natural convergence of the first two scenarios under realistic conditions.

Evaluated Models
The study evaluates one representative model per multimodal paradigm:

• Generative LVLM: AnomalyGPT.
• Training-free Discriminative: LogSAD (augmented with a foreground-masking extension to enable per-part inspection).
• Embedding-adaptive Discriminative: AA-CLIP.

Key Findings and Results
Across all three paradigms and scenarios, the textual interface conditions the decision only superficially:

• Object-Anchor Collapse: In Scenario 1, prompt content is largely absorbed unless the object noun is removed. For AnomalyGPT, removing the object noun caused a significant drop in image-level AUROC (I-AUROC) from 97.4 to 82.6, while paraphrasing or removing defect lists had no effect. This indicates the model is anchored on the object noun alone, treating the prompt as a static prior rather than a dynamic instruction.
• Localization-Decision Dissociation: In Scenario 2, when instructed to ignore defects outside a specific component, image-level detection scores collapsed (e.g., AnomalyGPT I-AUROC dropped from 90.3 to 66.3), while pixel-level localization metrics (AUPRO) degraded significantly less. This suggests models can localize defects correctly but fail to suppress the global decision based on textual constraints.
• Industrial Failure (APD): In Scenario 3, under realistic load, image-level discrimination for all models fell well below MVTec levels, with AA-CLIP dropping to 31.5 (below chance). Text-guided prompting improved localization for AnomalyGPT but failed to condition the decision for any model.

Key Contributions

1. TGAD Benchmark and Datasets: The release of a structured benchmark with three escalating scenarios, a component-tagged extension of MVTec AD, and the new Assembled Panel Dataset (APD).
2. Critical Analysis: A demonstration that current multimodal AD models use text as a weak class-level prior rather than an inference mechanism. The study highlights the "Object-Anchor Collapse" and the "Localization-Decision Dissociation" as recurring failure signatures.
3. Diagnostic Signature: The identification of the Localization-Decision Dissociation, where image-level detection collapses under instruction stress while region-level localization remains relatively stable, revealing that text selects what the model finds but cannot tell it what to ignore.

Significance and Claims
The paper argues that standard benchmarks overstate the text-guided capabilities of current multimodal anomaly detection systems. The authors claim that for industrial deployment, a protocol like TGAD is a prerequisite to ensure models can be reliably controlled through language. They conclude that current systems do not treat instructions as control signals for the decision-making process; rather, the prompt merely selects what the model already finds visually. The paper posits that closing this gap requires architectures where the image-level decision is conditioned on the instruction, moving beyond naming objects to specifying correct configurations.