💻 computer science

DriftAD: Visually-Guided Text Drift for Few-Shot Industrial Anomaly Detection

DriftAD is a few-shot industrial anomaly detection framework that overcomes the limitations of static text prompts by introducing a Visually-Guided Text Drift mechanism to dynamically generate spatially and layer-wise adaptive anomaly descriptors, significantly improving detection performance on MVTec-AD and VisA datasets.

Original authors: Wenyang Liu, Tianyi Liu, Dongshuo Zhang, Kejun Wu, Adams Wai-Kin Kong

Published 2026-08-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wenyang Liu, Tianyi Liu, Dongshuo Zhang, Kejun Wu, Adams Wai-Kin Kong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast, automated world of modern manufacturing, keeping products perfect is a constant battle against the invisible. Machines assemble thousands of identical items every hour, but the moment a tiny scratch appears on a metal surface or a subtle discoloration mars a fabric, the entire batch risks failure. For decades, computers have struggled to spot these flaws because they are rare and unpredictable. Traditional methods required teaching a computer to recognize what "normal" looks like for every single type of product, a slow and expensive process that broke down as soon as a factory introduced a new item. Recently, a new approach emerged using artificial intelligence models that were originally trained to understand both images and language. These models can read a description like "a damaged bottle" and look for it in a picture, offering a flexible way to find defects without needing thousands of examples. However, even these advanced systems have a blind spot: they tend to treat a defect as a single, uniform idea, failing to notice that a flaw might look different depending on where it is or how deep it sits within the layers of the image data.

Researchers at Nanyang Technological University and Huazhong University of Science and Technology have developed a new system called DriftAD to fix this specific weakness. Their work, presented for the 2026 ACM International Conference on Multimedia, introduces a way to make the computer's understanding of a defect "drift" or shift dynamically as it looks at different parts of an image. Instead of using a single, static description of a flaw that stays the same no matter what the computer sees, DriftAD allows that description to change based on the local visual context. Imagine a security guard who, instead of holding a single, rigid description of a suspect, adjusts their mental image of that suspect in real-time as they scan different areas of a crowd, accounting for lighting, distance, and angle. In the same way, this new method takes a frozen, unchanging text description of a defect and gently nudges it to match the specific visual details of the area it is currently examining.

The process begins by sharpening the computer's view of the image. The system first uses specialized branches to highlight subtle signals that might otherwise be hidden by the dominant, perfect patterns of a normal product. It looks at the image through both a spatial lens, comparing each small patch to its immediate neighbors to find deviations, and a frequency lens, analyzing the underlying patterns of the image to spot irregularities that the eye might miss. Once these faint defect signals are amplified, the system engages its core innovation: Visually-Guided Text Drift. Here, the computer takes the standard text description of an anomaly and, guided by the visual features it just amplified, shifts that description into a new, customized version for every single layer of its analysis. This means that as the computer peels back the layers of the image, from the broad shapes down to the fine textures, the definition of what constitutes a "defect" evolves to fit the specific scale and location of the potential flaw.

To ensure these shifting descriptions are useful, the system uses them as probes to scan the image again. It asks the visual features, "Does this part of the image match the current, customized description of a defect?" If the answer is yes, that part of the image is highlighted; if not, it is ignored. This creates a highly focused map of where the problem lies, filtering out the noise of the background. The researchers tested this approach on two major industrial datasets, MVTec-AD and VisA, which contain thousands of images of various products ranging from metal nuts to cables and carpets. They challenged the system with very limited data, providing it with only one, two, or four examples of a perfect product to learn from, a scenario known as few-shot learning.

The results were decisive. In tests measuring how well the system could identify defective images and pinpoint the exact location of the flaw, DriftAD outperformed all existing methods across every setting. On the MVTec-AD dataset, the system achieved an image-level accuracy score of 97.2 percent with just one example, and a pixel-level precision of 96.8 percent. On the VisA dataset, which features more complex and varied defects, it reached 93.1 percent accuracy for image detection and 97.4 percent for locating the specific pixels of the defect. These numbers represent a significant leap forward, surpassing the previous best methods by a clear margin. The study confirms that by allowing the text description of a defect to move and adapt to the visual reality of the image, rather than forcing the image to fit a rigid description, machines can detect industrial flaws with a level of precision that was previously out of reach. This approach does not require retraining the entire system for every new product, making it a scalable solution for the dynamic needs of modern manufacturing.

Technical Summary: DriftAD

Problem Statement

Few-shot anomaly detection (FSAD) in industrial settings aims to identify and localize rare defects using only a small number of normal support images. While recent approaches leverage Vision-Language Models (VLMs) like CLIP to achieve open-set generalization, existing methods suffer from a fundamental limitation: static text representations.

Current VLM-based methods (e.g., WinCLIP, PromptAD) typically rely on globally shared, static text embeddings or learnable prompts that are applied uniformly across all spatial locations and encoder depths. This "global-to-local" matching fails to capture the intrinsic nature of industrial defects, which are highly localized, scale-dependent, and context-specific. Consequently, these static descriptors lack awareness of the precise location and morphology of a defect in a specific test image, leading to diffuse anomaly maps that struggle to distinguish fine-grained defect regions from surrounding normal patterns.

Methodology

The authors propose DriftAD, a framework designed to transform frozen CLIP text embeddings into layer-wise, spatially-adaptive anomaly descriptors conditioned on the local visual context of each input image. The architecture consists of four primary components:

1. Anomaly Signal Amplification (ASA)

Before text-visual matching, subtle defect signals often suppressed by dominant normal patterns must be exposed. ASA enhances the visual features ( $V_\ell$ ) through two parallel branches:

Spatial Branch: Computes multi-scale residuals between features and their smoothed backgrounds (via 3×3 and 9×9 pooling) to highlight deviating regions.
Frequency Branch: Decomposes features into magnitude and phase spectra via 2D FFT, adaptively reweighting low- and high-frequency components to enhance abnormal spectral content.
The outputs are adaptively combined to produce amplified features ( $A_\ell$ ).

2. Visually-Guided Text Drift (VGTD)

This is the core innovation. Instead of using a fixed text token, VGTD dynamically "drifts" the frozen normal ( $t_+$ ) and abnormal ( $t_-$ ) text embeddings based on the visual context.

Context Extraction: Visual context is extracted from $A_\ell$ via adaptive pooling and fused top-down from deep to shallow layers to ensure semantic richness.
Drift Generation: A dedicated generator produces normal and abnormal drift fields ( $\Delta_{\ell, \pm}$ ) for each encoder depth.
Modulation: The frozen embeddings are displaced along these drift fields: $T_{\ell, \pm} = \text{Norm}(t_{\pm} + \Delta_{\ell, \pm} \cdot \rho)$ .
This results in descriptors ( $T_{\ell, \pm}$ ) that are unique to each spatial location and encoder depth, adapting to the specific defect pattern of the input image.

3. Drift-Guided Spatial Gating (DGSG)

To ensure the visual features align with the newly drifted text space, DGSG uses the drifted abnormal descriptor ( $T_{\ell, -}$ ) as a spatial probe.

It computes pixel-wise cosine similarity between the amplified visual features ( $A_\ell$ ) and $T_{\ell, -}$ to generate attention gates ( $G_\ell$ ).
These gates selectively enhance anomaly-relevant regions in the visual features, producing refined features ( $\tilde{A}_\ell$ ) that are better aligned with the drifted text descriptors.

4. Multi-Branch Anomaly Scoring (MBAS)

The final anomaly map is synthesized by fusing three complementary branches:

Clean Branch: Matches scale-aware features with frozen text embeddings (preserving zero-shot generalization).
Drift Branch: Matches refined features ( $\tilde{A}_\ell$ ) with the drifted descriptors ( $T_{\ell, \pm}$ ), capturing defect-specific signals.
Support Branch: (Inference only) Uses a memory bank of support set patches to measure nearest-neighbor distances.

Training Objectives

In addition to standard classification and segmentation losses, DriftAD introduces two specific losses:

Drift Separation Loss: Prevents the drifted normal and abnormal descriptors from collapsing into similar representations, enforcing a margin relative to the original frozen embeddings.
Gate Supervision Loss: Directly supervises the attention gates in DGSG to ensure spatially discriminative gating.

Key Contributions

Visually-Guided Text Drift (VGTD): A novel mechanism that shifts frozen CLIP embeddings into layer-wise, spatially-adaptive descriptors conditioned on local visual context, addressing the rigidity of static text prompts.
Drift-Guided Spatial Gating (DGSG): A module that uses the drifted abnormal descriptor as a spatial probe to selectively enhance anomaly-relevant visual features, resolving the text-visual mismatch.
Anomaly Signal Amplification (ASA): A preprocessing module that exposes subtle defect signals via spatial and frequency branches before matching.
Regularization Strategies: The introduction of drift separation and gate supervision losses to ensure robust optimization and prevent representational collapse.

Experimental Results

The method was evaluated on MVTec-AD and VisA datasets across 1-shot, 2-shot, and 4-shot settings.

Performance: DriftAD achieves state-of-the-art (SOTA) performance across all shot settings on both image-level (AUROC, AUPR) and pixel-level (pAUROC, PRO) metrics.
Comparisons: It outperforms existing prompt-based methods (e.g., PromptAD, AnomalyGPT) and visual-enhancement methods (e.g., KAG-prompt, FocusPatch-AD). Notably, in the 1-shot setting on MVTec-AD, it improves average AUROC by up to 1.5% over the best competing method (KAG-prompt).
Few-Shot vs. Full-Shot: With only 4-shot support, DriftAD achieves competitive or superior performance compared to full-shot baselines (e.g., DeSTSeg, OmniAL) and significantly outperforms 8-shot methods.
Qualitative Analysis: Visualizations show that DriftAD produces sharper anomaly maps with reduced background interference compared to baselines, particularly for small, scattered defects on complex objects (VisA).

Significance and Claims

The paper claims that DriftAD fundamentally rethinks the anomaly text representation in VLM-based detection. By moving away from fixed global tokens to dynamically drifting, spatially-adaptive descriptors, the method aligns the semantic strength of CLIP with the localized, context-dependent nature of industrial defects.

The authors emphasize that this approach allows the model to:

Adapt text descriptors to the specific shape, scale, and spatial distribution of defects in each image.
Achieve precise text-patch alignment without sacrificing the zero-shot generalization capabilities of the pre-trained CLIP model.
Effectively bridge the gap between global semantic priors and local visual anomalies, resulting in superior few-shot detection and localization performance.

The work is presented as a solution to the "rigid global-to-local matching" problem inherent in current VLM-based FSAD methods, demonstrating that conditioning text embeddings on local visual context is critical for high-precision industrial anomaly detection.