🤖 AI

AdapTS: Lightweight Teacher-Student Approach for Multi-Class and Continual Visual Anomaly Detection

AdapTS is a lightweight, unified Teacher-Student framework that addresses the challenges of multi-class and continual visual anomaly detection by utilizing a shared frozen backbone with dynamic, prototype-selected adapters, achieving state-of-the-art performance with significantly reduced memory overhead for efficient edge deployment.

Original authors: Manuel Barusco, Davide Dalle Pezze, Francesco Borsatti, Gian Antonio Susto

Published 2026-03-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Manuel Barusco, Davide Dalle Pezze, Francesco Borsatti, Gian Antonio Susto

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the quality control manager at a massive factory. Your job is to spot defects on a conveyor belt. But here's the catch: the factory doesn't just make one thing. It makes dozens of different products (toasters, shoes, circuit boards), and new products are added every week.

Traditionally, to do this job, you'd need a different expert inspector for every single product type.

The Old Way: You hire a "Toaster Expert," a "Shoe Expert," and a "Circuit Board Expert." When a new "Coffee Maker" arrives, you have to hire a whole new person and train them from scratch.
The Problem: This is expensive, takes up a huge amount of office space (memory), and is slow. If you try to teach one person to know everything at once, they get confused and forget how to spot defects on the toasters when they start looking at shoes.

This paper introduces AdapTS, a clever new way to solve this problem. Think of it as hiring one super-smart, permanent supervisor and giving them a set of tiny, magical cheat sheets.

The Core Idea: The "One Brain, Many Hats" Strategy

Instead of hiring a new person for every product, AdapTS uses one single, frozen "Brain" (a pre-trained AI model) that knows what "normal" looks like in general.

The Frozen Brain (The Teacher): Imagine a veteran inspector who has seen millions of perfect items. They are so experienced that they never need to be retrained; their knowledge is "frozen" in place. They act as the standard for perfection.
The Lightweight Hats (The Adapters): Instead of training a whole new brain for a new product, we just clip a tiny, lightweight "adapter" (a small cheat sheet) onto the main brain.
- When a Toaster comes down the line, we clip the "Toaster Hat" onto the brain.
- When a Shoe comes down, we swap it for the "Shoe Hat."
- These hats are so small they take up almost no space, but they teach the brain exactly what to look for in that specific product.

How It Detects a Flaw

The system works like a game of "Spot the Difference":

The Brain looks at a product and says, "This is what a perfect toaster should look like."
The Hat (the student part) tries to mimic the Brain but is slightly tuned to the specific toaster.
If the toaster has a scratch, the Hat gets confused because the scratch doesn't match the "perfect" pattern. The system spots the discrepancy between what the Brain expects and what the Hat sees. That confusion is the alarm bell: "Defect Found!"

The "Magic Trick" for New Products (Continual Learning)

In the real world, factories change. New products arrive constantly.

Old methods would forget how to inspect the old products when learning the new ones (like a student who studies for a math test and forgets how to read).
AdapTS is different. Because the main "Brain" is frozen and the "Hats" are separate, learning a new product is as easy as just making a new Hat. The Brain remembers everything about the Toaster, and the new Hat handles the Coffee Maker. They don't fight each other.

The "Cheat Sheet" for Speed

The paper also mentions a cool trick called Task Identification.
Imagine you walk into the factory and see a product, but you don't know what it is. How does the system know which "Hat" to put on?

AdapTS keeps a tiny "ID card" (a prototype) for every product it has ever seen.
When a new item arrives, the system quickly glances at it, matches it to the closest ID card, and instantly snaps on the correct Hat. It does this with 99% accuracy, so the system never gets confused about which product it's inspecting.

Why Is This a Big Deal? (The "Backpack" Analogy)

The biggest win is size.

Old Methods: To inspect 15 different products, old systems might need a backpack weighing 1,120 MB (like carrying a heavy laptop).
AdapTS: Our system only needs a backpack weighing 8 MB (like carrying a single sheet of paper).

It's 149 times lighter than the previous best methods! This means you can run this system on a small, cheap computer right on the factory floor (an "edge device") instead of needing a massive, expensive server room.

Summary

AdapTS is like having one genius supervisor who never forgets anything, equipped with a drawer full of tiny, specialized cheat sheets. It can switch between inspecting toasters, shoes, and new inventions instantly, using a fraction of the memory and space required by older methods. It's the perfect solution for factories that are always changing and always need to be efficient.

1. Problem Statement

Visual Anomaly Detection (VAD) is critical for industrial quality control, aiming to identify defects in products without requiring costly pixel-level annotations (unsupervised learning). However, existing VAD methods face three major limitations in real-world industrial environments:

Single-Category Limitation: Most methods train a dedicated model per product category ("one-category-one-model"), which is inefficient for lines processing multiple product types simultaneously.
Continual Learning Gap: Existing solutions struggle to adapt to new product categories over time without catastrophic forgetting or retraining from scratch.
High Resource Overhead: State-of-the-art Teacher-Student (TS) and hybrid methods (e.g., STFPM, RD4AD, DeSTSeg) often require massive memory footprints (hundreds of MBs to GBs) for separate student networks or replay buffers, making them unsuitable for edge deployment on resource-constrained devices.

Currently, no unified framework exists that simultaneously addresses multi-class and continual learning scenarios within the efficient Teacher-Student paradigm.

2. Methodology: AdapTS

The authors propose AdapTS, a unified framework that modifies the traditional Teacher-Student (TS) architecture to be parameter-efficient and adaptable.

A. Unified Teacher-Student Architecture with Adapters

Instead of training a separate, full-sized student network (which duplicates the teacher's parameters), AdapTS utilizes a single shared frozen backbone (pretrained on ImageNet) that acts as both the teacher and the student.

Mechanism: Lightweight, trainable adapter modules are injected directly into the student pathway of the frozen teacher network.
Training Objective: The adapters are trained to reconstruct the teacher's own intermediate feature representations. The loss function minimizes the discrepancy between the frozen teacher features and the adapter-modified student features.
Adapter Variants: The paper explores three adapter designs based on a residual structure with $1\times1$ $1 \times 1$ convolutions:
- Linear: Preserves channel dimension.
- Expansion: Projects features to a higher dimension before returning.
- Bottleneck: Compresses features to focus on informative signals.
Quantization: Adapters can be INT-8 quantized post-training to further reduce memory without significant performance loss.

B. Segmentation-Guided Training

To improve localization accuracy and feature separation, the authors introduce a segmentation-guided objective:

Synthetic Anomalies: During training, normal images are augmented with synthetic anomalies generated via Perlin noise and random textures.
Auxiliary Network: A simple segmentation head (1x1 convolution + sigmoid) is attached to the feature differences to predict anomaly masks.
Loss Function: The total loss combines the standard feature discrepancy loss ( $L_{STFPM}$ ) with a segmentation loss ( $L_{seg}$ ) comprising Focal Loss and L1 Loss.
Inference: The segmentation head is discarded after training; only the adapters are used for inference, ensuring no runtime overhead.

C. Task Identification for Multi-Class & Continual Learning

To handle multiple categories or a stream of new tasks without retraining the backbone:

Prototype-Based Selection: For each task (category), a feature prototype is computed by averaging the teacher's feature vectors on normal training samples.
Inference: Given a test image, the system calculates the similarity (Euclidean distance) between the image's features and the stored prototypes. The adapter corresponding to the most similar prototype is dynamically selected.
Result: This allows the model to switch between tasks instantly with 99% accuracy in task identification.

3. Key Contributions

Unified Framework: AdapTS is the first TS-based method to natively support both multi-class and continual learning settings within a single architecture.
Parameter Efficiency: By replacing the full student network with lightweight adapters, AdapTS drastically reduces the number of trainable parameters and memory usage.
Edge Deployment Optimization: The method is explicitly designed for edge devices, offering variants (AdapTS-S) that require minimal additional memory.
Novel Training Strategy: The integration of synthetic anomaly generation and a segmentation-guided loss improves feature discriminability without increasing inference complexity.

4. Experimental Results

Experiments were conducted on the MVTec AD and VisA benchmarks across Single, Multi-Class, and Continual scenarios.

Performance vs. Memory Trade-off

Accuracy: AdapTS achieves performance comparable to state-of-the-art methods (STFPM, RD4AD, DeSTSeg).
- On MVTec (Single Class), AdapTS-L achieves an Image-ROC of 0.88, matching STFPM (0.88) and DeSTSeg (0.98) while using significantly less memory.
- On VisA (Multi-Class), AdapTS-L achieves an Image-ROC of 0.83, outperforming STFPM (0.79) and matching DeSTSeg (0.87) in Pixel-ROC.
Memory Footprint: This is the primary advantage.
- AdapTS-S (lightest variant) requires only 8 MB of additional memory.
- Comparison:
  - 13x less than STFPM (95 MB).
  - 48x less than RD4AD (360 MB).
  - 149x less than DeSTSeg (1120 MB).
- In the Continual setting, AdapTS maintains low memory usage while other methods require large replay buffers (e.g., STFPM needs ~110 MB additional memory).

Ablation Studies

Adapter Placement: Placing adapters on layers 2 and 3 yields the best performance.
Quantization: INT-8 quantization reduces memory by ~4x with negligible accuracy drop (Image-ROC drops from 0.88 to 0.86).
Task Identification: The prototype-based mechanism achieves 99% accuracy, proving that anomaly categories are linearly separable in the feature space.

5. Significance and Conclusion

AdapTS represents a paradigm shift in industrial anomaly detection by solving the scalability bottleneck of existing TS methods.

Industrial Impact: It enables the deployment of sophisticated VAD models on edge devices (e.g., cameras, embedded controllers) where memory and compute are limited, a requirement for modern flexible manufacturing lines.
Scalability: It eliminates the need to train and store separate models for every new product, supporting a "one-model-fits-all" approach that adapts over time.
Future Directions: The authors suggest exploring model weight merging and other parameter-efficient training strategies to further optimize the framework.

In summary, AdapTS successfully bridges the gap between high-performance anomaly detection and the strict resource constraints of real-world industrial edge computing, offering a scalable, memory-efficient, and accurate solution for multi-class and continual learning environments.