💻 computer science

Memory-Distilled Selection for Noise-Robust Anomaly Detection

This paper proposes Memory-Distilled Selection (MeDS), a robust training algorithm that leverages ensemble memory subsampling and score distillation to effectively filter contaminated data and achieve state-of-the-art anomaly detection performance under high noise ratios without requiring noise-specific hyperparameter tuning.

Original authors: Sirojbek Safarov, Jaewoo Park, Yoon Gyo Jung, Kuan-Chuan Peng, Wonchul Kim, Seongdeok Bang, Octavia Camps

Published 2026-05-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sirojbek Safarov, Jaewoo Park, Yoon Gyo Jung, Kuan-Chuan Peng, Wonchul Kim, Seongdeok Bang, Octavia Camps

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a quality control inspector at a factory. Your job is to spot defective products on a conveyor belt. To do this, you need to learn what a "perfect" product looks like.

The Problem: The Messy Classroom
Usually, you'd train your eyes by looking at thousands of perfect examples. But in the real world, it's hard to get a perfectly clean pile of good products. Sometimes, a few broken or weird items accidentally get mixed into your training pile.

If you try to learn from this messy pile, your brain gets confused. You might start thinking the broken items are actually "normal" because you saw them so often during training. This causes your inspection system to fail, missing real defects later on. Existing methods are like students who panic when they see a single wrong answer in their study guide; they get thrown off completely.

The Solution: MeDS (Memory-Distilled Selection)
The authors propose a new method called MeDS. Think of it as a three-step training program designed to ignore the noise and focus on the truth.

Step 1: The "Blurry Photo" Filter (Bootstrapped Memory)

Imagine you have a giant photo album of all the products. Instead of looking at the whole album at once (which includes the bad photos), you create hundreds of tiny, random snapshots of just a few pages from the album.

The Analogy: If you look at a crowd of people where 40% are wearing clown noses (the noise), and you take a random photo of just 10 people, there's a good chance you won't catch any clowns in that specific snapshot.
How it works: MeDS creates an "ensemble" (a team) of these tiny, sparse snapshots. Because the snapshots are so small and random, the "clowns" (defects) tend to get left out or diluted. The system averages these snapshots together. The result is a "low-pass filter"—a tool that smooths out the weird noise and keeps the clear, normal patterns. It gives a rough, coarse idea of what's normal, ignoring the messy outliers.

Step 2: The "Student Teacher" (Distillation)

Now, the system has a rough idea of what's normal, but it's a bit rigid because it's just looking at static snapshots. MeDS takes this rough idea and teaches a flexible, trainable "student" network (a type of AI) to mimic it.

The Analogy: Imagine a master painter (the memory snapshots) who can only paint broad, simple strokes. They teach an apprentice (the student network) to copy their work.
The Magic: Neural networks have a quirk called "early learning." When they start training, they learn the simple, common patterns first and only memorize the rare, weird ones later. By teaching the student to copy the "clean" master, the student learns to recognize normal products very quickly and accurately, while naturally ignoring the weird noise that hasn't been "learned" yet.

Step 3: The "Self-Correction" Loop (Progressive Selection)

Here is the final trick. The student network is now good, but if we let it keep training on the messy pile forever, it will eventually start memorizing the clowns (overfitting).

The Analogy: Imagine the student is taking a test. Instead of grading every single question, the system acts like a strict tutor. It looks at the student's answers, identifies which questions the student is sure are correct (the clean data), and says, "Okay, let's only practice on these questions for the next round."
How it works: The system constantly filters the training data. It picks out the samples that look the most "normal" based on the student's current score and trains only on those. As the student gets smarter, the filter gets stricter, allowing the system to train for a long time without ever getting confused by the noise. This allows for fine-grained detection—finding the exact pixel where a scratch is, not just saying "this whole image is bad."

The Results

The paper claims that MeDS is incredibly robust. Even when the training data is 40% contaminated with defects (a very messy scenario), MeDS still performs with near-perfect accuracy (99.16% on the MVTecAD benchmark).

It beats other methods that either:

Assume the data is mostly clean (and fail when it's not).
Assume the data is mostly dirty (and fail when it's clean).

MeDS doesn't need to know how much noise is in the data. It just works.

Bonus: Cleaning the Dataset

The authors also show that MeDS can be used as a tool for humans. By sorting the training images based on how "suspicious" they look, a human inspector can quickly find and remove the bad samples from their dataset with very little effort. It's like having a smart assistant that highlights the "clowns" in the photo album so you can throw them away.

In Summary:
MeDS is a smart training strategy that uses random sampling to ignore noise, teaches a flexible AI to learn the "normal" patterns first, and then constantly filters the training data to ensure the AI never learns the mistakes. It turns a messy, imperfect dataset into a highly reliable defect detector.

Technical Summary: Memory-Distilled Selection for Noise-Robust Anomaly Detection

Problem Statement

In industrial manufacturing, vision-based anomaly detection (AD) systems are critical for quality control. These systems are typically trained under the assumption that the training dataset consists exclusively of normal (defect-free) samples. However, in real-world scenarios, curating perfectly clean training sets is costly and often impractical; manual screening inevitably overlooks a non-negligible fraction of anomalous samples, leading to data contamination.

Existing methods for AD under noisy conditions suffer from significant performance degradation as the noise ratio increases. Current approaches generally rely on strong assumptions, such as:

Specific statistical properties of the training data (e.g., unimodal priors).
Prior knowledge of the anomaly ratio in the training set.
Knowledge of the score range for anomalous samples.

These dependencies necessitate dataset-specific hyperparameter tuning and often fail when noise levels deviate from expected ranges. Furthermore, many existing methods lack the capability to perform fine-grained pixel-level localization of defects under severe contamination.

Methodology: Memory-Distilled Selection (MeDS)

The authors propose Memory-Distilled Selection (MeDS), a training framework designed to maintain robust performance across diverse noise ratios for both image-level detection and pixel-level localization without requiring noise-ratio-specific hyperparameter tuning. The method proceeds in three sequential phases:

1. Bootstrapped Memory Construction (Coarse Filtering)

The first phase constructs an ensemble of partial memory banks via random subsampling of patch-level features extracted from the training data.

Mechanism: Instead of using a single full memory bank, MeDS creates $B$ partial memories ( $M_b$ ), each containing a random subset of features with a subsampling ratio $\rho$ (e.g., 0.1).
Theoretical Basis: The authors theoretically demonstrate that sparse subsampling acts as a low-pass filter. Under appropriate subsampling ratios, the expected distance gap between normal and anomalous features is maximized. The ensemble averaging of these sparse memories reduces variance and isolates nominal patterns, effectively downweighting contaminated samples at a coarse level.
Output: An ensemble anomaly score $s_M(x)$ that provides robust, coarse-level separation between normal and anomalous patches.

2. Memory Score Distillation (Initialization)

The second phase addresses the limitation that memory scores rely solely on frozen, pretrained features, which may not sufficiently capture domain-specific anomaly patterns.

Mechanism: A student network (reconstruction score network) is trained via score-space distillation. The student learns to minimize the L2 distance between its own reconstruction scores ( $s_\theta$ ) and the ensemble memory scores ( $s_M$ ).
Benefit: This process leverages the early-learning phenomenon of neural networks, where networks fit dominant patterns (normal data) before memorizing rare ones (noise). The distillation initializes the student network with a model that has already learned to separate normal from anomalous patterns, effectively denoising the memory scores.

3. Fine-Tuning with Progressive Selection (Refinement)

The third phase resolves the trade-off between early stopping (which preserves robustness but limits pixel-level precision) and extended training (which risks overfitting to noise).

Mechanism: The student network is fine-tuned on a self-selected subset of the training data ( $S_t$ ).
Selection Criterion: At each epoch, samples are selected based on a criterion $\eta_t(x)$ that interpolates between the initial distilled scores and the current reconstruction scores. A threshold $\tau_t$ , derived from the median and Median Absolute Deviation (MAD) of the scores, is used to filter out high-scoring (likely anomalous) samples.
Progressive Relaxation: The selection threshold is progressively relaxed over training iterations, allowing the model to incorporate more normal samples as its own score estimates become more reliable. This creates a positive feedback loop, enabling extended training for fine-grained localization without overfitting to the remaining noise.

Key Contributions

Contamination-Resilient Algorithm: The introduction of MeDS, which exploits the sparsity of bootstrapped memory ensembles as a low-pass filter to isolate nominal patterns, followed by refinement via distillation and iterative self-selection.
Comprehensive Empirical Validation: Extensive experiments across three industrial benchmarks (MVTecAD, VisA, Real-IAD) under varying noise ratios, including ablation studies dissecting each component's role and hyperparameter sensitivity analyses.
Theoretical and Practical Insights: A theoretical proof demonstrating how small-ratio memory subsampling yields a high-recall starting point under heavy contamination, alongside a demonstration of MeDS's utility for active label correction (efficiently identifying and removing contaminated samples with minimal human effort).

Experimental Results

MeDS was evaluated on MVTecAD, VisA, and Real-IAD with noise ratios ranging from 0% to 40%.

Robustness: Unlike baselines (e.g., SoftPatch, InReaCh, FUN-AD) which show significant performance drops as noise increases, MeDS maintains stable performance across all noise levels.
Performance:
- On MVTecAD, MeDS achieved 99.16% image-level AUROC at a 40% noise ratio, significantly outperforming state-of-the-art baselines.
- On VisA and Real-IAD, MeDS established state-of-the-art performance in both image-level and pixel-level metrics under noisy settings.
- When applied as a framework to other baselines (HVQ, Dinomaly, INP-Former), MeDS consistently improved robustness under contamination while incurring only marginal degradation on clean data.
Active Label Correction: In human-in-the-loop experiments, MeDS demonstrated superior ranking quality (AUPRC) and reduced inspection depth, allowing annotators to identify all contaminated samples by reviewing a significantly smaller fraction of the dataset compared to vanilla methods.

Significance and Claims

The paper claims that MeDS represents a systematic formulation of progressive selection based on memory distillation, tailored specifically for industrial anomaly detection. Its primary significance lies in:

Eliminating the need for noise-ratio-specific tuning: The method achieves robustness across a wide range of contamination levels without requiring knowledge of the noise ratio.
Bridging the gap between robustness and precision: By transitioning from low-pass filtering to fine-tuning with data selection, MeDS achieves both high image-level detection and precise pixel-level segmentation, a capability often lacking in existing noise-robust methods.
Practical Utility: Beyond automated detection, the framework serves as a practical tool for dataset curation, enabling efficient identification and removal of contaminated samples in industrial settings.

The authors acknowledge limitations, noting that MeDS may exhibit marginally lower performance on fully clean datasets compared to some baselines due to the median-based selection mechanism, and it incurs additional computational overhead for memory construction and distillation stages. However, these costs are justified in scenarios where training data is likely contaminated.