🤖 machine learning

Memory-Bounded Continuation of Greedy Sampling for Continual Anomaly Detection

This paper introduces ContCore, a continual anomaly detection method that maintains a fixed memory budget by iteratively applying greedy sampling to new and existing data, thereby preserving coreset representativeness with theoretical guarantees and achieving state-of-the-art performance across multiple benchmarks.

Original authors: Yoon Gyo Jung, Jaewoo Park, Kuan-Chuan Peng, Seongdeok Bang, Octavia Camps

Published 2026-08-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yoon Gyo Jung, Jaewoo Park, Kuan-Chuan Peng, Seongdeok Bang, Octavia Camps

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the factories that build our cars, the hospitals that scan our bodies, and the networks that secure our data, a silent guardian is needed: a system that can spot the one thing that is wrong. This is the job of anomaly detection. Unlike a security guard who knows exactly what a burglar looks like, these systems are often trained only on the "normal" things—the perfect gears, the healthy tissues, the clean code. They learn the shape of what is right so well that when something deviates, even slightly, they raise an alarm. The challenge grows when the environment changes. Imagine a factory that starts producing a new type of engine part, then another, and another. A system trained only on the first part might forget how to recognize it once it learns the new ones, or it might simply run out of room in its memory to store the details of everything it has ever seen. This is the problem of "catastrophic forgetting," where learning new things causes a machine to lose its knowledge of old things, and the problem of "memory limits," where a system cannot keep an infinite list of examples.

Researchers have long tried to solve this by creating a small, perfect summary of all the normal data, a collection of key examples that represents the whole. The best way to build such a summary is to pick examples that are as different from each other as possible, ensuring that no part of the "normal" world is left out. However, when new tasks arrive one after another, keeping this summary perfect without letting it grow infinitely large is incredibly difficult. If you simply add new examples, the memory fills up. If you delete old ones to make room, you risk losing the ability to recognize the original tasks. A team of researchers led by Yoon Gyo Jung and colleagues has found a way to keep this summary both small and perfect, no matter how many new tasks the system encounters. They developed a method called ContCore, which allows a machine to learn a continuous stream of new normal patterns without ever forgetting the old ones, all while using a fixed amount of memory.

The core of their discovery lies in how they update the memory. Instead of trying to store every single new example or using complex neural networks that can easily forget, they use a simple but powerful strategy of selection. When a new batch of normal data arrives, the system first looks for the new examples that are most different from what it already remembers. It picks these "most distant" examples to add to its collection. Then, to ensure the total number of examples stays within the strict memory limit, it performs a second selection. It looks at the entire collection—the old examples plus the new ones—and picks out the most spread-out group of examples that fits the size limit. This two-step process, which they call "greedy expansion" followed by "greedy consolidation," ensures that the memory always contains the most representative samples of everything it has seen so far.

The researchers showed that this method works because it preserves the "shape" of the normal data. Even though the system discards many examples at every step to stay within its memory budget, the ones it keeps are chosen so carefully that they still cover the entire range of normal variations. They proved mathematically that the gap between this small, constantly updated memory and a perfect, unlimited memory is always small and controlled. In practice, this means the system does not suffer from the catastrophic forgetting that plagues other methods. When tested on a wide variety of tasks, from simple single-class updates to complex sequences of many different classes, their system consistently outperformed existing state-of-the-art methods. On standard industrial datasets, it achieved higher accuracy in spotting defects while forgetting almost nothing of what it learned previously.

What makes this approach particularly robust is its efficiency. Unlike other methods that require massive amounts of computing power or huge amounts of data to retrain, this system updates its memory with a simple selection process. It does not need to re-learn old tasks or store vast amounts of raw images. In tests where the system had to learn from data one image at a time, a scenario where other methods failed significantly, ContCore maintained high performance. The researchers also demonstrated that this method works well across different types of data and hardware, proving that the principle of carefully selecting a small, representative group is more powerful than simply throwing more computing power at the problem. By keeping the memory bounded but the representativeness high, they have provided a reliable way for machines to adapt to a changing world without losing their past.

Technical Summary: Memory-Bounded Continuation of Greedy Sampling for Continual Anomaly Detection

Problem Statement

Unsupervised Anomaly Detection (AD) is critical for industries like manufacturing and cybersecurity, yet discovering decision boundaries between normal and unbounded anomalies remains challenging. While early models achieved high performance by training separate models per class, they suffer from high annotation costs and large model sizes. Multi-class AD methods address this by training single models on multiple classes but fail in dynamic environments where new tasks arrive sequentially due to catastrophic forgetting.

The specific challenge in Continual Anomaly Detection (CAD) is preventing catastrophic forgetting while detecting newly introduced anomalies under an unsupervised setup (no class labels, only normal training images). Existing approaches face significant limitations:

Regularization-based methods (e.g., EWC, SI) struggle to balance stability and plasticity and are sensitive to hyperparameters.
Replay-based methods suffer from scalability issues as memory usage grows proportionally with the number of tasks.
Coreset-based methods (e.g., DNE, UCAD) avoid re-training but face memory overflow as tasks accumulate, as they typically require unbounded memory to maintain representativeness.
Dedicated CAD solutions (e.g., CDAD, IUF) often suffer from catastrophic forgetting in long-term schedules or require class labels, limiting their applicability.

The core question addressed is: Can we design a bounded coreset that preserves the representational quality of an unbounded one?

Methodology: ContCore

The authors propose ContCore, a memory-based anomaly detector that constructs a greedy-continued coreset under strict, fixed memory constraints. The method relies on the observation that continued greedy sampling—iteratively applying greedy selection over previously greedy-sampled sets—effectively preserves representativeness.

Core Mechanism

ContCore operates in two steps at each task transition $t$ :

Greedy Expansion: Given a set of patch embedding features $Z_t$ from the new task, the method selects features that are maximally distant from the current coreset $M_{t-1}$ . This is formalized as greedy sampling on a base set:
$G_{n_t}(Z_t; M_{t-1}) = \{z^*_i \in Z_t : z^*_i = \arg\max_{z \in Z_t} d(z, Z^*_{i-1} \cup M_{t-1})\}$
where $n_t$ is the sampling size for the current task.
Greedy Consolidation: The union of the newly expanded features and the previous coreset is processed to enforce the fixed memory budget $m$ . The method selects the maximally separated features from this combined set:
$M_t = G_m(G_{n_t}(Z_t; M_{t-1}) \cup M_{t-1})$

Practical Implementation

Efficiency Approximation: To mitigate the computational cost of strict greedy sampling during consolidation, the authors approximate the process using nearest neighbors. A fraction $q$ of samples is selected based on nearest-neighbor distances rather than full greedy selection, significantly reducing time complexity while maintaining coverage.
Inference: Anomaly scores are computed based on the nearest distance to the current coreset $M_t$ for each patch, following the PatchCore approach.

Theoretical Justification

The paper provides a theoretical guarantee (Theorem 4.1) showing that the resulting greedy-continued coreset $M_T$ approximates the oracle coreset (the result of greedy sampling on all data from all tasks combined) within a bounded gap. The Hausdorff distance between the oracle and the constructed coreset is shown to be bounded by the sum of per-task approximation errors, proving that representativeness degrades gracefully rather than catastrophically.

Key Contributions

Observation & Theory: The authors demonstrate that continued greedy sampling preserves representativeness under strict memory limits. They provide theoretical justification showing the greedy-continued coreset approximates the oracle coreset within a bounded gap.
ContCore Algorithm: The instantiation of this principle into a method that constructs a greedy-continued coreset via greedy expansion and consolidation, maintaining fixed memory usage.
Empirical Performance: ContCore achieves state-of-the-art performance across 11 different task schedules on the MVTecAD and VisA datasets.
Robustness in Online Settings: The method is validated in online continual AD settings (single-pass, batch size 1), where prior methods degrade significantly.

Experimental Results

Experiments were conducted on MVTecAD and VisA using 11 different task schedules (ranging from simple incremental $1 \times 15$ to complex mixed schedules like $10 \times 1 \times 5$ ).

Performance: ContCore achieved the highest task-average AUROC and the lowest Forgetting Measurement (FM) across almost all schedules. For instance, on the MVTecAD $1 \times 15$ schedule, ContCore achieved an image-level AUROC of 98.8%, outperforming the next best (UCAD at 93.0%) and significantly reducing forgetting (FM: 0.1 vs. 1.0 for UCAD).
Cross-Dataset: In cross-dataset setups (MVTecAD $\to$ VisA and vice versa), ContCore surpassed previous baselines like CDAD, showing a 8.9% margin improvement in VisA $\to$ MVTecAD image-level AUROC.
Efficiency: ContCore is highly efficient compared to neural methods. It requires significantly fewer GFLOPs (9.24 vs. 5628 for CDAD) and parameters (45.3M vs. 11G for CDAD), with faster training and inference times.
Large-Scale Validation: On the Real-IAD dataset (30 classes, ~100k images), ContCore outperformed both UniAD-Oracle and PatchCore-OneClass models despite being trained with continual task schedules.
Online Learning: In the strict "online CAD" setting (batch size 1, single epoch), ContCore maintained high performance (97.2% AUROC on MVTecAD), whereas other methods (DNE, UCAD, IUF, CDAD) suffered significant performance drops.

Significance and Claims

The paper claims that ContCore offers a robust solution to the memory-constrained continual anomaly detection problem. Its significance lies in:

Breaking the Memory-Performance Trade-off: Unlike coreset accumulation which requires unbounded memory, or regularization methods which suffer from forgetting, ContCore maintains a fixed memory footprint while achieving state-of-the-art performance.
Theoretical Guarantees: The method is not just empirical; it is backed by a proof that the representational gap to the optimal oracle coreset remains bounded.
Generalizability: The approach is effective across diverse task schedules, different backbone architectures (WideResNet, EfficientNet, DINOv2), and even in challenging online learning scenarios where data is processed only once.
Practical Deployment: By avoiding catastrophic forgetting and unbounded memory growth, ContCore is presented as a viable solution for real-world deployment in dynamic environments where new anomaly classes are introduced over time without access to historical data.