💻 computer science

CDGP: Contrastive Dual Gaussian Processes for Weakly Supervised Anomaly Segmentation

The paper proposes Contrastive Dual Gaussian Processes (CDGP), a weakly supervised framework that combines a posterior-dominance statistic derived from dual Gaussian processes with hierarchical reconstruction residuals to achieve state-of-the-art anomaly segmentation without requiring pixel-level annotations.

Original authors: Seungjun Chu, Seokhee Han, Mateusz Nowak, Peter Chin

Published 2026-08-20

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Seungjun Chu, Seokhee Han, Mateusz Nowak, Peter Chin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the bustling world of modern manufacturing, machines are increasingly tasked with the delicate job of inspecting products for flaws. A factory might produce thousands of identical components every hour, and the goal is to catch the rare defective one before it reaches a customer. This requires two things: deciding if a product is broken, and pinpointing exactly where the break is. For decades, the most reliable way to teach computers to do this has been to show them thousands of perfect examples and let them learn what "normal" looks like. When the computer sees something that deviates from that normal pattern, it flags it as a defect. This approach works well because perfect items are easy to find, while broken ones are rare and varied. However, this method has a blind spot. It often struggles to tell the difference between a genuine crack and a harmless, unusual texture that happens to look a bit strange. It sees both as "not normal" and raises an alarm for both, leading to confusion and wasted time.

Researchers have long wondered if they could improve this by showing the computer examples of broken items, even if they didn't know exactly where the cracks were. In the real world, factories often know which batch of products failed a quality check, even if they haven't taken the time to draw a map of every single flaw. This is the challenge of "weakly supervised" learning: using the broad label of "defective" to teach the machine to find the specific location of the error without a detailed map. A new study introduces a method called Contrastive Dual Gaussian Processes, or CDGP, which tackles this problem by changing how the computer compares the good and the bad. Instead of just asking how strange an image looks, the system asks a more direct question: does this part of the image look more like a defect or more like a normal product?

The researchers built a system that treats every small patch of an image as a candidate for being either normal or defective. Imagine the image as a grid of tiny tiles. For each tile, the system runs two separate mental models simultaneously. One model is trained to predict what a normal tile should look like, while the other is trained to predict what a defective tile should look like. These models are not just guessing; they are built on a statistical framework that allows them to express not just a prediction, but also how confident they are in that prediction. The system then compares the two predictions for every single tile. If the "defective" model is much more likely to be right than the "normal" model, and the system is confident in that difference, that tile is marked as a flaw. This approach is powerful because it forces the computer to weigh the evidence for a defect against the evidence for normality, rather than just looking for anything that looks odd.

To make this work without detailed maps of where the defects are, the team had to be clever about how they started the training. They began by scanning the defective images and picking out the tiles that looked the most different from the normal ones. These became the initial "candidates" for the defective model to learn from. It was a noisy starting point, as some of these candidates were just normal parts that happened to look slightly different, but the system was designed to learn through the process. As the training continued, the system used the simple "defective" or "normal" label for the whole image to refine its understanding. It learned to push the predictions for the defective model higher in the areas that actually contained flaws, while keeping the normal model strong in the background areas. This process allowed the system to gradually sharpen its focus, separating the true defects from the background noise without ever seeing a single hand-drawn outline of a crack.

The study also recognized that looking at the big picture isn't always enough to find thin, hairline cracks. To solve this, they added a second layer of inspection that focuses on reconstruction. This part of the system tries to rebuild a perfect version of the image from a slightly damaged version. If the system struggles to rebuild a specific area, it suggests that area was unusual. By combining the direct comparison of the two models with this reconstruction check, the system gets a double dose of evidence. The researchers tested this method on several standard industrial datasets, including MVTec AD 2, KSDD2, and VisA. The results were striking. On the MVTec AD 2 dataset, which is a rigorous benchmark for this type of work, the new method outperformed all other approaches tested, ranking first in every metric used to measure how well the system could locate defects. It also performed at the top level or very competitively on the other two datasets.

What makes this finding particularly significant is how it handles the uncertainty. The system doesn't just output a score; it calculates a standardized measure of how much more likely a defect is than normality, taking into account how sure the models are. This prevents the system from being fooled by random noise. The researchers also showed that this improvement wasn't just due to a more complex setup, but specifically because of the way they contrasted the two models and calibrated their confidence using only the training data. They verified that the system works without needing any human to draw boxes around defects during the learning phase, relying solely on the simple "good" or "bad" labels that factories already have.

The study concludes that by explicitly modeling both the normal and the defective possibilities and comparing them directly, machines can learn to find flaws much more accurately than before. This approach bridges the gap between the abundance of normal data and the scarcity of detailed defect maps. It suggests that in industrial settings, where time and resources for detailed annotation are limited, we can achieve high-precision inspection by teaching computers to weigh the evidence for a flaw against the evidence for normality, rather than just looking for anything that stands out. The method stands as a robust solution that ranks at the top of current performance benchmarks, offering a practical path forward for automated quality control in manufacturing.

Problem Statement

Industrial visual inspection requires both detecting whether a product is defective and localizing the specific defect regions. While pixel-level anomaly segmentation is crucial for downstream tasks like diagnosis and quality control, collecting dense pixel-level masks is costly and time-consuming in real manufacturing environments. Consequently, most existing methods rely on a "normal-only" learning paradigm, training exclusively on defect-free images and identifying anomalies as deviations from normality.

However, the normal-only paradigm has limitations:

Ambiguity: Both true defects and unusual-but-normal regions can deviate significantly from the learned normal distribution, receiving similarly high anomaly scores.
Supervision Shift: While avoiding pixel masks, normal-only training still requires curated datasets free of defects, shifting the burden from annotation to dataset curation.
Weak Supervision Gap: In many pipelines, image-level labels (normal/defective) are available but pixel masks are not. Discarding defective images in these scenarios forfeits useful information about anomaly appearance.
Localization Challenge: Standard weakly supervised approaches (e.g., using Class Activation Maps from image classifiers) often highlight discriminative context sufficient for image-level recognition but fail to align with actual defect regions, especially since industrial anomalies are heterogeneous and lack coherent visual categories.

Methodology: Contrastive Dual Gaussian Processes (CDGP)

CDGP is a weakly supervised framework that models both normal and anomaly predictive distributions over dense multi-scale tokens. Instead of scoring deviations from normality, it scores the posterior dominance of an anomaly function over a normal function.

1. Dense Multi-Scale Token Representation

The method uses a WideResNet-50 backbone to extract feature maps from three stages. These are projected to a shared latent dimension, normalized, and aligned to a single spatial resolution to form a dense grid of unit-norm tokens ( $t_{hw}$ ).

2. Hard-Candidate Inducing Initialization

CDGP employs Sparse Gaussian Processes (GPs) with fixed inducing points.

Normal GP ( $Z_N$ ): Inducing tokens are selected from normal images using greedy farthest-point sampling to cover the nominal token space.
Anomaly GP ( $Z_A$ ): Since defective images contain mostly normal background, direct sampling is ineffective. Instead, tokens from defective images are scored by their cosine distance from $Z_N$ . The top 20% most dissimilar tokens form a "noisy candidate pool," from which $Z_A$ is selected via farthest-point sampling. This serves as an initialization prior, not a pseudo-mask.

3. Contrastive Dual-GP Evidence

CDGP models normal ( $f_N$ ) and anomaly ( $f_A$ ) evidence as two sparse GP functions over the shared token space using a linear kernel ( $k(t, t') = t^\top t'$ ).

Predictive Distributions: For each token, the model learns a predictive mean ( $\mu$ ) and variance ( $\sigma^2$ ) for both classes.
Posterior Dominance Statistic: The core metric is the standardized difference between the anomaly and normal predictive distributions:
$g(t) = \frac{\mu_A(t) - \mu_N(t)}{\sqrt{\sigma^2_A(t) + \sigma^2_N(t)}}$
This statistic provides a spatial semantic map where a token is anomalous if the anomaly function is likely to exceed the normal function, standardized by joint uncertainty.

4. Weakly Supervised Training

The Dual-GP core is trained end-to-end using image-level labels via three complementary losses:

Max-Margin MIL: Ensures defective images have at least one token with a high positive margin ( $\mu_A - \mu_N$ ) and normal images have no positive margins.
Covariance-Normalized Normal Compactness: Encourages high normalized evidence for the normal GP over nominal images.
Covariance-Normalized Anomaly Attraction: Reinforces normalized anomaly evidence at locations identified as anomalous by the current margin score (using a soft attention mechanism).

5. Hierarchical Normal-Residual Evidence

To address the limitation that token-level GP evidence may under-resolve thin defects, CDGP incorporates a normal-only reverse student branch. A frozen teacher network processes clean normal images, while a student reconstructs the teacher's hierarchy from corrupted inputs. The cosine reconstruction residuals provide fine-scale spatial precision complementary to the semantic GP evidence.

6. Calibration and Inference

Fusion: The GP dominance map and the reconstruction residual map are standardized using statistics from a held-out normal calibration subset and fused with a weighted sum.
Image-Level Confidence: The fused spatial evidence is pooled (log-sum-exp) to generate an image-level anomaly score, which is calibrated using a logistic regressor trained on image-level labels.
Final Output: The final anomaly map combines the calibrated spatial evidence with the image-level confidence score, ensuring that images judged as likely normal suppress all pixel scores.

Key Contributions

Formulation: The paper formulates weakly supervised anomaly segmentation as a posterior dominance problem between sparse normal and anomaly predictive functions over dense tokens.
Training Strategy: It introduces a mask-free, defect-enriched initialization and a minimal three-loss objective (Max-margin MIL, covariance-normalized normal compactness, and covariance-normalized anomaly attraction) to train the Dual-GP core.
Hybrid Architecture: It complements semantic GP contrast with hierarchical normal-reconstruction residuals and training-only calibration to achieve spatially precise predictions without test-time fitting.
Performance: Under a matched weak-supervision split, CDGP achieves state-of-the-art results on MVTec AD 2 and competitive results on KSDD2 and VisA.

Experimental Results

The authors evaluated CDGP on MVTec AD 2, KSDD2, and VisA.

MVTec AD 2: CDGP ranked first among all evaluated methods (both normal-only and image-level supervised) on all three localization metrics (AUROC-P, AUPRO@0.3, AUPRO@0.05), outperforming the second-best method by significant margins (e.g., +12.7 points in AUPRO@0.3).
KSDD2 and VisA: CDGP achieved first-place or competitive localization performance, ranking among the top two methods across all metrics.
Ablation Studies: Factorized ablations confirmed that both the spatial GP path and the image-level calibration path contribute complementarily. The paper notes that the gain from posterior dominance (standardizing by variance) over simple mean-margin scoring is a modest refinement, suggesting the primary value lies in the coherent inducing-variable mean-variance construction rather than non-linear expressivity.

Significance and Claims

The paper claims that CDGP successfully leverages image-level labels to learn anomaly segmentation without requiring pixel masks or test-time fitting. Its significance lies in:

Addressing the "Unusual-but-Normal" Problem: By explicitly modeling both normal and anomaly distributions, it avoids the ambiguity of one-sided normality scoring.
Efficient Weak Supervision: It utilizes defective images (often discarded in normal-only methods) to learn anomaly characteristics, even without knowing exact defect locations.
Robustness: The use of training-only calibration and the combination of semantic (GP) and structural (reconstruction) evidence allows for robust localization across diverse industrial datasets.
Modesty on GP Claims: The authors explicitly state that the linear kernel makes the training margin linear; the advantage comes from the token-dependent predictive variance (uncertainty) provided by the GP formulation, rather than universal superiority over equivalent reparameterizations like engineered heteroscedastic linear models. The "GP claim" is framed as a compact construction of mean and uncertainty, bounded by matched linear-head controls.