💻 computer science

LADMIM: Logical Anomaly Detection with Masked Image Modeling in Discrete Latent Space

This paper introduces LADMIM, an unsupervised anomaly detection framework that leverages masked image modeling in a discrete latent space to effectively identify logical anomalies by learning long-range dependencies and focusing on global structural patterns rather than local pixel variations.

Original authors: Shunsuke Sakai, Tatushito Hasegawa, Makoto Koshino

Published 2026-03-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shunsuke Sakai, Tatushito Hasegawa, Makoto Koshino

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a quality control inspector at a factory. Your job is to look at products on a conveyor belt and spot anything that looks "wrong."

For a long time, inspectors (and the computer programs they used) were really good at spotting physical damage. If a bottle had a scratch, a dent, or a stain, the system would scream, "Defect!" This is called detecting structural anomalies.

But there's a harder problem: Logical anomalies.
Imagine a box of screws where one screw is missing, or a bottle of juice where the cap is on the bottom instead of the top. The bottle itself isn't scratched; the cap isn't broken. The relationship between the parts is wrong. Traditional systems often miss these because they are too focused on the tiny details of the surface, not the big picture of how things fit together.

This paper introduces a new system called LADMIM to solve this puzzle. Here is how it works, explained with simple analogies.

The Two-Tool Strategy

The authors realized that one tool isn't enough to catch both types of mistakes. So, they built a system with two distinct "inspectors" working together:

1. The "Detail Detective" (HVQ-Trans)

What it does: This part is great at finding scratches, dents, and stains.
How it works: Imagine you have a perfect photo of a normal product. You take a blurry, low-resolution copy of it and try to redraw it perfectly. If the original had a scratch, your blurry copy will try to "fix" it to look normal. When you compare your redrawn version to the real thing, the scratch stands out because the redraw didn't match.
The Analogy: It's like a child trying to trace a picture. If the original has a smudge, the child's tracing will look different, revealing the smudge.

2. The "Logic Puzzle Solver" (ViT with MIM)

What it does: This part is the star of the show for finding logical errors (like a missing screw or a swapped part).
The Problem: If you try to predict exactly what a missing part of an image looks like (pixel-by-pixel), the computer gets confused. It might guess the right color but in the wrong spot, or it might get distracted by tiny noise. It's like trying to finish a jigsaw puzzle while blindfolded; you might guess the shape, but you won't know exactly where it goes.
The Solution (The Magic Trick): Instead of asking the computer to guess the exact picture of the missing piece, they ask it to guess the list of ingredients in that missing piece.
- Analogy: Imagine you are covering up a part of a picture of a sandwich. Instead of asking, "Draw exactly what is under the cover," you ask, "What ingredients are in the covered area?"
- If the covered area is supposed to be just "bread and cheese," but the hidden part actually has "pickles" (an anomaly), the computer will say, "Wait, I predicted bread and cheese, but the reality has pickles!"
- Because the computer is only counting what is there (the ingredients) and not worrying about exactly where every crumb is, it becomes much better at spotting when the "recipe" is wrong.

How They Work Together

The system combines the scores from both inspectors:

The Detail Detective checks for scratches and dents.
The Logic Puzzle Solver checks if the parts are in the right order or if anything is missing.

If either one says, "Hey, something is weird," the system flags the product.

Why This is a Big Deal

No Pre-Training Needed: Many current systems need to be taught by humans using thousands of labeled examples or pre-trained on massive datasets. This system learns by itself (unsupervised) just by looking at "normal" products. It figures out the rules of "normal" on its own.
Better at the Hard Stuff: Previous computer systems were terrible at logical anomalies. This new method brings the accuracy up to the level of the best existing systems, but without needing complex, pre-trained tools.
Robustness: It doesn't get confused by small shifts in position. If a screw moves slightly to the left, the system knows it's still a screw, but if a screw is missing, it knows something is wrong.

The Catch (Limitations)

The system isn't perfect yet.

The "Blindfold" Issue: Because it guesses based on "ingredients" rather than exact positions, it sometimes struggles to tell you exactly where the error is located on the image. It knows the sandwich is wrong, but it might not point a finger at the specific slice of bread.
Masking Strategy: The system works by "hiding" parts of the image to test itself. If it hides too much, it gets confused. If it hides too little, it doesn't learn enough. Finding the perfect balance is still a work in progress.

Summary

In short, LADMIM is a new way for computers to inspect factory products. It uses a clever trick: instead of trying to redraw a missing piece of an image perfectly, it guesses the "recipe" of that missing piece. This allows it to spot not just broken things, but also things that are arranged incorrectly, making it a much smarter quality control inspector.

1. Problem Definition

The paper addresses a critical gap in Unsupervised Anomaly Detection (AD): the detection of logical anomalies.

Structural Anomalies: Local deviations such as scratches, stains, or cracks. These are well-handled by existing methods that focus on local feature patterns.
Logical Anomalies: Global deviations involving the relationships between objects, such as incorrect object combinations, misalignments, or missing/extra objects in multi-object scenes.
The Challenge: Conventional AD methods (reconstruction-based or memory-bank-based) often struggle with logical anomalies because they rely heavily on local features or require pre-trained segmentation models. Furthermore, standard Masked Image Modeling (MIM) approaches often fail due to positional uncertainty; predicting pixel-level features in masked regions leads to blurred reconstructions because the model cannot determine the exact position of features within the mask.

2. Methodology: LADMIM

The authors propose LADMIM, a novel framework that combines Hierarchical Vector Quantized Transformer (HVQ-Trans) and a Vision Transformer (ViT) trained with a Masked Image Modeling objective. The framework operates in two distinct stages to detect structural and logical anomalies separately.

A. Core Components

HVQ-Trans (Structural Anomaly Detector & Tokenizer):
- Based on a hierarchical vector quantization mechanism, it reconstructs image features from a backbone network (PDN).
- It detects structural anomalies via reconstruction error (Mean Squared Error).
- Crucially, it acts as a tokenizer, converting continuous image features into a hierarchical set of discrete latent variables (codes). These discrete codes serve as the prediction target for the logical anomaly detector.
ViT with MIM (Logical Anomaly Detector):
- A ViT is trained to predict the probability distribution (histogram) of the discrete latent variables in masked regions, rather than predicting specific pixel values or specific token positions.
- Input: The ViT receives visible patches and a learnable mask token.
- Target: Instead of predicting which code is at which position (which suffers from positional ambiguity), the model predicts the global composition (histogram) of codes present in the masked area.
- Loss Function: The model minimizes the L1 distance between the predicted code distribution ( $P$ ) and the ground truth code distribution ( $Q$ ) derived from the HVQ-Trans tokenizer.

B. Training and Inference

Training:
1. Train HVQ-Trans to reconstruct features and generate discrete latents.
2. Freeze HVQ-Trans weights. Train the ViT using the MIM objective (predicting code histograms from HVQ-Trans) on normal images only.
Inference:
- Structural Score ( $S_{HVQ}$ ): Calculated as the MSE between the original and reconstructed features.
- Logical Score ( $S_{MIM}$ ): Calculated as the average L1 distance between predicted and actual code histograms across multiple random masks.
- Final Score: The standardized scores are aggregated (using Z-score normalization to handle outliers) to determine the final anomaly status.

3. Key Contributions

Novel Framework (LADMIM): A unified approach that leverages MIM for logical anomalies and HVQ-Trans for structural anomalies without requiring pre-trained segmentation models.
Distribution-Based Prediction: The core insight is that predicting the probability distribution of discrete latents (histograms) rather than position-specific tokens mitigates positional uncertainty. This allows the model to focus on logical dependencies (e.g., "a bottle usually has a cap") rather than exact spatial arrangements.
Scalability and Robustness: Unlike methods relying on tight information bottlenecks (which require careful hyperparameter tuning), LADMIM is robust to model scaling and does not suffer from the "Identity Shortcut" problem common in reconstruction-based methods.
Comprehensive Analysis: The paper provides extensive ablation studies on masking strategies, prediction targets, and loss functions, revealing that L1 distance on code histograms outperforms KL/JS divergence and pixel-level predictions for logical AD.

4. Experimental Results

The method was evaluated on MVTec LOCO (focused on logical anomalies) and MVTec AD (focused on structural anomalies).

MVTec LOCO Performance:
- LADMIM achieved an average image-level AUROC of 84.7% for logical anomalies and 88.5% for structural anomalies.
- It significantly outperformed other MIM-based methods (e.g., SSM, SMAE) and competitive memory-bank methods (e.g., PatchCore, ComAD).
- While slightly behind the current State-of-the-Art (SoTA) methods like EfficientAD and PUAD in single-class settings, LADMIM showed superior scalability in multi-class settings. As model dimensions increased, LADMIM's performance improved, whereas EfficientAD's performance dropped due to information bottleneck limitations.
MVTec AD Performance:
- Achieved 93.4% AUROC, outperforming other MIM-based approaches, though trailing behind specialized SoTA methods focused solely on structural defects.
Ablation Insights:
- Prediction Target: Predicting the code histogram yielded the best results, balancing structural and logical detection. Predicting raw pixels favored structural detection but failed on logical anomalies.
- Masking Strategy: Adaptive masking worked best for structural anomalies, while block-random masking with moderate coverage (20-40%) was optimal for logical anomalies.

5. Significance and Future Directions

Paradigm Shift: LADMIM demonstrates that MIM can be effectively adapted for logical anomaly detection by shifting the prediction target from spatial tokens to distributional representations. This bypasses the need for complex segmentation priors.
Unified Framework: It offers a path toward a unified AD system capable of handling both local (structural) and global (logical) defects without task-specific architectural changes.
Limitations & Future Work:
- The current histogram-based approach loses order information, making it difficult to detect anomalies that rely on specific sequences or permutations.
- Localization of logical anomalies is currently coarse due to random masking.
- Future work suggests integrating stochastic prediction modules (e.g., diffusion models) and semantic masking to improve localization and order-sensitive detection.

In summary, LADMIM represents a significant step forward in unsupervised anomaly detection by effectively modeling long-range dependencies and logical relationships through discrete latent distribution prediction, offering a robust alternative to current segmentation-dependent or bottleneck-reliant methods.