💻 computer science

Multimodal Industrial Anomaly Detection via Geometric Prior

This paper introduces GPAD, a novel multimodal industrial anomaly detection network that leverages a point cloud expert model to extract geometric priors via differential normal vectors and employs a two-stage fusion strategy to significantly outperform state-of-the-art methods in detecting complex 3D surface defects.

Original authors: Min Li, Jinghui He, Gang Li, Jiachen Li, Jin Wan, Delong Han

Published 2026-03-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Min Li, Jinghui He, Gang Li, Jiachen Li, Jin Wan, Delong Han

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a quality control inspector at a factory. Your job is to spot defects on products coming down the assembly line. Some defects are easy to see, like a scratch on a red car. But others are tricky: a tiny dent in a curved surface, a subtle warping of a shape, or a texture change that looks normal from the front but is wrong from the side.

Traditional inspection cameras (2D methods) are like looking at a flat photograph. They are great at seeing colors and patterns, but they struggle to understand depth and shape. If a surface is slightly warped, a flat photo might miss it entirely.

This paper introduces GPAD, a new "super-inspector" that doesn't just look at photos; it also understands the 3D shape of the object. Here is how it works, explained simply:

1. The Problem: The "Flat Photo" Blind Spot

Current systems try to combine a standard camera photo (RGB) with a depth map (a picture showing how far away things are). However, they often make a mistake: they let the colorful photo "talk too loud."

The Analogy: Imagine trying to listen to a quiet violin (the 3D shape) while a rock band (the colorful photo) is playing right next to it. The rock band drowns out the violin. The system sees the color perfectly but misses the subtle shape defects because the "color" data overpowers the "shape" data.

2. The Solution: The "Geometric Expert"

The authors created a new system called GPAD (Geometric Prior-based Anomaly Detection). Think of it as hiring a specialist to help the main inspector.

The Point Cloud Expert: Before looking at the final image, GPAD sends the 3D shape data (a cloud of dots representing the object) to a specialized "Geometric Expert" model.
The "Normal Vector" Trick: This expert doesn't just look at where the dots are; it calculates the direction every single dot is facing (like tiny arrows sticking out of the surface).
- Analogy: If you run your hand over a smooth table, your hand glides smoothly. If there is a tiny bump, your hand feels a sudden change in direction. The "Normal Vector" is like a super-sensitive hand that feels these tiny directional changes instantly. This creates a "Geometric Prior"—a cheat sheet of the object's true shape.

3. The Fusion: The "Smart Team Meeting"

Now, GPAD brings the "Color Inspector" (the camera) and the "Shape Expert" (the 3D model) together. But instead of just smashing their data together, they use a Two-Stage Strategy:

Stage 1 (The Setup): They mix the photo and the depth map early on, but they make sure the depth map gets a voice so the color doesn't dominate.
Stage 2 (The Smart Merge): This is the magic part. They use a mechanism called Geometry-Conditioned Attention.
- The Analogy: Imagine the Color Inspector and the Shape Expert are looking at a specific spot on a product.
  - If the spot is a smooth, flat area, the Color Inspector takes the lead (looking for scratches).
  - If the spot is a complex curve or a corner, the Shape Expert takes the lead (looking for dents or warps).
- The "Normal Vectors" act as a traffic cop, telling the system: "Hey, this area is tricky! Pay more attention to the 3D shape here!" This ensures the system focuses on the right type of data for the right part of the object.

4. The Result: Spotting the Invisible

Once the data is merged, the system tries to "reconstruct" what the object should look like if it were perfect.

If the real object matches the perfect version, it's good.
If there is a mismatch (a dent, a hole, a weird curve), the system flags it as an Anomaly.

Because GPAD uses the "Geometric Prior" to guide its attention, it is much better at finding defects that are hidden in complex shapes, like a dent in a tire or a warp in a cable gland.

Why Does This Matter?

In the real world, factories make thousands of products an hour. Missing a tiny defect can lead to broken machines or unsafe products.

Old Way: Misses subtle 3D defects because it relies too much on 2D photos.
GPAD Way: Uses a "Geometric Expert" to feel the shape, then uses that knowledge to decide where to look closely.

The Bottom Line:
GPAD is like giving a factory inspector a pair of 3D glasses and a super-sensitive touch, allowing them to see and feel defects that were previously invisible to standard cameras. It's faster, smarter, and catches more errors than the current best methods.

1. Problem Statement

Industrial anomaly detection (IAD) is critical for quality control but faces significant challenges when dealing with complex geometric defects (e.g., subtle surface deformations, irregular contours) that are difficult to detect using 2D-based methods alone.

Limitations of Existing Multimodal Methods: Current Multimodal Industrial Anomaly Detection (MIAD) methods often suffer from:
- Modality Bias: Early fusion methods tend to let high-dimensional RGB color information dominate, suppressing low-bit depth/3D geometric data during backpropagation.
- Geometric Distortion: Intermediate fusion methods often lack explicit geometric constraints, leading to inconsistent representation spaces between RGB and point cloud modalities. This causes geometric information loss during projection and asymmetric feature propagation.
- Lack of Fine-Grained Geometry: Existing methods fail to effectively utilize crucial geometric information like surface normal vectors and 3D topology, resulting in low accuracy for subtle structural defects.

2. Methodology: GPAD Framework

The authors propose GPAD (Geometric Prior-based Anomaly Detection), a hierarchical fusion framework that integrates geometric priors to guide multimodal feature fusion and segmentation. The architecture consists of three main stages:

A. Multimodal Feature Extraction

RGB-D Stream: Uses a pre-trained Depth-Aware Discrete Autoencoder (DADA) to extract low-level texture features ( $F_T$ ) and high-level semantic features ( $F_S$ ) from concatenated RGB and depth images.
Point Cloud Stream (Geometric Prior Extraction):
- Converts depth images to point clouds.
- Employs a Point Cloud Expert Model (PCExpert), a specialized 1D convolutional network, to process point groups.
- Key Innovation: Computes surface normal vectors via differential coordinate analysis (covariance analysis of local neighborhoods). These normals, combined with coordinates, form an enriched input ( $P_c$ ) that captures micrometer-level shape variations.
- The PCExpert extracts fine-grained geometric features ( $F_p$ ), serving as the Geometric Prior.

B. Geometry-Conditioned Attention Fusion

To address modality misalignment, GPAD introduces a two-stage fusion strategy:

Projection: The geometric prior ( $F_p$ ) is projected into the 2D feature space of the RGB-D stream using an affine transformation.
Geometry-Conditioned Attention:
- A cross-modal attention mechanism dynamically calculates attention weights based on the relevance of point cloud groups to specific image regions.
- Normal Vector Constraint: The normal vectors act as positional constraints during feature interaction.
- Adaptive Fusion: Texture and geometric features are fused in proportions related to local shape complexity. This ensures that geometric details guide the integration of RGB features, preventing RGB dominance and preserving 3D structural integrity.

C. Anomaly Segmentation (PCUnet)

Reconstruction: The fused features are passed through a Subspace Restriction Network (SRN) and a dual-decoder structure to reconstruct normal images (removing anomalies) and anomalous images (retaining defects).
Segmentation Network: A specialized PCUnet is used for segmentation. Unlike standard U-Nets, PCUnet injects the geometric prior ( $F_p$ ) at multiple levels of the decoding stage.
Mechanism: This hierarchical injection enhances the network's ability to perceive geometric boundaries and localize defects precisely, even on complex surfaces.

D. Loss Function

The model is trained using a composite loss function balancing:

Feature Reconstruction Loss (MSE between original and reconstructed features).
Image Reconstruction Loss (Pixel-level similarity).
Segmentation Loss (Focal Loss to handle class imbalance and boundary precision).

3. Key Contributions

GPAD Framework: A novel multimodal detection method that explicitly leverages geometric priors to overcome modality bias and feature distortion.
Point Cloud Expert Model: A pre-trained model designed for fine-grained geometric feature extraction using differential normal vectors to capture subtle surface variations.
Two-Stage Fusion Strategy:
- Stage 1: Early fusion of RGB-D to establish a coarse geometric basis.
- Stage 2: Integration of fine-grained geometric priors via Geometry-Conditioned Attention, ensuring texture and geometry are fused based on local shape complexity.
Geometric-Guided Segmentation: A segmentation module (PCUnet) that injects geometric priors during decoding to improve the localization of geometric defects.

4. Experimental Results

The method was evaluated on two datasets: MVTec-3D AD and Eyecandies.

Performance on MVTec-3D AD:
- I-AUROC (Image-level): GPAD achieved 98.9%, outperforming State-of-the-Art (SOTA) methods like 2M3DF (96.6%) and M3DM (94.5%).
- P-AUROC (Pixel-level): Achieved 99.6%, matching the best-performing method (Shape-Guided).
- AUPRO: Achieved 97.7%, significantly higher than competitors.
- Qualitative Results: Visualizations show GPAD produces more accurate segmentation maps with fewer false positives, particularly on complex objects like "Cable Gland" and "Tire," where other methods suffer from over-segmentation.
Performance on Eyecandies:
- GPAD achieved a mean I-AUROC of 91.2%, outperforming the current best method (2M3DF) by 0.3% and showing superior robustness across various candy types.
Ablation Studies:
- Removing Normal Vectors caused a significant drop in accuracy, proving their necessity for capturing surface details.
- Removing Geometry-Conditioned Attention led to suboptimal feature alignment.
- Removing Geometric Prior from the segmentation module reduced localization precision.
- Group Quantity Analysis: Setting the point cloud groups to 200 provided the optimal balance between feature granularity and computational efficiency.
Efficiency:
- GPAD achieves an inference speed of 25.8 FPS with a memory consumption of 708.5 MB. While slightly slower than 2M3DF (29.8 FPS), it offers a 2.1x improvement in false detection rate, offering a better trade-off for industrial applications.

5. Significance

Solving the "Geometric Blindness": GPAD addresses a critical gap in industrial AI by ensuring that 3D geometric information (normals, topology) is not lost or suppressed by 2D color data.
Robustness to Complex Defects: The method is particularly effective for detecting subtle deformations and irregular contours that are invisible to purely 2D or standard multimodal approaches.
Practical Applicability: By balancing high detection accuracy with real-time inference capabilities (25+ FPS), GPAD is well-suited for deployment in actual industrial production lines where both speed and precision are paramount.
Future Direction: The authors acknowledge the need for further optimization (e.g., knowledge distillation, model pruning) to create lightweight student networks for even faster real-time deployment.