=== SUMMARY ===

Technical Summary: CAFM – A Cross-Modal Local Alignment Fusion Method for RGB-3D Industrial Anomaly Detection

1. Problem Statement

Industrial anomaly detection faces a fundamental limitation when relying on single-modality data. Single RGB images provide rich texture and semantic cues but struggle to characterize geometric structural anomalies (e.g., height variations, depressions, bulges) due to susceptibility to illumination and viewpoint changes. Conversely, 3D point clouds effectively capture spatial morphology and geometric structures but lack the color and texture details necessary to identify subtle appearance-based defects.

While multimodal fusion (combining RGB and 3D) is a promising direction, existing approaches suffer from two critical flaws:

1. Misalignment and Irrelevance: Most methods use global feature concatenation or simple interpolation. This fails to account for the inherent mismatch in spatial scales and representation forms between 2D image patches and 3D spatial tokens, often introducing spatially irrelevant information that disrupts local anomaly detection.
2. Anomaly Compensation: In anomaly detection, the goal is to maximize the discrepancy between normal and defective samples. However, standard fusion modules with high representational capacity may "compensate" for anomalous regions in one modality using intact information from the other. This pulls defective features closer to the normal distribution, weakening the model's ability to distinguish defects.

2. Methodology: CAFM

The proposed Cross-Modal Local Alignment Fusion (CAFM) method addresses these issues within an unsupervised memory-bank paradigm. The framework consists of four primary stages:

A. Bottleneck Compression for RGB Feature Enhancement

To prevent the high-representation-capacity RGB stream from compensating for 3D anomalies, the authors introduce a bottleneck compression constraint specifically on the RGB branch.

• Mechanism: The RGB features undergo random dropout perturbation, followed by encoding into a low-dimensional bottleneck space ($Z_r$) and decoding back to the original space ($\hat{F}_r$).
• Residual Injection: To avoid excessive information loss, a controlled residual injection strategy is applied: $F^b_r = (1-\beta)\hat{F}_r + \beta F_r$.
• Effect: This forces the model to learn stable structural patterns prevalent in normal samples. During inference, anomalous perturbations cannot be fully reconstructed within the compact latent space, creating a prominent embedding shift that enhances anomaly separability. The 3D branch remains uncompressed to preserve fine-grained geometric cues.

B. Cross-Modal Local Alignment Fusion Module

Instead of global interaction, this module establishes fine-grained spatial correspondences within local windows.

• Scale Mapping: RGB and 3D features are projected into a unified embedding dimension. A scale mapping function aligns 3D spatial coordinates $(i, j)$ to corresponding central positions $(i', j')$ on the RGB feature map.
• Local Window Attention: For each 3D token, a local candidate region $\Omega_t$ is defined on the RGB map. Cross-modal attention is computed only within this window, where 3D features act as queries and local RGB features act as keys/values.
• Benefit: This restricts information propagation to spatially relevant neighbors, reducing computational complexity from $O(N_x N_r)$ to $O(N_x K)$ and preventing the dilution of local anomaly signals by distant normal regions.

C. Symmetric Contrastive Learning

To ensure fused features retain discriminative information from both modalities without bias, symmetric contrastive learning is employed.

• Bidirectional Constraints: The fused features are aligned with both the 3D features and the RGB features. The loss function includes bidirectional alignment terms ($L_{q \to k}$ and $L_{k \to q}$) for both modalities.
• Symmetry: This prevents the fused embedding from drifting toward the modality with richer information or easier optimization, ensuring a balanced representation of both texture and geometry.

D. Multi-Memory Bank Detection

Following the M3DM paradigm, the system constructs three independent memory banks using only normal training samples:

1. RGB feature memory bank.
2. 3D feature memory bank.
3. Fused cross-modal feature memory bank.
   Anomaly scores are generated via nearest-neighbor distance metrics for each bank, followed by late-stage decision fusion to produce final image-level and pixel-level anomaly maps.

3. Key Contributions

1. Cross-Modal Local Alignment: A fusion module that establishes spatial correspondences within local windows, eliminating irrelevant global interactions and enhancing the representation of local anomalous regions.
2. Bottleneck Compression Constraint: A tailored mechanism that limits the representational capacity of the RGB stream to prevent the "compensation" of anomalous regions by normal modal information, thereby preserving feature discrepancy.
3. Symmetric Contrastive Learning: A regularization strategy that enforces consistency between fused embeddings and both single-modal embeddings, ensuring balanced retention of RGB texture and 3D geometric information.

4. Experimental Results

The method was evaluated on the MVTec 3D-AD and Eyecandies datasets.

• MVTec 3D-AD:
  • Image-Level (I-AUROC): CAFM achieved 0.981, a 3.6 percentage point improvement over the baseline M3DM (0.945) and outperforming all other competitors (e.g., G2SF, CFM, AST).
  • Pixel-Level Localization (P-AUPRO): Achieved 0.977, surpassing M3DM by 1.3 points.
  • Pixel-Level Discrimination (P-AUROC): Achieved 0.998, ranking first among all baselines.
• Eyecandies:
  • Achieved an average I-AUROC of 0.901, P-AUPRO of 0.897, and P-AUROC of 0.981, consistently outperforming M3DM and other state-of-the-art methods.
• Ablation Studies:
  • Removing the bottleneck compression or the local alignment module resulted in significant performance drops, confirming that both components are essential for the observed gains.
  • The local alignment fusion contributed approximately 63.9% of the total I-AUROC improvement over the baseline.

5. Significance and Claims

The paper claims that CAFM effectively addresses the specific challenges of multimodal industrial anomaly detection, particularly the risk of anomaly compensation and the need for fine-grained local alignment. By integrating bottleneck compression, local window attention, and symmetric contrastive learning, the method creates a fused feature space where normal samples form compact manifolds while defective samples maintain distinct offsets.

The authors assert that their approach validates the effectiveness of local rather than global cross-modal interaction for industrial defects, which are typically local in nature. While the method achieves state-of-the-art results, the authors modestly acknowledge limitations: the fixed local window size and unified fusion strategy may struggle with tiny defects, blurred boundaries, or cases with drastic cross-modal response discrepancies. They suggest future work should explore multi-scale or deformable alignment mechanisms to further improve adaptability.