Understanding and Overcoming Cross-modal Fusion Bias in Multimodal Anomaly Detection From A Fisher Information Perspective

Technical Summary: Understanding and Overcoming Cross-modal Fusion Bias in Multimodal Anomaly Detection From A Fisher Information Perspective

Problem Statement
Multimodal Anomaly Detection (MAD) aims to leverage complementary information from RGB and depth data to identify subtle defects in industrial products, outperforming unimodal approaches. However, current advancements largely focus on developing novel fusion strategies (e.g., early, hybrid, or architecture-oriented fusion) while neglecting a critical underlying issue: cross-modal fusion bias. This bias occurs when one modality suppresses the other during the learning process, leading to suboptimal performance. The paper posits that this bias, particularly the dominance of one modality over another in the early stages of training, creates a performance bottleneck that existing methods have failed to address.

Methodology: The UCFB Framework
To address this, the authors propose UCFB, a plug-and-play framework grounded in Fisher Information. The methodology consists of three core components:

1. Fisher Information Analysis for Bias Detection:
   The authors utilize the trace of the Fisher Information Matrix (FIM), denoted as $Tr(F_\kappa)$, to quantify the information acquisition of each modality ($\kappa \in \{RGB, Depth\}$). Empirical analysis reveals that in multimodal settings, a significant gap often emerges early in training where one modality acquires information much faster than the other, leading to the suppression of the slower modality.

2. Unimodal Adaptive Adjustment (UAA):
   This module dynamically calibrates modality-specific regularization weights to mitigate the dominance of the "fast" modality.

   • Identification: The system identifies the dominant modality by calculating the information acquisition gap ($\Delta_\kappa$) between modalities.
   • Critical Period Detection: It determines if the dominant modality is in a "critical learning epoch" (characterized by rapid information acquisition) using a threshold $\Upsilon$ on the rate of change of the FIM trace.
   • Adjustment: If a dominant modality is detected in the critical period, UAA applies a regularization term $\Gamma_{\kappa, b}$ based on the gradient norm. This term, derived from the gradient update rule and high-dimensional orthogonality assumptions (Lemma 1), slows down the information acquisition of the dominant modality without altering the convergence rate of the original objective (Lemma 2).

3. Canonical Similarity Analysis (CSA):
   While UAA slows the dominant modality, the CSA module actively enhances the information acquisition of the suppressed modality. It employs nonlinear transformation layers to extract modality-specific representations and maximizes the correlation between the RGB and Depth feature outputs. This encourages unimodal encoders to collaboratively capture shared, task-relevant information, thereby strengthening cross-modal interactions.

Key Contributions

• Theoretical Insight: This work is the first to systematically scrutinize the impact of cross-modal fusion bias in MAD from a Fisher Information perspective. It empirically demonstrates that bias arises primarily from early-stage modality suppression and that this suppression causes irreversible performance degradation.
• Novel Framework: The authors propose UCFB, a simple yet effective framework that integrates Unimodal Adaptive Adjustment and Canonical Similarity Analysis. It is designed as a modular component that can be seamlessly embedded into various existing MAD architectures.
• Design Principles: The study identifies that dynamically calibrating modality-specific weights during the critical early learning epoch is essential to prevent one modality from suppressing another.

Experimental Results
The authors evaluated UCFB on two benchmark datasets: MVTec 3D-AD (real-world industrial data) and Eyecandies (synthetic data). The framework was integrated into four state-of-the-art baselines: EasyNet, CFM, IUF, and 3D-ADNAS.

• Single-Class and Multi-Class Settings: UCFB consistently improved performance across all baselines. On MVTec 3D-AD, integrating UCFB with EasyNet yielded a maximum I-AUROC gain of 1.2% (93.8 vs 92.6) in the single-class setting, while CFM achieved a gain of 0.7% (96.1 vs 95.4). Similar improvements were observed in multi-class settings.
• Few-Shot Settings: In scenarios with limited training data (5, 10, and 50 shots), UCFB demonstrated significant robustness, improving I-AUROC scores by up to 1.2% compared to baselines.
• Ablation Studies: Experiments confirmed that both UAA and CSA are necessary; using them jointly outperformed using either module alone. Furthermore, the study validated that interfering with the learning process in the early stages causes permanent performance drops, reinforcing the necessity of early-stage calibration.

Significance and Claims
The paper claims that the suboptimal performance of current MAD methods is largely due to unaddressed cross-modal fusion bias. By shifting the focus from complex fusion architectures to the dynamics of information acquisition via Fisher Information, the authors demonstrate that a simple, plug-and-play intervention can break existing performance bottlenecks. The work suggests that mitigating bias through dynamic calibration and enhanced interaction is a fundamental requirement for advancing industrial anomaly detection, offering a generalizable solution that improves detection accuracy across single-class, multi-class, and few-shot scenarios without increasing computational complexity significantly.