Asymmetry-Aware Routing for Industrial Multimodal Monitoring: A Diagnostic Framework

1. Problem Statement

In industrial monitoring, multimodal fusion (combining heterogeneous sensors like thermal cameras, vibration, and electrical signals) is the standard approach. The prevailing assumption is that fusing more modalities always improves detection performance. However, real-world industrial data often exhibits informativeness asymmetry, where one modality carries significantly more discriminative information than the other.

Current fusion architectures and training-time mitigation strategies (e.g., gradient modulation, attention mechanisms) assume both modalities are useful and attempt to balance them. They fail to diagnose when fusion is actually detrimental. Deploying a fusion model in an asymmetric scenario leads to:

• Degraded detection accuracy (the model learns to rely on noise or weak features).
• Wasted computational resources.
• Reduced trust in safety-critical systems.

There is currently no systematic, pre-deployment diagnostic procedure to determine whether a specific sensor combination should be fused, processed separately (cascaded), or routed selectively.

2. Methodology: Asymmetry-Aware Routing Framework

The authors propose a three-step diagnostic framework designed to be run before committing to a multimodal architecture. The framework outputs one of three routing decisions: CASCADE (process separately), FUSE (standard fusion), or SELECTIVE (adaptive per-sample routing).

The three diagnostic steps are:

Step 1: Unimodal Performance Gap Analysis

• Procedure: Train and evaluate each modality independently (zeroing out the other).
• Metric: Calculate the Gap Ratio ($r$) between the macro F1 scores of the stronger and weaker modalities:
  $$r = \frac{\max(F_{strong}, F_{weak})}{\min(F_{strong}, F_{weak}) + \epsilon}$$
• Logic: A high ratio (e.g., $>2.0$) indicates severe asymmetry where the weaker modality contributes little to no useful signal.

Step 2: Gate Weight Attribution

• Procedure: Train a fusion model with an explicit gating mechanism (e.g., Gated Fusion, Selective Routing) and analyze the learned weights on the test set.
• Metric: Compute the Bias Magnitude ($B$), defined as the absolute deviation of the expected gate weight for the weaker modality from 0.5 ($B = |E[g_{weak}] - 0.5|$).
• Logic: If the model systematically over-weights the weak modality (high bias) despite its poor performance, it indicates the fusion mechanism is failing to learn the true informativeness hierarchy.

Step 3: Modality Corruption Testing

• Procedure: Systematically "corrupt" (zero out) one modality while keeping the other intact in a fused model and measure the change in F1 score ($\Delta F$).
• Metric: $\Delta F_{weak} = F_{corrupted\_weak} - F_{baseline}$.
• Logic: If zeroing out the weak modality improves performance ($\Delta F > 0$), it causally confirms that the modality is actively harming the fusion process.

Decision Criteria

The framework combines these steps using thresholds ($\tau_c, \tau_f, \beta$):

• CASCADE: If Gap Ratio $> \tau_c$ AND zeroing the weak modality improves performance. (The weak modality is harmful; process separately).
• FUSE: If Gap Ratio $< \tau_f$ AND Gate Bias $< \beta$. (Both modalities are complementary; standard fusion is optimal).
• SELECTIVE: Otherwise. (Ambiguous cases where unimodal performance is balanced, but the fusion model fails to utilize them effectively; use adaptive per-sample gating).

3. Key Contributions

1. Diagnostic Framework: A formal, three-step procedure with validated decision boundaries to route multimodal systems toward the correct strategy (Fuse vs. Cascade vs. Selective) prior to deployment.
2. Multi-Dataset Validation: The framework was validated on three distinct datasets covering different asymmetry regimes:
   • OHT/AGV (Industrial): Thermal + Electrical sensors.
   • SSCC (Chain Conveyor): Audio + Vibration.
   • CWRU (Bearing): Controlled noise vs. signal features.
3. Robustness Analysis: Demonstrated that routing decisions remain stable across a wide range of threshold perturbations.
4. Protocol Necessity: Proved that relying solely on the unimodal gap (Step 1) is insufficient for moderate-asymmetry cases, necessitating the full protocol.

4. Experimental Results

Case A: OHT/AGV Dataset (CASCADE Outcome)

• Scenario: 8 electrical sensor channels vs. Thermal images.
• Findings:
  • Unimodal Gap: Sensors achieved 93.2% F1; Thermal images achieved only ~30% F1 (Gap Ratio $\approx 3.1$).
  • Fusion Impact: Every fusion architecture (including advanced methods like MulT, OGM-GE, and MLA) performed worse than the sensor-only Random Forest baseline. The best fusion method (Selective Routing) scored 91.9%, still below the 93.2% baseline.
  • Diagnosis: The framework correctly identified severe asymmetry and recommended CASCADE. Zeroing the thermal input improved fusion performance, confirming thermal data was noise in this context.
  • Operational Benefit: The sensor-only Random Forest required no GPU and ran in single-digit milliseconds, offering superior speed and accuracy over complex fusion models.

Case B: SSCC Dataset (FUSE Outcome)

• Scenario: Audio (microphones) vs. Vibration (accelerometers).
• Findings:
  • Unimodal Gap: Moderate asymmetry (Gap Ratio $\approx 1.06$). Both modalities carried genuine diagnostic information.
  • Fusion Impact: All fusion methods outperformed the best unimodal baseline (Vibration-only at 98.3%). The best fusion method (Selective) reached 99.9% F1.
  • Diagnosis: The framework correctly identified balanced contributions and recommended FUSE.

Case C: CWRU Dataset (Controlled Validation)

• Findings:
  • High-Asymmetry (Real vs. Noise): Correctly recommended CASCADE.
  • Low-Asymmetry (Time vs. Frequency features): Gap ratio was $\approx 1.0$ (suggesting FUSE), but Step 2 revealed high gate bias ($B > \beta$). The framework correctly recommended SELECTIVE, preventing the deployment of a suboptimal standard fusion model.

Comparison Against Simpler Diagnostics

• Using Step 1 only (Gap Ratio) yielded 75% accuracy. It failed on the CWRU-Low-Asymmetry case, incorrectly recommending FUSE because it missed the learned gate bias.
• The Full 3-step protocol achieved 100% accuracy across all test cases.

5. Significance and Implications

• Paradigm Shift: Challenges the "more sensors = better" assumption in industrial AI. It establishes that diagnosis must precede fusion.
• Cost Efficiency: In asymmetric cases (like OHT/AGV), the framework prevents the deployment of expensive, GPU-requiring fusion models in favor of lightweight, interpretable unimodal models (e.g., Random Forest), saving compute and reducing latency.
• Practical Workflow: Provides a concrete pre-deployment checklist for engineers:
  1. Evaluate unimodal ceilings.
  2. Inspect gate weights for bias.
  3. Run corruption tests.
• Tool Availability: The authors release the framework as a standalone Python module, making it immediately applicable to industrial practitioners.

In conclusion, the paper demonstrates that fusion is not universally beneficial. By implementing this diagnostic framework, industries can avoid the "negative transfer" of fusing uninformative modalities, ensuring that multimodal systems are only deployed when they genuinely add value.