FDCNet: Frequency-Aware and Dynamic Curve Network for Adversarially Robust Infrared and Visible Image Fusion

Technical Summary: FDCNet for Adversarially Robust Infrared and Visible Image Fusion

Problem Statement
Infrared and Visible Image Fusion (IVIF) integrates complementary multi-modal information to overcome the limitations of single sensors, serving critical roles in autonomous driving, object tracking, and salient object detection. However, existing IVIF methods predominantly assume interference-free inputs, rendering them fragile against deliberate adversarial attacks. Classical adversarial algorithms (e.g., FGSM, PGD, C&W) can induce severe degradation in model inference by injecting imperceptible perturbations. While adversarial training has been explored to enhance robustness, it often creates an intrinsic trade-off: strategies designed to suppress high-frequency adversarial noise frequently result in over-conservative feature inhibition, leading to structural over-smoothing, loss of fine-grained details, and a contraction of the dynamic range. This degradation fundamentally restricts the performance stability of downstream vision tasks under complex conditions.

Methodology
To address the contradiction between adversarial defense and feature fidelity, the authors propose FDCNet (Frequency-Aware and Dynamic Curve Network). The framework operates as an end-to-end adversarial training system utilizing a U-Net backbone, integrating three core components to synergistically balance defense efficacy and visual quality:

1. Spatio-Frequency Dynamic Adversarial Loss (LSFDA):
   Proposed at the optimization level, this loss function guides robust training by adaptively balancing structural fidelity and high-frequency feature constraints. It employs a min-max optimization strategy where the inner maximization generates adversarial perturbations (via PGD) against pseudo-labels, and the outer minimization updates network parameters. The loss comprises a baseline structural spatial fidelity term and a normalized frequency-aware tolerance term. The latter utilizes Discrete Wavelet Transform (DWT) to decouple low-frequency and high-frequency subbands, imposing a "soft upper-bound" constraint on high-frequency components to prevent detail loss while penalizing excessive noise. A multi-level dynamic weighting system adjusts the focus between structural reconstruction and anti-interference capabilities throughout the training process.

2. Frequency-Aware Defense Module (FADM):
   Embedded within the multi-scale skip connections of the U-Net backbone, this module acts as the front-end anti-interference architecture. It intercepts cross-layer noise penetration through a cascaded series of five Frequency Decoupling Attention Blocks (FDAB). Each FDAB unit performs:

   • Spatial Smoothing and Wavelet Decoupling: Uses depthwise separable convolutions and DWT to separate features into low-frequency (LL) and high-frequency (LH, HL, HH) subbands.
   • Affine Invertible Frequency Generation (AIFG): Utilizes Affine Coupling Blocks (ACB) to extract features while preserving high-frequency details through strict mathematical invertibility, preventing information loss during forward propagation.
   • Invertible Directional Dual Attention (IDDA): Combines a Mercer Global Attention (MGA) branch for macro-feature smoothing (using kernel mapping to reduce complexity) and a Local Window Attention (LWA) branch for handling local high-frequency anomalies.
   • Residual Reconstruction: Reassembles purified subbands via pixel-shuffle and residual connections to output robust features.

3. Extrema-guided Dynamic Tone Curve Module (EDTC):
   Designed as a back-end post-compensation mechanism, the EDTC addresses the mutual constraints between the LSFDA and FADM. Inspired by Retinex theory and curve estimation strategies, it couples the preliminary robust features with multi-source extreme value distributions (the pixel-wise maximum envelope of the source images) as spatial priors. The module predicts pixel-wise, high-order non-linear mapping parameters via a lightweight network and executes iterative quadratic curve reconstruction. This process effectively restores the pixel distribution, alleviating dynamic range contraction and recovering visual sharpness without compromising the underlying defense system.

Key Contributions

• LSFDA: A novel loss function that dynamically adjusts weights between structural fidelity and high-frequency constraints, mitigating the detail loss typically associated with conventional adversarial defenses.
• FADM: A front-end architecture that cascades Frequency Decoupling Attention Blocks within skip connections to intercept and filter high-frequency adversarial noise while preserving low-level spatial details through invertible transformations.
• EDTC: A back-end compensation mechanism that utilizes multi-source extreme priors to perform non-linear pixel-wise mapping, effectively resolving the performance trade-offs between the defense modules and restoring image visual quality.
• Unified Framework: The integration of these components into FDCNet achieves a synergistic balance between adversarial robustness and feature fidelity, outperforming state-of-the-art (SOTA) methods in both objective metrics and downstream task stability.

Experimental Results
Extensive experiments were conducted on the M³FD and MFNet datasets, comparing FDCNet against eight SOTA fusion models (including SeAFusion, TarDAL, PAIF, and A²RNet).

• Fusion Quality: Under adversarial attacks (PGD), FDCNet consistently outperformed baselines across five objective metrics: Entropy (EN), Standard Deviation (SD), Peak Signal-to-Noise Ratio (PSNR), Correlation Coefficient (CC), and Sum of Structured Cross Difference (SCD). Qualitative analysis showed that FDCNet preserved infrared thermal saliency and visible texture details with minimal signal waveform deviation between clean and attacked states, whereas other methods exhibited severe noise, distortion, or structural collapse.
• Downstream Task Stability: When fused images were fed into pre-trained object detection and semantic segmentation networks, FDCNet demonstrated superior robustness. On the M³FD and MFNet datasets, FDCNet achieved the highest Mean Average Precision (mAP) and Mean Intersection over Union (mIoU) under attack conditions, with the smallest performance degradation (e.g., mAP drop of only 0.012 on M³FD) compared to other methods. Visual detection results confirmed that FDCNet successfully detected all targets (pedestrians and vehicles) with high confidence, while other models suffered from missed detections and fragmented segmentation.
• Ablation Studies: Experiments verified that while individual modules (LSFDA, FADM, EDTC) improved performance, their joint operation without the EDTC led to performance constraints due to conflicting optimization objectives. The full model (M6) incorporating all three components achieved optimal results, confirming the necessity of the EDTC for reconciling defense and fidelity.

Significance
The paper claims that FDCNet effectively mitigates the adverse effects of adversarial perturbations on IVIF tasks, a domain previously vulnerable to such attacks. By introducing a frequency-aware defense strategy coupled with a dynamic tone curve compensation mechanism, the method successfully overcomes the traditional trade-off between anti-interference immunity and image fidelity. The results demonstrate that FDCNet not only maintains high-fidelity fusion results but also significantly stabilizes the performance of critical downstream vision tasks (object detection and semantic segmentation) in adversarial environments. The authors acknowledge that the multi-scale frequency decoupling and non-linear iterations increase computational complexity, suggesting future work will focus on lightweight design and inference efficiency optimization for practical engineering deployment.