Hyperbolic Cycle Alignment for Infrared-Visible Image Fusion

1. Problem Statement

Infrared-Visible Image Fusion aims to combine the thermal radiation sensitivity of infrared (IR) images with the high-resolution texture details of visible (VIS) images. However, the success of this fusion relies heavily on accurate pixel-level registration (alignment) between the two modalities.

• The Core Challenge: Existing registration methods typically operate in Euclidean space and rely on image translation. They fail to handle cross-modal misalignment effectively because:
  • IR and VIS images have fundamentally different imaging mechanisms (thermal radiation vs. light reflection), leading to nonlinear disparities in edge features and structures.
  • Real-world factors like viewpoint changes, motion blur, and sensor displacement exacerbate these nonlinear misalignments.
  • Traditional Euclidean methods often result in structural distortions, loss of semantic information, and suboptimal fusion quality (e.g., ghosting artifacts).
• The Gap: There is a lack of research applying non-Euclidean geometry (specifically hyperbolic space) to pixel-level multi-modal image registration, despite its proven ability to model hierarchical and tree-like structures efficiently.

2. Methodology: Hy-CycleAlign

The authors propose Hy-CycleAlign, the first image registration framework based on Hyperbolic Space. The method integrates a dual-path cyclic consistency framework with a novel contrastive alignment module.

A. Theoretical Motivation

The paper posits that Hyperbolic Space (specifically the Poincaré model) is superior to Euclidean space for this task because:

• Negative Curvature: It naturally models hierarchical structures and complex nonlinear relationships found in multi-modal data.
• Sensitivity to Misalignment: The authors provide a theoretical proof (Theorem 1) showing that the distance metric in Poincaré space is more sensitive to small displacements than Euclidean distance. As points approach the boundary of the Poincaré space, the gradient magnitude of the distance increases, making the network more responsive to correcting misalignments.

B. Network Architecture

The framework consists of three main components:

1. Dual-Path Cyclic Registration Framework:

   • Forward Path: Aligns the Infrared image ($T$) to the Visible image ($V$) to produce an aligned image ($T_v$).
   • Backward Path: Inversely aligns $T_v$ back to the original Infrared space to reconstruct the original image ($T_{vt}$).
   • Cycle Consistency: This closed-loop structure ensures geometric consistency without requiring perfectly aligned ground-truth pairs for training.
   • Fusion: The aligned image ($T_v$) and the original Visible image ($V$) are fused to generate the final output ($F$).

2. Hyperbolic Hierarchy Contrastive Alignment (H2CA) Module:

   • This is the core innovation that maps features from Euclidean space to Poincaré space.
   • It imposes constraints at two levels:
     • Pixel-level ($H2CA-p$): Maps deep features (extracted via VGG-16) into hyperbolic space to align global structures.
     • Edge-level ($H2CA-e$): Maps edge features (extracted via Sobel operator) into hyperbolic space to align structural boundaries.
   • It uses Möbius addition and hyperbolic geodesic distance to calculate contrastive losses, effectively reducing interference caused by modality discrepancies.

3. Adversarial Discriminators:

   • Two discriminators ($D_v$ and $D_t$) are employed to distinguish between the edges of the original images and the aligned images, ensuring the registered images preserve realistic edge structures.

C. Loss Function

The total loss ($L$) combines four components:

• Adversarial Loss ($L_{adv}$): Ensures edge realism.
• Cycle Consistency Loss ($L_{cc}$): Minimizes the difference between the reconstructed image and the original input.
• Hyperbolic Hierarchical Contrastive Loss ($L_{h2c}$): Enforces alignment in Poincaré space for both pixels and edges.
• Smoothness Loss ($L_{sm}$): Ensures the deformation field is smooth.
• Fusion Loss ($L_f$): Ensures the fused image retains maximum information from both sources.

3. Key Contributions

1. First Hyperbolic Registration Method: Hy-CycleAlign is the first work to apply hyperbolic space constraints to pixel-level multi-modal image registration.
2. Dual-Path Cyclic Framework: Introduces a closed-loop registration structure that eliminates the need for costly, manually aligned training data.
3. H2CA Module: A novel module that leverages the hierarchical properties of hyperbolic space to align both pixel features and edge structures, effectively handling nonlinear cross-modal discrepancies.
4. Theoretical Proof: Provides a mathematical demonstration that hyperbolic space offers higher sensitivity to misalignment errors compared to Euclidean space, justifying its use for complex registration tasks.

4. Experimental Results

The method was evaluated on three benchmark datasets: DroneVehicle, LLVIP, and MFNet. The datasets included both naturally misaligned and artificially generated nonlinear misalignments.

• Quantitative Performance:
  • Hy-CycleAlign outperformed State-of-the-Art (SOTA) methods (including DIDFuse, CDDFuse, SuperFusion, ReCoNet, MURF, etc.) across multiple metrics: Hausdorff Distance (HD/HD95), Average Symmetric Surface Distance (ASSD), Dice Similarity Coefficient (DSC), and Entropy (EN).
  • For example, on the DroneVehicle dataset, it achieved the lowest HD (70.36) and highest DSC (0.75) among alignment-based fusion methods.
• Qualitative Performance:
  • Visual results show superior alignment of thermal signatures with visible edges, eliminating ghosting artifacts common in other methods.
  • The method effectively preserves fine textures and structural details even under severe nonlinear deformations.
• Downstream Task (Object Detection):
  • When applied to object detection (using YOLOv11-m on DroneVehicle), the fused images generated by Hy-CycleAlign yielded the highest mAP@0.5 (13.7) and Precision (12.3), demonstrating that better registration directly improves downstream analytical tasks.
• Ablation Studies:
  • Removing the cyclic structure or the hyperbolic constraints significantly degraded performance, confirming the necessity of both the cycle consistency and the hyperbolic geometry.
  • Combining pixel-level and edge-level hyperbolic constraints yielded the best results.

5. Significance and Impact

• Paradigm Shift: This work challenges the dominance of Euclidean space in image registration, demonstrating that non-Euclidean (hyperbolic) geometry is better suited for the nonlinear, hierarchical nature of multi-modal data.
• Robustness: The method is highly robust to complex misalignments (rotation, scaling, nonlinear warping) that typically break traditional registration algorithms.
• Practical Application: By improving the alignment of IR and VIS images, Hy-CycleAlign enhances critical applications such as all-weather surveillance, search and rescue, and autonomous driving, where accurate perception is vital.
• Efficiency: Despite the added complexity of hyperbolic operations, the model maintains a computational footprint (FLOPs) comparable to other SOTA methods (e.g., 16.66 G FLOPs vs. 16.36 G for SuperFusion), making it viable for practical deployment.

In conclusion, Hy-CycleAlign represents a significant advancement in multi-modal image fusion by leveraging the unique geometric properties of hyperbolic space to solve the persistent problem of cross-modal misalignment.