Technical Summary: MEFP-Net for Urban Traffic Object Detection

Problem Statement

Urban road traffic safety faces significant challenges, particularly in detecting small and occluded objects within complex traffic scenarios. While single-modal detection methods (relying solely on RGB or infrared data) are mainstream, they struggle in diverse conditions: RGB cameras lack effectiveness in low-light environments, while infrared (IR) imagery suffers from low resolution and a lack of texture/color information, making small, low-contrast objects difficult to identify.

Existing multimodal fusion approaches, whether CNN-based or Transformer-based, often fail to meet real-time requirements or effectively handle complex real-world traffic scenarios. Specific limitations include:

• Feature Imbalance: Inherent differences between RGB and IR modalities (e.g., texture vs. thermal signatures) lead to suboptimal fusion.
• Fusion Misalignment: Early fusion strategies often result in information loss or noise interference.
• Illumination Sensitivity: Many methods are sensitive to lighting fluctuations, reducing robustness in conditions like nighttime, fog, or glare.
• Small Object Detection: Conventional methods often lack the global semantic capture efficiency and receptive field adaptability required for small, distant, or occluded targets.

Methodology

The paper proposes MEFP-Net (Multiscale Enhanced Feature Pyramid Network), a multimodal recognition algorithm designed to improve small object detection through hierarchical mid-stage fusion. The architecture employs a dual-path backbone to preserve modality-specific details before fusing features at a key intermediate stage.

Core Components

1. Dual-Path Backbone with Mid-Stage Fusion:
   Instead of early fusion, the network processes RGB and IR images through separate branches to maintain unique modality details. Fusion occurs at the mid-stage (P4/P5 layers), preventing information loss and preserving the complementary nature of the bimodal data.

2. BDR Module (Better Depth-Wise Residual):
   Replacing the standard C3k2 module, the BDR module addresses the insufficient receptive field issues in detecting occluded or blurred small objects.

   • Structure: It utilizes a three-branch convolutional structure with dilation rates of 1, 2, and 4 to capture multi-scale receptive fields without introducing feature pixel discontinuities.
   • Mechanism: It decomposes contextual information into regional and semantic residual components, integrating SE (Squeeze-and-Excitation) attention for global channel calibration. This reduces the "grid effect" of dilated convolutions and balances feature weights.

3. EMFP Module (Enhanced Multi-scale Feature Pyramid):
   Designed to enhance small object identification, this module integrates several advanced techniques:

   • SPDConv: Captures fine-grained features from the backbone's P2 layer to prevent feature loss.
   • SNI (Soft Nearest Neighbor Interpolation): Replaces traditional upsampling in the P5 layer to reduce alignment artifacts and preserve texture information during feature fusion.
   • G CSPOmniKernel: A cross-stage hierarchical structure that employs dual-domain channel attention and frequency-based spatial attention. It transforms features into the frequency domain to extract global contextual information and optimize receptive fields with minimal parameter overhead.

4. TMF Module (Traffic Multi-scale Feature):
   This module addresses the disparity in multi-scale resolutions within the feature pyramid, which often leads to the loss of small targets.

   • Function: It aligns multi-scale input features using 1×1 convolutions and SiLU activation.
   • Attention Mechanism: It generates a 3D attention coefficient matrix via global/local pooling and MLP to refine features and suppress thermal noise.
   • Outcome: It produces a single-scale feature map that filters background clutter and infrared noise while maintaining critical details.

Key Contributions

The paper outlines three primary contributions:

1. Dual-Path Independent Architecture: The introduction of a dual-path independent P5 backbone with P4/P5 dual-layer mid-stage fusion. This design prevents information loss associated with single-scale early fusion and fully preserves the complementary aspects of RGB-IR data.
2. Enhanced Feature Extraction (EMFP): The development of the EMFP module, which integrates SNI, SPDConv, G CSPOmniKernel, and TMF modules to improve the identification accuracy of small objects in multimodal contexts.
3. Balanced Detection Head (BDR): The introduction of the BDR module to achieve a trade-off between detection accuracy and model deployment efficiency, balancing computational complexity with feature augmentation.

Experimental Results

The model was evaluated on the M3FD dataset, a multimodal traffic scene dataset containing 4,200 pairs of aligned RGB and IR images (1024×768 resolution) covering four scene categories (day, night, fog, glare) and six object classes (pedestrians, bicycles, cars, buses, trucks, motorcycles).

• Performance Metrics: MEFP-Net achieved 85.5% mAP50 and 57.8% mAP50-95.
• Comparative Analysis: The model outperformed state-of-the-art multimodal fusion models, including:
  • SuperFusion: +2.0% in mAP50 and +1.8% in mAP50-95.
  • CFT (Transformer-based): +4.8% in mAP50 and +8.3% in mAP50-95.
  • Other competitors like ICAFusion, DetFusion, and DEYOLO were also significantly outperformed.
• Ablation Studies: Experiments confirmed that the integration of EMFP, TMF, and BDR modules provided additive improvements. Specifically, EMFP alone increased mAP50 by 3.4%, while the full integration of all modules yielded the optimal results.
• Robustness: The model demonstrated superior robustness in challenging conditions (nighttime, fog, glare, occlusion), effectively reducing false negative and false positive rates for distant and partially obscured targets.

Significance

The paper claims that MEFP-Net offers a practical solution for enhancing traffic safety by improving the detection of difficult targets in complex urban environments. By addressing feature imbalance, fusion misalignment, and illumination sensitivity through a hierarchical mid-stage fusion approach, the model enhances the accuracy of detecting small and occluded objects. The results suggest that the proposed architecture not only surpasses existing numerical benchmarks but also exhibits superior adaptability to real-world traffic challenges, validating its potential for deployment in multimodal small object detection tasks.