Technical Summary: InfraNet: Quality-Aware RGB Guidance for Efficient Infrared Object Detection

1. Problem Statement

Robust object detection under adverse visual conditions (e.g., low-light, glare, fog) remains a significant challenge for multi-modal perception systems. While Infrared (IR) sensing offers robustness by capturing thermal radiation independent of ambient light, existing fusion-based methods typically treat RGB and IR modalities equally during both training and inference. This "equal treatment" assumption overlooks a critical flaw: the unreliability of the RGB modality in degraded conditions.

When RGB images are severely degraded (e.g., becoming dark or noisy), indiscriminately fusing them with reliable IR features can introduce irrelevant or harmful information, leading to "negative transfer" that impairs overall detection accuracy. Furthermore, most current state-of-the-art (SOTA) fusion methods mandate dual-modal input at inference, rendering them impractical when RGB sensors are unavailable or corrupted. There is a need for a framework that leverages RGB as a guidance signal during training to enhance IR learning but allows for efficient, high-accuracy IR-only deployment.

2. Methodology

Core Framework: InfraNet

The authors propose InfraNet, an IR-centric, quality-aware framework designed to regulate RGB guidance during training while supporting flexible deployment in either RGB–IR or IR-only modes. InfraNet employs an asymmetric architecture consisting of:

1. Primary IR Pathway: Extracts multi-scale infrared features for predictions.
2. Auxiliary RGB Pathway: Provides reliability-controlled supervisory signals during training.

Key Component: QualGate

The core innovation of InfraNet is QualGate, a quality-aware fusion module deployed at multiple scales within the network. QualGate learns a task-oriented control signal to dynamically regulate cross-modal interactions. Its operation involves two parallel mechanisms:

• Reliability Control Score ($q$): A scalar score $q \in (0, 1)$ is predicted from the auxiliary RGB features ($A_{in}$) using a lightweight MLP. This score is optimized for the detection objective rather than serving as a hand-crafted image quality metric.
• RGB Suppression Gate: The auxiliary contribution is weighted by $w_{rgb}(q) = q$. When RGB quality is poor (low $q$), the RGB guidance is down-weighted to prevent noise injection.
• IR Amplification Gate: Simultaneously, an inverse reliability gain $\alpha(q)$ is applied to the IR stream. Defined as $\alpha(q) = \text{clip}(1.5 - q; 1.0, 1.5)$, this mechanism increases the contribution of IR features when RGB guidance is weak, ensuring the IR representation remains robust.
• Fusion: The final output $A_{out}$ is a fusion of the suppressed auxiliary term and the amplified IR injection:
  $$A_{out} = w_{rgb}(q) \cdot A_{in} + \sum_{k=1}^{K} \alpha(q) \cdot g_k \cdot F_k$$
  where $F_k$ represents aligned IR features and $g_k$ is a content gate.

Architectural Variants

Based on InfraNet, two specific architectures are designed:

1. InfraNet-IR: A lightweight, single-branch network optimized for computational efficiency. It is trained with paired RGB–IR inputs (using RGB as privileged auxiliary supervision) but deployed with IR-only inference. The RGB branch and all fusion modules are removed at test time, incurring zero additional overhead.
2. InfraNet-RGB-IR: A higher-capacity dual-branch structure that processes both RGB and IR inputs during both training and inference, utilizing QualGate for reliability-controlled cross-modal fusion.

Training Strategy

The framework uses an asymmetric loss function:
$$\mathcal{L} = \mathcal{L}^{(\text{main})}_{\text{det}} + w_{\text{aux}} \mathcal{L}^{(\text{aux})}_{\text{det}}$$
where the auxiliary loss weight $w_{\text{aux}}$ is set to 0.25. This ensures optimization is dominated by the infrared pathway, while the auxiliary branch provides adaptive guidance to enhance IR representations.

3. Key Contributions

• InfraNet Framework: A quality-aware, RGB-guided framework for infrared object detection that treats RGB as a reliability-controlled auxiliary signal during training, enabling flexible IR-only deployment.
• QualGate Module: A novel fusion mechanism that learns a task-oriented control signal to suppress unreliable RGB guidance and compensate IR features, mitigating the risk of negative transfer from degraded RGB data.
• Dual-Mode Deployment: The design of two variants (InfraNet-IR and InfraNet-RGB-IR) that achieve competitive performance in both efficient single-modal inference and high-capacity dual-modal fusion.

4. Experimental Results

The method was evaluated on four benchmark datasets: LLVIP, FLIR-Aligned, M3FD, and DroneVehicle.

• Performance: InfraNet achieved strong or competitive accuracy across all datasets.
  • On LLVIP, the RGB–IR variant (YOLOv8) achieved 70.5% mAP, outperforming the SOTA WaveMamba (66.0%). The IR-only variant achieved 68.8% mAP with no inference overhead.
  • On FLIR-Aligned, the IR-only variant (ResNet50) achieved 9.4ms inference time with 53.0% mAP, significantly faster than dual-modal competitors while maintaining competitive accuracy.
  • On DroneVehicle, the IR-only YOLOv12 variant achieved 84.7% mAP50, surpassing WaveMamba by +4.9% absolute gain, with notable improvements in small-object detection (e.g., Cars, Buses).
  • On M3FD, the method demonstrated robustness in night and challenge subsets, where RGB visibility is severely compromised.
• Efficiency: InfraNet-IR maintains high efficiency in IR-only inference, removing the RGB branch and fusion modules entirely at deployment.
• Ablation Studies: Experiments confirmed that both the RGB suppression and IR amplification branches of QualGate are necessary; removing either leads to performance drops. The learned control score $q$ was shown to adapt effectively to controlled RGB degradations (fog, brightness) without explicit quality annotations.

5. Significance and Claims

The paper claims that InfraNet addresses the long-standing issue of RGB unreliability in adverse conditions by shifting the paradigm from "equal fusion" to "IR-centric quality-aware guidance."

• Robustness: By suppressing unreliable RGB cues and amplifying IR features, the method prevents the contamination of IR representations, leading to more stable detection in low-light and adverse weather.
• Deployment Flexibility: Unlike existing fusion methods that require dual-modal input at inference, InfraNet-IR offers a practical solution for scenarios where RGB sensors are unavailable or corrupted, achieving high accuracy without the computational cost of maintaining a dual-stream architecture at runtime.
• Efficiency: The framework demonstrates that auxiliary supervision during training can significantly boost IR-only performance without incurring deployment overhead, making it suitable for resource-constrained applications like autonomous driving and UAV surveillance.