💻 computer science

Real-Time Underwater Image Enhancement via Frequency-Guided Dual-Path Attention

This paper proposes a lightweight, real-time underwater image enhancement framework that integrates frequency-guided dual-path attention and reparameterizable convolutions with fixed DCT priors to achieve state-of-the-art performance with minimal computational cost and high inference speed.

Original authors: Leshen Zhang, Ao Li, Ce Zhu

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Leshen Zhang, Ao Li, Ce Zhu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to take a photo underwater. It's like looking through a thick, greenish-blue fog. The light gets absorbed, colors get washed out, and the edges of things look blurry. This is a huge problem for robots or cameras that need to "see" clearly underwater to navigate or find things.

This paper introduces a new, super-fast computer program designed to clean up these blurry underwater photos. The authors call it a "Real-Time Underwater Image Enhancement" system. Think of it as a magical photo editor that works so fast it can run on a small robot's brain without slowing it down.

Here is how they did it, using simple analogies:

The Problem: The "Heavy" vs. The "Light"

Usually, to fix a blurry photo, you need a very heavy, complex computer brain (a large AI model). But these heavy brains are too slow and use too much power for small robots.
On the other hand, there are "ultra-lightweight" models that are fast and small, but they are like a person trying to fix a painting while wearing blindfolds. They only look at the picture's colors and shapes (the "spatial" view) but ignore the frequency (the hidden patterns of detail and color waves). Because underwater problems are all about how light waves get messed up, ignoring the "frequency" means the fix isn't perfect.

The Solution: A Two-Part Superpower

The authors built a tiny, fast model that learns to "see" both the picture and the hidden waves at the same time. They did this with two main tricks:

1. The "Pre-Loaded" Filter (MBRConv-DCT)
Imagine you are teaching a student to draw. Instead of letting them guess how to draw a straight line or a diagonal line, you give them a set of pre-drawn stencils (templates) that they can trace over.

How it works: The model has a special "training mode" where it uses fixed, pre-made mathematical patterns (called DCT kernels) that act like stencils for horizontal, vertical, and diagonal lines. These help the model understand the specific ways underwater images get blurry.
The Magic Trick: Once the student (the model) learns from these stencils, the stencils are removed! The model folds the knowledge of the stencils directly into its own brain. So, when it actually takes a photo (inference), it doesn't need the stencils anymore. It runs just as fast as a normal model, but it's already "smart" about those patterns.

2. The "Dual-Path" Detective (FGDPA)
Imagine a detective solving a crime. One detective looks at the physical evidence (the photo's colors and shapes), while a second detective looks at the "vibe" or the overall energy of the scene (the frequency).

How it works: This module has two paths.
- Path A (Spatial): Looks at the picture normally to find important details.
- Path B (Frequency): Takes a tiny, quick snapshot of the picture's "sound waves" (using a math tool called FFT) to see how the colors and edges are distributed globally. It's like listening to the hum of a room to know if a party is happening, rather than looking at every person.
The Result: These two detectives talk to each other. The "Frequency Detective" tells the "Spatial Detective," "Hey, the whole image is too green, let's fix that," or "The edges are too weak, let's sharpen them." This happens very quickly because the frequency check is done on a tiny, fixed-size grid, not the whole huge image.

The Results: Fast and Sharp

The authors tested their creation and found:

It's Tiny: The whole model is incredibly small (only about 4,200 "parameters," which is like having a very small vocabulary compared to other models that have millions).
It's Fast: It can process over 600 photos per second on a powerful computer, and over 100 photos per second on a small, embedded robot chip (like the Jetson AGX Orin).
It Works: When they compared it to other methods, their model produced clearer, more colorful, and sharper images than even much larger, heavier models. It fixed the "green fog" and restored the details better than the competition.

Summary

In short, the authors took a tiny, fast AI model and taught it to pay attention to the "hidden waves" of underwater images. They did this by giving it special training tools that disappear after learning, and by adding a quick "frequency check" that helps it fix colors and details. The result is a photo cleaner that is small enough for a robot to carry but smart enough to see clearly in the deep blue.

Technical Summary: Real-Time Underwater Image Enhancement via Frequency-Guided Dual-Path Attention

Problem Statement

Real-time underwater image enhancement (UIE) is essential for mobile photography, autonomous underwater vehicles (AUVs), and embedded vision systems. These applications demand models with low latency, compact memory footprints, and minimal power consumption. While recent ultra-lightweight Convolutional Neural Networks (CNNs) based on structural re-parameterization (e.g., RepVGG, MobileOne) meet these efficiency constraints, they operate primarily in the spatial domain. This approach ignores the frequency-sensitive nature of underwater degradation, where light absorption reshapes global spectral distributions (affecting color) and scattering attenuates high-frequency components (eroding edges). Existing frequency-aware methods often rely on heavy architectures or complex frequency decompositions that introduce computational overhead incompatible with real-time, resource-constrained edge deployment. Furthermore, classical lightweight attention mechanisms (e.g., SENet, CBAM) lack explicit frequency modeling, while current re-parameterized models tend to lose directional and spectral diversity when branches merge into single kernels during inference.

Methodology

The authors propose a lightweight UIE framework that integrates frequency-domain modeling into a re-parameterized CNN architecture without incurring inference overhead. The framework consists of two primary components:

1. Multi-Branch Reparameterizable Convolution with Fixed DCT Priors (MBRConv-DCT)

This module enhances the spectral representational capacity of standard re-parameterizable convolutions.

Mechanism: During training, MBRConv-DCT adds a dedicated linear branch containing fixed, non-DC $3 \times 3$ Discrete Cosine Transform (DCT) kernels. These kernels encode structured directional frequency responses (horizontal, vertical, diagonal) spanning low to high frequencies.
Training: The branch includes a learnable pointwise convolution and a scalar gate (initialized to zero) to ensure the branch does not disrupt early training.
Inference: Crucially, the DCT branch is entirely linear. During the re-parameterization phase, these fixed kernels and learned weights are analytically fused into a standard $3 \times 3$ convolution kernel. Consequently, MBRConv-DCT enriches the model's sensitivity to directional frequency cues during training but introduces zero additional parameters or FLOPs during inference.

2. Frequency-Guided Dual-Path Attention (FGDPA)

This module performs adaptive feature modulation by fusing spatial and spectral cues.

Frequency Path: To ensure resolution invariance and stability, the input feature map is downsampled to a fixed $32 \times 32$ grid. A 2D Fast Fourier Transform (FFT) is applied to obtain a log-compressed magnitude spectrum. Global average pooling aggregates these statistics to form a compact spectral descriptor. This descriptor guides channel modulation directly in the frequency domain, avoiding the computational cost of inverse FFTs.
Spatial Path: A parallel branch operates purely in the spatial domain, using max and average pooling to generate a spatial saliency map.
Fusion: The two paths are combined via a static, learnable hybrid gating mechanism. The final attention map is a weighted sum and element-wise product of the frequency-guided channel attention and the spatial saliency map.
Efficiency: The frequency branch operates at a constant $32 \times 32$ resolution, and all convolutions are $1 \times 1$ , ensuring minimal runtime overhead.

The framework retains the Feature Self-Transform (FST) module from MobileIE to enhance nonlinear feature interactions. The entire network adopts a re-parameterizable training structure that collapses into a lightweight feed-forward model for inference.

Key Contributions

FGDPA Module: An ultra-lightweight attention mechanism that constructs a resolution-invariant spectral descriptor from fixed-resolution FFT magnitudes. It leverages global spectral statistics to guide channel modulation without inverse FFTs, capturing global degradation cues efficiently.
MBRConv-DCT: A multi-branch re-parameterizable convolution that injects fixed non-DC DCT directional priors during training. This linear branch is analytically collapsible, enriching spectral diversity without adding inference cost.
Integration of Frequency Guidance: The work demonstrates that explicit frequency guidance can be seamlessly integrated into ultra-lightweight, re-parameterized CNNs, achieving state-of-the-art performance with only 4.23K parameters.

Experimental Results

The proposed method was evaluated on the UIEB dataset (supervised) and the EUVP and UIQS datasets (unsupervised/no-reference).

Performance: On the UIEB benchmark, the model achieved 23.97 dB PSNR and 0.9155 SSIM, outperforming the strongest lightweight competitor (MobileIE) by over 1.1 dB while maintaining a comparable parameter count (~4.2K). It also surpassed the much larger U-Shape Transformer (31.5M parameters) in both distortion-based and perceptual metrics (lowest LPIPS of 0.1188).
Generalization: On real-world datasets (EUVP and UIQS), the model demonstrated superior robustness, achieving state-of-the-art UIQM and UCIQE scores on EUVP and ranking second overall on UIQS.
Efficiency: The model operates at 628+ FPS at $480 \times 640$ resolution on an NVIDIA RTX 3090. On an embedded platform (Jetson AGX Orin), it achieves 106.9 FPS with a latency of 9.35 ms.
Ablation Studies: Removing the frequency magnitude from the attention pathway caused a significant performance drop (from 23.97 dB to 22.81 dB), confirming that frequency-aware channel modulation is indispensable. The fixed DCT kernels provided a robust spectral inductive bias, with performance remaining stable across different kernel counts ( $K=2$ to $8$).

Significance and Claims

The paper claims that explicit frequency modeling can be effectively integrated into ultra-lightweight CNNs without sacrificing real-time performance. By utilizing structured spectral priors (via MBRConv-DCT) and lightweight FFT-based statistics (via FGDPA), the proposed framework addresses the specific frequency-sensitive degradation of underwater images that spatial-only models miss.

The authors position this work as a solution to the "efficiency-effectiveness gap" in current ultra-lightweight re-parameterized models. They demonstrate that performance gains in this domain do not necessarily require increasing model depth or parameter counts, but rather improving the representational efficiency of each parameter through frequency-aware design. The results suggest strong potential for deploying these models on resource-constrained platforms such as underwater drones and embedded robotic systems.