💻 computer science

BVI-Mamba: Video Enhancement Using a Visual State-Space Model for Low-Light and Underwater Environments

This paper introduces BVI-Mamba, a novel video enhancement framework that leverages a Visual State-Space (VSS) model to efficiently address noise, low contrast, and color imbalance in low-light and underwater videos while significantly reducing computational resources compared to existing Transformer and convolution-based methods.

Original authors: Guoxi Huang, Ruirui Lin, Yini Li, David R. Bull, Nantheera Anantrasirichai

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Guoxi Huang, Ruirui Lin, Yini Li, David R. Bull, Nantheera Anantrasirichai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to watch a movie, but the screen is covered in fog, the colors are washed out, and the picture is shaky. This is what happens when cameras try to record in very dark places (like a cave at night) or underwater (where light gets scattered and absorbed). The resulting video is often grainy, blurry, and hard to understand.

The paper introduces a new tool called BVI-Mamba to fix these messy videos. Think of it as a "smart video restorer" that uses a new kind of artificial intelligence brain to clean up the footage quickly and efficiently.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Heavy Backpack" of Old AI

Previously, fixing these videos required AI models that were like heavy backpacks. They were very powerful but took up a lot of memory and time to run, especially for video (which is just a long series of images).

Old methods were like trying to fix a whole movie by looking at one frame at a time, or by using a giant, slow-moving robot to compare frames.
The new method (BVI-Mamba) is like a lightweight, agile runner. It uses a new type of AI architecture called "Visual State Space" (or Mamba). This allows it to process long sequences of video without getting tired or needing a massive computer.

2. The Two-Step Process: Aligning and Polishing

The BVI-Mamba system works in two main stages, like a two-person team cleaning a dirty window:

Step 1: The "Steady Hand" (Feature Alignment)
When you film in the dark or underwater, your hand might shake, or the subject might move. This causes the video to look jittery or "ghostly" when you try to combine frames to make it clearer.

The Analogy: Imagine trying to stack a deck of cards that are all slightly shifted. If you just glue them together, the picture is blurry.
What BVI-Mamba does: It acts like a super-precise card shuffler. Before it tries to clean the image, it looks at the "features" (the shapes and edges) of the video frames and lines them up perfectly in the digital space. It uses a special "Pyramid" system to handle big movements and small details, ensuring that the frames match up so the video doesn't flicker.

Step 2: The "Magic Polisher" (Enhancement)
Once the frames are lined up, the system needs to fix the darkness, noise, and color.

The Analogy: Think of a standard AI as a paintbrush that only knows how to paint small, local spots. It struggles to see the "big picture" of the whole room.
What BVI-Mamba does: It replaces the paintbrush with a laser scanner (called a Visual State Space block). This scanner can look at a tiny speck of dust and the entire room at the same time. It understands how the light travels across the whole scene. This allows it to remove the "snow" (noise), brighten the dark areas, and fix the weird blue/green tints of underwater footage much better than older methods.

3. Why It's Better (The Results)

The authors tested this new "runner" against the old "heavy backpacks" (models based on CNNs and Transformers).

The Race: In tests using real low-light and underwater video datasets, BVI-Mamba won.
The Score: It produced clearer images with higher "signal-to-noise" ratios (less grain) and better structural similarity (the shapes looked more natural).
The Efficiency: It did this while using less computer memory and time. It's not just a better cleaner; it's a faster one.

4. The Underwater Challenge

Underwater video is tricky because water acts like a filter that eats red light and scatters blue light, making everything look green or blue.

Since there are no "perfect" underwater videos to compare against (no "ground truth"), the team used a clever trick. They trained the AI on clear, low-light land videos first. Then, before feeding it underwater footage, they gave the AI a "color correction glasses" (chromatic adaptation) to adjust the white balance.
The result? The AI successfully removed the murky blue haze and brought out details that were previously invisible, outperforming other specialized underwater tools.

Summary

BVI-Mamba is a new video enhancement tool that uses a "lightweight" AI brain to fix shaky, dark, and murky videos. Instead of struggling with heavy calculations, it lines up the frames perfectly and then uses a smart scanning technique to clean up the image. The paper claims it is faster, uses less memory, and produces clearer results than previous AI methods for both dark environments and underwater scenes.

1. Problem Statement

Videos captured in low-light and underwater environments suffer from severe quality degradations, including:

Low-light issues: Noise, low contrast, color imbalance, and motion blur (caused by slow shutter speeds or high ISO).
Underwater issues: Light scattering and absorption leading to blurring, halo effects, color distortion, and reduced contrast.

These degradations compromise both aesthetic quality and the performance of downstream computer vision tasks (e.g., object detection, tracking). While AI-based image enhancement exists, video enhancement remains challenging due to:

Temporal inconsistency: Flickering and brightness fluctuations between frames.
Computational cost: Existing deep learning methods (CNNs, Transformers) are computationally expensive and memory-intensive, making real-time processing difficult.
Data scarcity: A lack of large-scale, paired training datasets, particularly for underwater scenarios.

2. Methodology: BVI-Mamba

The authors propose BVI-Mamba, a novel framework that replaces traditional convolutional and Transformer blocks with Visual State Space Models (VSS) to achieve linear computational complexity while maintaining high performance. The architecture follows a two-stage pipeline:

A. Feature Alignment Module

To ensure temporal consistency and prevent flickering, the model aligns multiple input frames before enhancement.

Mechanism: Utilizes a Pyramid, Cascading, and Deformable (PCD) alignment module (adapted from EDVR) integrated with Temporal and Spatial Attention (TSA) fusion.
Function: Registers spatio-temporal displacement between frames in the feature space, extracting and processing temporal details to maintain consistency across consecutive frames.

B. Enhancement Module

This module performs noise removal and brightness adjustment using a UNet-like architecture.

Core Innovation: All standard convolutional layers and Swin Transformer blocks are replaced with Visual State Space (VSS) blocks.
VSS Block Structure:
- Based on the Mamba framework, specifically utilizing the 2D-Selective-Scan (SS2D) mechanism.
- SS2D Process: Unfolds input patches into sequences along four directions (Left-Right, Top-Down, Right-Left, Down-Top), processes them via a Selective State Space Model (SSM), and merges the results.
- Advantage: Unlike CNNs (limited receptive field) or Transformers (quadratic complexity), VSS blocks capture both local details and long-range global dependencies with linear computational complexity.
Mathematical Formulation: The VSS block in layer $l$ is defined as:
$h_l = SS2D(LN(h_{l-1})) + h_{l-1}$
$h_{l+1} = FFN(LN(h_l)) + h_l$
(Where $LN$ is Layer Normalization and $FFN$ is a Feed-Forward Network.)

3. Key Contributions

First Application of Mamba to Low-Light Video: The paper introduces the first framework applying the Visual State Space (VSS) model to video enhancement (previously limited to images), specifically targeting low-light and underwater scenarios.
Efficient Architecture: By replacing heavy Transformer blocks with VSS blocks, BVI-Mamba achieves linear computational complexity, significantly reducing memory usage and inference time compared to CNN and Transformer-based counterparts.
Robust Temporal Alignment: The integration of PCD alignment with VSS enhancement ensures high temporal consistency, effectively mitigating flickering and brightness fluctuations common in video enhancement.
Underwater Adaptation: The framework includes a pre-processing step using chromatic adaptation to address severe color imbalances in underwater data, compensating for the lack of ground truth in underwater datasets.

4. Experimental Results

The model was evaluated on three datasets: DID (Low-light), BVI-RLV (Low-light), and BVI-Coral (Underwater).

Quantitative Performance (Low-Light):
- On the BVI-RLV dataset, BVI-Mamba achieved a PSNR of 22.93 and SSIM of 0.820, outperforming:
  - PCDUNet (CNN): PSNR 19.50, SSIM 0.757.
  - STA-SUNet (Transformer): PSNR 20.64, SSIM 0.765.
  - BVI-CDM (Diffusion): PSNR 22.20, SSIM 0.773.
- It also achieved the lowest LPIPS (0.146), indicating superior perceptual similarity to the ground truth.
Qualitative Performance:
- Visual comparisons show BVI-Mamba effectively reduces noise, sharpens details, and preserves fine textures better than competing methods.
Underwater Results:
- Using a model pretrained on BVI-RLV and adapted with chromatic adaptation, BVI-Mamba produced visually superior results compared to Zero-TIG, correcting color casts and improving visibility in deep-sea footage.

5. Significance

Paradigm Shift: This work demonstrates that State Space Models (SSMs) are a viable and superior alternative to Transformers for video processing tasks, offering a better balance between performance and computational efficiency.
Practical Deployment: The reduced memory footprint and linear complexity make BVI-Mamba a strong candidate for deployment on edge devices or in real-time surveillance and autonomous navigation systems where resources are constrained.
Domain Expansion: It successfully bridges the gap between advanced generative/discriminative models and challenging physical environments (low-light/underwater), providing a robust solution for critical applications like underwater exploration and night-time security.