💻 computer science

BVI-RLV: A Fully Registered Dataset for Low-Light Video Enhancement

This paper introduces BVI-RLV, a fully registered low-light video dataset featuring over 30,000 sub-pixel aligned frames across diverse dynamic scenes, which demonstrates that precise registration is critical for training deep learning models to achieve superior low-light video enhancement performance compared to existing unregistered datasets.

Original authors: Ruirui Lin, Guoxi Huang, Joanne Lin, Qi Sun, Alexandra Malyugina, David R Bull, Nantheera Anantrasirichai

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ruirui Lin, Guoxi Huang, Joanne Lin, Qi Sun, Alexandra Malyugina, David R Bull, Nantheera Anantrasirichai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Blurry, Noisy" Night Vision

Imagine trying to watch a movie in a dark room where the projector is broken. The picture is grainy, the colors are weird, and if someone moves, the image gets smeared or leaves ghostly trails. This is what happens when cameras try to record videos in low light.

In the world of computers (specifically "computer vision"), this is a huge headache. If a self-driving car or a security robot can't see clearly in the dark, it might miss a pedestrian or fail to spot an intruder.

Scientists have been trying to fix this using Artificial Intelligence (AI). Think of AI as a student that needs to study examples to learn how to fix these bad videos. To learn effectively, the student needs a textbook with two things side-by-side:

The Bad Picture: The dark, noisy video.
The Good Picture: A perfect, bright, clear version of the exact same moment.

The problem is, finding these "Good Pictures" for moving videos is incredibly hard. If the camera moves even a tiny bit between taking the dark photo and the bright photo, the two images won't line up. It's like trying to trace a drawing while your hand is shaking; the result is a messy blur.

The Solution: BVI-RLV (The Perfect Textbook)

The authors of this paper created a new dataset called BVI-RLV. Think of this as a brand-new, perfectly organized textbook for AI students.

Here is what makes it special:

1. The "Robot Arm" Setup (Precision Alignment)
Usually, when people try to take a dark photo and a bright photo of the same scene, they do it by hand or with shaky equipment. The images end up slightly misaligned, like two puzzle pieces that don't quite fit.

The Paper's Trick: They built a setup using a motorized dolly (a robot cart that moves the camera) in a controlled studio. They programmed the cart to move in perfect loops.
The Analogy: Imagine a dancer spinning in a circle. If you take a photo of them in the dark, then immediately take a photo in the light, they might have moved an inch. But if you have a robot that spins the dancer exactly the same way every time, you can take the dark photo, reset, take the light photo, and they will be in the exact same spot.
The Result: They achieved "sub-pixel registration." This means the images are aligned so perfectly that the error is smaller than a single dot on a screen. 99.24% of their data is perfectly lined up.

2. The "Realism" Factor (Not Just Fake Noise)
Some old datasets tried to fake low light by putting a dark filter (like sunglasses) over a bright camera. But that's like trying to learn how to swim by wearing a life jacket in a pool; it doesn't feel like real water.

The Paper's Trick: They actually turned down the lights in the room to 10% and 20% of normal brightness.
The Result: The noise and blur in their videos are real. They capture the actual graininess and motion blur that happens when a camera struggles in the dark, not just a computer simulation.

3. The "Action Movie" Variety (Motion Diversity)
Many old datasets only had cameras that stayed still or moved in a straight line. Real life is messy; cars turn, people run, and cameras shake.

The Paper's Trick: They filmed 40 different scenes with objects moving in straight lines, curves, rotations, and angles.
The Result: The AI gets trained on a wide variety of "action," making it smarter about handling movement.

The Experiments: Does the New Textbook Work?

The authors didn't just make the dataset; they tested it. They took four different types of AI "students" (based on CNNs, Transformers, Mamba, and Diffusion Models) and taught them using:

The new BVI-RLV textbook.
Three old textbooks (existing datasets).

The Results:

The "Perfect Alignment" Advantage: When they trained the AI on the new dataset, the results were much sharper. In fact, simply having the images perfectly lined up (registered) improved the quality by up to 5.85 dB (a significant jump in video quality) compared to using messy, misaligned data.
The "Generalization" Test: They tested the AI on videos it had never seen before, including real outdoor night scenes. The AI trained on BVI-RLV performed better than the AI trained on the old datasets. It was better at removing noise and keeping details clear.
The "Downstream" Test: They even checked if the improved videos helped other tasks, like counting objects or tracking them. The videos made by the BVI-RLV-trained AI helped these other tasks work better, too.

The Big Takeaway

The paper argues that alignment is everything. You can have the most powerful AI in the world, but if you train it on blurry, misaligned data, it will learn bad habits.

By building a dataset where the "dark" and "bright" videos are perfectly synchronized using a robot, the authors gave AI a clear, high-quality map to learn from. This allows the AI to learn how to fix low-light videos much better than before, even in real-world situations outside the studio.

In short: They built a perfect training ground for AI to learn how to see in the dark, and the results show that when the training data is precise, the AI becomes a master at restoring dark, noisy videos.

Technical Summary of BVI-RLV: A Fully Registered Dataset for Low-Light Video Enhancement

Problem Statement
Low-light video enhancement (LLVE) is critical for applications in surveillance, robotics, and natural history filmmaking. However, existing deep learning approaches face a significant bottleneck: the scarcity of pixel-aligned, high-quality training data. Low-light footage suffers from spatiotemporally incoherent noise, motion blur, and color distortions. While image-based enhancement has advanced, applying these methods frame-by-frame to video causes temporal inconsistencies. Furthermore, existing LLVE datasets are often limited by misalignment issues, lack of motion diversity, or reliance on synthetic degradations that fail to capture real-world noise and blur. Crucially, pixel-wise loss functions used in supervised learning are highly sensitive to spatial offsets; even minor misalignments between low-light inputs and ground truth can introduce ghosting artifacts and bias quantitative evaluation.

Methodology
To address these challenges, the authors introduce BVI-RLV (Registered Low-light Videos), a fully registered dataset designed for supervised learning. The acquisition methodology prioritizes precise alignment over uncontrolled outdoor capture:

Acquisition Environment: Videos were captured in a controlled indoor studio using a Sony Alpha 7SII camera. Lighting was managed via Cineo MavX lights (6,500 K) at three levels: 100% (normal-light ground truth), 20%, and 10% (low-light).
Motion Control: A programmable Kessler CineDrive motorized dolly system was employed to generate repeatable motion profiles, including linear, nonlinear, rotational, and angled trajectories. This setup ensures consistent motion across repeated captures.
Ground Truth Generation: To achieve sub-pixel registration without warping or altering noise characteristics, the authors employed a repetition-and-selection strategy. For each low-light sequence, multiple normal-light sequences were recorded using identical motion profiles. A pseudo-ground truth frame was selected from these repetitions by minimizing the Mean Absolute Error (MAE) after histogram matching, rather than applying computational warping.
Alignment Verification: The alignment accuracy was verified using Normalized Cross-Correlation (NCC) and Harris Corner Detection. The results indicate that 99.24% of data pairs achieve sub-pixel registration accuracy (average error of 0.256 pixels via NCC).
Baseline Implementations: To ensure reproducibility and benchmarking, the authors implemented four representative architectures trained on BVI-RLV and existing datasets: a CNN-based model (PCD alignment with U-Net), a Transformer-based model (Swin Transformer), a State Space Model (Mamba), and a Diffusion Model (DM).

Key Contributions

BVI-RLV Dataset: A publicly available dataset comprising 40 diverse indoor scenes and over 31,800 paired frames at Full HD resolution. It features two controlled low-light conditions (10% and 20% intensity) paired with 100% normal-light ground truth.
High-Precision Registration: Unlike existing datasets that rely on Neutral Density (ND) filters or suffer from misalignment, BVI-RLV achieves sub-pixel registration for 99.24% of frames, verified by cross-correlation and corner detection.
Motion and Content Diversity: The dataset covers a wider range of motion types (linear, nonlinear, rotational, angled) and content textures compared to existing benchmarks like DRV, SDSD, and DID.
Comprehensive Benchmarking: The paper provides standardized baseline implementations across CNN, Transformer, Mamba, and Diffusion architectures, facilitating fair comparison across datasets.

Experimental Results

Cross-Dataset Performance: Models trained on BVI-RLV consistently outperform those trained on existing datasets (DRV, SDSD, DID) when evaluated on cross-dataset test sets. For instance, the Mamba-based model trained on BVI-RLV achieved the highest PSNR/SSIM/LPIPS scores across all test sets.
Importance of Registration: An ablation study demonstrated that training on unregistered data leads to significant performance degradation. Specifically, using registered data improved PSNR by up to 5.85 dB for the CGAN architecture under $\ell_1$ loss compared to unregistered training.
Generalization to Outdoor Scenes: Despite being captured indoors, models trained on BVI-RLV generalized effectively to real-world outdoor scenes (tested on SDSD and DID outdoor sets), producing sharper textures and better noise suppression than models trained on outdoor datasets.
Downstream Task Improvement: Enhanced frames from BVI-RLV-trained models improved downstream computer vision tasks, specifically increasing Mask AP by 5.28 points and Box AP by 3.40 points for video instance segmentation and object detection on the ELVIS-S dataset.
Illumination Robustness: Training on mixed illumination levels (10% and 20%) yielded better robustness to unseen lighting conditions compared to training on single levels, improving cross-light performance by approximately 3.0 dB.

Significance
The paper asserts that accurate registration is a fundamental prerequisite for effective supervised low-light video enhancement. The authors claim that the superior performance of models trained on BVI-RLV is driven by data quality—specifically the combination of sub-pixel alignment, authentic noise characteristics, and diverse motion patterns—rather than architectural complexity. By providing a fully registered benchmark, BVI-RLV addresses the "ill-posed" nature of LLVE where obtaining accurate ground truth is typically challenging, thereby enabling more reliable training and evaluation of future enhancement algorithms. The dataset is publicly available to foster further research in robust, real-world low-light video enhancement.

The Problem: The "Blurry, Noisy" Night Vision

The Solution: BVI-RLV (The Perfect Textbook)

The Experiments: Does the New Textbook Work?

The Big Takeaway

More like this