💻 computer science

VDFP: Video Deflickering with Flicker-banding Priors

This paper addresses the challenge of digital screen banding in smartphone videos by introducing the DeViD real-world dataset and proposing VDFP, a novel perception-guided generation framework that utilizes degradation field modeling and spatial-temporal continuous priors to effectively eliminate flicker while preserving high-fidelity details and temporal consistency.

Original authors: Zhiyi Zhou, Libo Zhu, Zihan Zhou, Yulun Zhang, Xiaokang Yang

Published 2026-05-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhiyi Zhou, Libo Zhu, Zihan Zhou, Yulun Zhang, Xiaokang Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Ghost Stripes" on Your Phone

Imagine you are at a concert or watching a big game on a giant stadium screen. You pull out your smartphone to record a video of the screen. When you play it back, the video looks terrible. Instead of a clear image, you see thick, dark, and light horizontal stripes rolling across the screen, or maybe jagged, cracked patterns.

This is called flicker-banding. It happens because your phone's camera and the big screen are trying to talk to each other, but they are speaking different "languages" of time.

The Screen: Flashes its light on and off incredibly fast (like a strobe light) to create the image.
The Camera: Doesn't take a picture of the whole scene at once. It scans the image from top to bottom, row by row (like a scanner in an office), a split-second at a time.

Because they aren't perfectly synchronized, the camera catches the screen at different stages of its flashing cycle. The top of the video sees the screen when it's bright, the middle sees it when it's dim, and the bottom sees it again when it's bright. This creates those annoying, moving stripes.

The Challenge: No Recipe Book

Before this paper, fixing this problem was like trying to bake a cake without a recipe.

No Data: There were no good video datasets (collections of "bad" videos and their "good" versions) specifically for this screen-banding problem. Researchers had to guess what the problem looked like.
Wrong Tools: Existing video repair tools were like trying to fix a broken watch with a hammer. They were designed for other problems (like removing grainy noise or blurring) and just made the banding worse or turned the video into a blurry mess.

The Solution: VDFP (The "Smart Detective")

The authors created a new system called VDFP (Video Deflickering with Flicker-banding Priors). Think of it as a two-step detective process.

Step 1: Building the "Training Gym" (The Datasets)

You can't train a detective without crime scenes.

The Real-World Gym (DeViD): The team went out and filmed 108 different real-life scenarios (sports, cartoons, ads) using phones and LED screens to create a massive library of "bad" videos. This is the DeViD dataset.
The Simulation Gym (DFM): Since they couldn't film every possible type of stripe, they built a "physics simulator." This is a computer program that mathematically generates fake banding. It mimics how the camera scans the screen, creating complex patterns like curved stripes, cracked lines, and diamond shapes. This teaches the AI what to look for without needing a real camera for every single test.

Step 2: The Two-Stage Repair Process

The VDFP model works like a master restorer with a special assistant.

Stage 1: The "Spotter" (CPP Module)
Before trying to fix the video, the system needs to know where the stripes are.

Imagine looking at a dirty window. Instead of just guessing where the dirt is, you use a special flashlight that highlights the grime.
This module creates a "Confidence Map." It doesn't just say "Stripe here" or "No stripe there." It draws a smooth, continuous map showing exactly how bright or dark the stripes are and how they move over time. It understands that the stripes are fluid and shifting, not just static blocks.

Stage 2: The "Restorer" (The Diffusion Model)
Now that the "Spotter" has drawn the map, the main AI (a powerful video generator) gets to work.

The Zero-Initialization Trick: Usually, when you add a new tool to a powerful AI, it gets confused and forgets what it already knows. The authors used a clever trick: they added the "Stripe Map" to the AI but told the new connection to start with a value of zero.
The Analogy: Imagine a master chef (the AI) who knows how to cook a perfect steak. You hand them a new ingredient (the stripe map) but tell them, "Don't use this yet; just hold it." Because the connection starts at zero, the chef doesn't ruin their perfect steak recipe immediately. As they practice, they slowly learn how to use that new ingredient to remove the "burnt bits" (the stripes) without ruining the meat (the actual video details).

The Results: Clearer Than Ever

The team tested their new system against other video repair tools.

The Competition: Other tools either left faint stripes behind, made the video look blurry, or couldn't handle the complex, moving patterns.
VDFP: It successfully removed almost all the stripes, even the tricky, curved, and cracked ones, while keeping the video sharp and the colors true. It preserved the fine details (like a player's face or text on a screen) that other methods destroyed.

Summary

In short, the authors realized that fixing screen-banding on video was a unique problem that no one had solved well because they lacked data and the right tools. They built a massive library of real and fake "bad" videos, created a simulator to teach the AI how stripes move, and built a two-step system that first "maps" the stripes and then carefully "erases" them without blurring the rest of the picture. The result is a video that looks like it was filmed perfectly, even if the phone and screen were out of sync.

Technical Summary: VDFP (Video Deflickering with Flicker-banding Priors)

1. Problem Definition

The paper addresses the prevalent yet under-explored issue of video deflickering caused by capturing digital screens (specifically LED matrices) with smartphones. This phenomenon results in severe, structured, periodic luminance fluctuations known as banding artifacts. These artifacts arise from a temporal synchronization mismatch between the camera's rolling shutter mechanism (which reads the scene row-by-row) and the display's high-frequency light modulation (e.g., PWM in OLEDs or multiplexed scanning in LEDs).

Existing video restoration methods (e.g., super-resolution, deblurring, denoising) and specific flicker removal models (e.g., for ambient light bursts) fail to handle this task effectively. They often leave residual banding or produce over-smoothed textures because they do not account for the complex, simultaneous occurrence of different banding types (e.g., thin high-frequency stripes and thick low-frequency bands) and the lack of suitable training data.

2. Methodology: VDFP

The authors propose VDFP (Video Deflickering with Flicker-banding Priors), a novel perception-guided generation framework consisting of two main stages:

A. Data Construction and Simulation

To address the scarcity of real-world data, the authors introduce DeViD, a real-world dataset containing 108 videos across diverse scenes (sports, advertisements, cartoons, etc.).

Degradation Field Modeling (DFM): Since real-world capture is limited, the authors propose a physics-informed simulation pipeline based on the rolling shutter mechanism.
- Kinematic Spatiotemporal Modeling: It generates parameterized degradation fields ( $O_l$ ) by computing the projection of pixel coordinates onto the stripe axis, incorporating a time-dependent phase shift to simulate the rolling shutter delay.
- Dual-layer Banding Fusion: The model synthesizes complex scenarios by fusing two independent layers: a base stripe layer (high-frequency banding) and a thick stripe layer. The final degradation is the maximum of these two layers, simulating the coexistence of multiple artifact types found in reality.

B. The VDFP Framework

The restoration process is a two-stage pipeline built upon a pre-trained video diffusion model (specifically adapting the STAR architecture):

Spatial-Temporal Continuous Prior Perception (CPP):
- Instead of binary segmentation, a Video-SwinUNet is trained to predict a continuous artifact confidence map ( $\hat{A}_{flicker} \in [0, 1]$ ).
- This module is optimized using a Flicker-Aware Mean Squared Error (FA-MSE) loss, which captures the full spectrum of luminance transitions and stripe edges, rather than just classifying pixels as "flicker" or "no flicker."
Banding-Aware Condition Injection:
- The predicted confidence map is injected into the pre-trained diffusion model as an additional channel.
- Zero-Initialization Strategy: The input layer is expanded from 4 to 5 channels. The weights corresponding to the new prior channel are zero-initialized. This ensures that at the start of training, the model relies entirely on its pre-trained generative priors, preventing catastrophic distribution shifts. As training progresses, the model learns to utilize the prior map to guide the restoration.
- This approach allows the model to selectively suppress flicker-banding while preserving high-fidelity spatial details and temporal consistency without over-smoothing the background.

3. Key Contributions

The paper outlines four primary contributions:

DeViD Dataset: The construction of a real-world video deflickering dataset covering various daily life scenes to facilitate training and evaluation.
Degradation Field Modeling (DFM): A simulation pipeline based on rolling shutter physics that synthesizes complex, multi-banding scenarios (combining base and thick banding) to address data scarcity.
Spatial-Temporal Continuous Prior Perception (CPP): A novel module utilizing FA-MSE loss to generate continuous confidence maps, coupled with a zero-initialized condition injection strategy to guide the diffusion model.
Performance Superiority: Extensive experiments demonstrating that VDFP significantly outperforms existing methods in both quantitative metrics and visual quality, effectively removing complex banding while maintaining structural integrity.

4. Experimental Results

The authors evaluated VDFP against state-of-the-art methods in video restoration (e.g., STBN, DLoRAL, FPANet) and flicker removal (Flickerformer).

Quantitative Metrics: VDFP achieved the best results across all reference-based (SSIM, LPIPS, DISTS) and no-reference (MUSIQ, BRISQUE, DOVER, $E^*_{warp}$ ) metrics on the DeViD test set. Notably, it achieved a DOVER score of 0.6484 and an $E^*_{warp}$ of 10.5442, outperforming the second-best method (DLoRAL) significantly.
Visual Comparison: Qualitative results show that VDFP eliminates both thick and background banding nearly completely. In contrast, other methods either leave residual stripes, blur the video content, or fail to maintain temporal consistency when viewing the video as a whole.
Ablation Study: The study confirms that both the DFM module (for better training data simulation) and the CPP module (for precise prior injection) contribute incrementally to the final performance, with the full pipeline achieving the highest metrics.

5. Significance and Limitations

The paper claims that VDFP establishes a highly effective framework for real-world video deflickering, a task where previous models struggled due to degradation complexity and data lack. By leveraging a perception-guided generation approach, the method successfully balances aggressive artifact removal with the preservation of fine details.

Limitations:
The authors modestly note that the current DeViD dataset exclusively contains banding captured from LED matrices. Consequently, the model's performance on banding induced by other display technologies, such as LCD and OLED screens, remains an area for future investigation.

Societal Impact:
Positively, the work aims to help users restore videos and save precious memories that are otherwise degraded by screen-induced banding artifacts.