💻 computer science

PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow

The paper proposes PVRF, a unified framework for adverse weather removal that integrates zero-shot weather perception from frozen vision-language models with a velocity-constrained rectified flow to achieve high-fidelity, generalizable restoration of real-world images under heterogeneous degradations.

Original authors: Wei Dong, Han Zhou, Terry Ji, Guanhua Zhao, Shahab Asoodeh, Yulun Zhang, Guangtao Zhai, Jun Chen, Xiaohong Liu

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wei Dong, Han Zhou, Terry Ji, Guanhua Zhao, Shahab Asoodeh, Yulun Zhang, Guangtao Zhai, Jun Chen, Xiaohong Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to look through a dirty, foggy, rainy window to see a beautiful landscape. Your goal is to clean the glass so you can see the view clearly without losing any of the original details. This is what computer scientists call "Adverse Weather Removal."

However, real-world weather is messy. Sometimes it's just rain, sometimes it's snow, and often it's a confusing mix of rain, fog, and darkness all at once. Previous computer programs tried to fix these images, but they often made two mistakes:

The "Blurry Photo" Problem: They cleaned the image so much that it looked smooth and plastic, losing all the fine details (like the texture of a tree bark).
The "Wrong Guess" Problem: They tried to guess what the weather was (e.g., "It's raining!") and applied a single fix. If they guessed wrong, or if it was actually raining and foggy, the fix failed.

The paper introduces a new system called PVRF that solves these problems using a clever three-step strategy. Here is how it works, explained with simple analogies:

1. The "Smart Detective" (The AWR-QA Module)

Instead of forcing the computer to pick just one label like "Rain" or "Snow," PVRF uses a pre-trained "Smart Detective" (a frozen Vision-Language Model).

How it works: Imagine you have a detective who can look at a messy photo and answer specific questions like, "Is there rain?" (Yes/No), "Is it foggy?" (Yes/No), and "How dark is it?" (Good/Poor).
The Innovation: Instead of giving a simple "Yes" or "No," this detective gives a confidence score. It might say, "I'm 40% sure it's rain, 40% sure it's fog, and 20% sure it's snow."
Why it helps: This "soft" understanding allows the system to handle mixed weather perfectly. It doesn't have to choose just one; it knows the image is a blend, so it prepares a blended solution.

2. The "Custom Tailor" (The Conditioning Design)

Once the detective gives the confidence scores, the system needs to use them to clean the image. Think of the image restoration network as a tailor trying to fix a torn suit.

Attribute-Modulated Normalization (AMN): This is like the tailor checking the severity of the damage. If the "darkness" score is high, the tailor knows to be extra careful with the shadows. If the "blur" score is high, they know to focus on sharpness. It adjusts the "tightness" of the cleaning process based on how bad the damage is.
Weather-Weighted Adapters (WWA): This is like the tailor having a toolbox with different tools for different problems (a brush for dust, a solvent for oil, a needle for tears). Instead of picking just one tool, the tailor uses all the tools at once, but weighs them based on the detective's confidence scores. If the detective says "40% rain, 40% fog," the tailor uses 40% of the rain-tool and 40% of the fog-tool simultaneously. This prevents the "Wrong Guess" problem.

3. The "Steady Hand" (The Terminal-Consistent Flow)

The final step is the actual cleaning process. The paper uses a mathematical concept called "Rectified Flow," which is like a guided path from a messy image to a clean one.

The Problem: Usually, when you guide a path from "Messy" to "Clean," the end of the path (the clean image) can get wobbly or unstable, leading to weird artifacts.
The Solution: PVRF adds a "Steady Hand" mechanism. It anchors the process to a reliable starting point (the "Smart Detective's" best guess) and ensures that as the image gets cleaner, the movement becomes perfectly predictable and stable.
Adaptive Perturbation: Imagine you are shaking a box of marbles to settle them. If the box is only slightly messy, you give it a tiny shake. If it's a total disaster, you give it a big shake. PVRF automatically decides how hard to "shake" the image based on the detective's scores. This ensures the system doesn't over-correct mild problems or under-correct severe ones.

The Result

By combining a Smart Detective (to understand mixed weather), a Custom Tailor (to apply the right mix of fixes), and a Steady Hand (to ensure the final result is stable and realistic), PVRF produces images that are:

Sharper: They keep the fine details (like grass or brick textures) that other methods blur out.
More Realistic: They look like natural photos, not plastic paintings.
Versatile: They work great even when the weather is a confusing mix of rain, snow, fog, and darkness, or when the system encounters a type of weather it has never seen before.

In short, PVRF doesn't just guess what's wrong with a photo; it understands the nuance of the mess and applies a precise, balanced, and stable fix.

Technical Summary: PVRF

Problem Statement

Adverse Weather Removal (AWR) in real-world scenarios faces two primary challenges. First, real-world images often suffer from heterogeneous and mixed degradations (e.g., simultaneous rain, snow, and haze) that are difficult to infer and condition upon. Existing approaches relying on "hard" one-hot conditioning (predicting a single discrete weather label) are brittle; they struggle with ambiguous or combined degradations and are sensitive to cross-dataset appearance shifts. Second, there is a fundamental perception-distortion trade-off. Distortion-driven methods (minimizing MSE) often produce overly smooth results, while diffusion-based methods, though visually compelling, may lack fidelity. Achieving a restoration that is both faithful to the ground truth and photo-realistic remains a significant hurdle for all-in-one AWR models.

Methodology

The authors propose PVRF (Prior-modulated and Velocity-constrained Rectified Flow), a unified framework that integrates zero-shot soft weather perceptions with a velocity-constrained rectified-flow refinement process. The method operates in two main stages:

1. Zero-Shot Soft Perception Prior via AWR-QA

Instead of training a classifier to predict discrete weather labels, PVRF utilizes a frozen Vision-Language Model (VLM) to extract soft perception priors.

AWR-QA Module: The system queries the VLM using definition-augmented questions.
- Type Perception: For each weather type (e.g., "Is this image rainy?"), the VLM provides raw logits for "yes" and "no." These are converted into soft probabilities ( $P_{type}$ ) using a sigmoid-like function, retaining uncertainty (allowing multiple types to have non-negligible weights).
- Attribute Perception: The VLM is queried for low-level attributes (visibility, contrast, sharpness) to gauge degradation severity.
Conditioning Mechanisms: These soft priors modulate the restoration backbone (an AWR-specific network) via two components:
- Attribute-Modulated Normalization (AMN): Uses attribute scores to perform feature-wise affine transformations on layer-normalized features, providing continuous severity modulation.
- Weather-Weighted Adapter (WWA): Implements multiple parallel adapter branches (one per weather type). The soft type probabilities serve as mixture weights to aggregate the outputs of these branches, enabling the network to handle mixed degradations without hard switching.

2. Terminal-Consistent Residual Rectified Flow

To refine the initial anchor estimate ( $\mu$ ) produced by the perception-conditioned backbone, PVRF employs a rectified flow model that learns a transport map from a perturbed source distribution to the clean image distribution.

Posterior-Anchored Residual Space: The flow is defined in a residual space ( $r_t = Z_t - \mu$ ) where the source is $Z_0 = \mu + \delta\epsilon$ . This formulation ensures that the expected residual at the terminal state is zero by construction, simplifying the learning objective.
Perception-Adaptive Source Perturbation: Unlike standard methods using a fixed global perturbation scale, PVRF dynamically adjusts the perturbation magnitude ( $\delta$ ) based on the soft perceptions. A difficulty score is derived from type uncertainty and attribute severity, allowing mild cases to be perturbed less (preserving fidelity) and severe/ambiguous cases to be perturbed more (stabilizing learning).
Terminal-Consistent Velocity Parameterization: The velocity field is parameterized as $v_\phi(r_t, t) = t r_t + t(1-t)\Phi_\phi(\dots)$ . The term $t(1-t)$ vanishes at $t=1$ , guaranteeing that the velocity at the terminal state is exactly $r_1$ regardless of the network's prediction. This explicitly enforces terminal consistency, stabilizing learning near $t \to 1$ .

Key Contributions

Definition-Guided Zero-Shot Perceptions: The introduction of the AWR-QA module, which leverages frozen VLMs to generate soft, uncertainty-aware type and attribute priors without requiring task-specific fine-tuning.
Perception-Guided Modulation: The design of AMN and WWA modules that effectively integrate soft priors into the restoration backbone, enabling degradation-aware posterior estimation.
Terminal-Consistent Residual Flow: A novel rectified flow formulation featuring perception-adaptive source perturbation and a terminal-consistent velocity parameterization to balance photo-realism and fidelity.

Experimental Results

The authors evaluated PVRF on three settings:

Setting I (3-Task AWR): Dehazing, deraining, and desnowing. PVRF outperformed state-of-the-art (SOTA) CNN, Transformer, and Diffusion-based baselines (e.g., AdaIR, UniRestore, PromptIR) in both fidelity (PSNR, SSIM) and perceptual quality (LPIPS, FID, MUSIQ). Notably, it achieved a 1.14 dB average PSNR improvement over the best SOTA.
Setting II (4-Task AWR): Included low-light enhancement, showing consistent superiority.
Setting III (5-Task General Restoration): Included deblurring. PVRF maintained top performance across all metrics.
Generalization: The model demonstrated strong generalization on unseen datasets (e.g., HIDE, RealRain-1k) and, crucially, on combined degradations (e.g., Haze + Rain, Low-light + Snow). In these challenging mixed-degradation scenarios, PVRF significantly outperformed baselines in perceptual metrics (MUSIQ, MANIQA), where other methods often failed to handle multiple simultaneous degradations.

Significance

The paper claims that PVRF addresses the critical limitations of existing all-in-one AWR models by moving away from brittle, hard-conditioning paradigms toward a flexible, perception-driven approach. By leveraging the semantic understanding of frozen VLMs for soft priors and stabilizing the generative refinement process with terminal-consistent constraints, PVRF achieves a superior balance between distortion minimization and perceptual realism. The authors emphasize that the method's ability to handle mixed and unseen degradations without retraining makes it particularly suitable for real-world applications where weather conditions are complex and unpredictable.