Technical Summary: Self-supervised Dynamic Heterogeneous Degradation Modeling for Unified Zero-Shot Image Restoration

1. Problem Statement

Zero-shot image restoration aims to recover clean images from diverse, unpredictable degradations without task-specific retraining. While existing approaches offer flexibility, they face significant limitations:

• Insufficient Degradation Priors: Current methods often treat degradations as black-box disturbances, relying on implicit feature learning or stacked layers. This lacks physically consistent priors, leading to weak representation capabilities.
• High Training and Sampling Costs: The lack of explicit physical prompts forces diffusion models to infer degradations through step-by-step stochastic sampling, imposing a heavy training burden and high sampling costs.
• Suboptimal Convergence: Fixed inference trajectories often collapse into local minima under complex or mixed corruptions (e.g., low-light combined with noise and haze), failing to achieve globally optimal solutions.
• Generalization Limits: Task-specific models lack generalization across different degradation types, while "all-in-one" models often rely on deep networks to implicitly capture degradation characteristics, constraining representation capacity.

2. Methodology: UP-ZeroIR

The authors propose UP-ZeroIR (Unified Physical Zero-shot Image Restoration), a framework that shifts from implicit feature learning to explicit degradation distribution modeling. The core insight is that heterogeneous degradations can be reparameterized into a minimal set of physically coherent parameters within the latent diffusion space.

The framework consists of four key components:

A. Physically Coherent Degradation Modeling (PCDM)

Instead of learning high-dimensional degradation features, UP-ZeroIR models heterogeneous degradations as a unified, homogeneous distribution in the latent space.

• Parametrization: The authors employ the Generalized Gaussian Distribution (GGD) to approximate the latent distribution of degraded images. The distribution is defined by two low-dimensional sufficient statistics: scale ($\sigma$) and shape ($\alpha$).
• Homogeneous Translation: Empirical analysis shows that diverse degradations (noise, haze, low light) manifest as systematic shifts in pixel-domain distributions. PCDM maps these heterogeneous degradations into a single, physically grounded parametric representation in the latent diffusion space.
• Implementation: A lightweight network maps the predicted clean latent representation ($\hat{z}_0$) to a physically plausible degraded observation $\phi(\hat{x}_0)$ using the estimated GGD parameters.

B. Degradation-Aware Posterior Sampling

The framework utilizes a pre-trained Latent Diffusion Model (LDM) but conditions the sampling process on the explicit degradation cues provided by PCDM.

• Likelihood Gradient: The posterior sampling direction is guided by a correction term derived from the likelihood gradient $\nabla_{z_t} \log p(x_{lq} | z_t)$.
• Explicit Control: By estimating the degradation likelihood $p(x_{lq} | \hat{z}_0)$, the sampler performs degradation-aware updates rather than unguided diffusion, steering the process toward physically consistent restoration.

C. Degradation Alignment Loss (DegLoss)

To ensure the estimated latent distribution aligns with the physical degradation model, the authors introduce a novel loss function.

• Kullback-Leibler Divergence (KLD): The loss measures the divergence between the distribution of the synthesized degraded image ($\phi(\hat{x}_0)$) and the actual input degraded image ($x_{lq}$) in the latent space.
• Total Objective: The optimization combines DegLoss with standard reconstruction losses (MSE, Perceptual, Adversarial) and a quality term ($Q$) that evaluates luminance and chrominance to ensure perceptual fidelity.

D. Dynamic Quality-Refinement (DQR) Strategy

To address the issue of suboptimal convergence under complex corruptions, UP-ZeroIR introduces an adaptive inference strategy.

• Self-Assessment: During inference, a pre-trained no-reference image quality assessment model (Arniqa) evaluates the restoration quality at specific intervals.
• Adaptive Trajectory: If the quality score does not improve sufficiently, the strategy adaptively adjusts the sampling trajectory. Specifically, if the process stalls, noise is reintroduced to the current latent representation to re-initiate posterior sampling, allowing the model to escape local minima and move toward a globally optimal solution.

3. Key Contributions

1. Unified Physical Parameterization: The paper proposes a unified physical parameterization of heterogeneous degradations using a compact set of interpretable GGD parameters. This enables explicit control of the latent-space distribution for zero-shot restoration, moving away from implicit feature learning.
2. Dynamic Quality-Refinement: A self-assessed, dynamic strategy is introduced to adaptively adjust the diffusion trajectory. This ensures global optimality with minimal inference costs by escaping local minima during complex corruptions.
3. State-of-the-Art Performance: Extensive experiments demonstrate that UP-ZeroIR consistently outperforms prior methods (both task-specific and zero-shot posterior sampling methods) on single degradations (denoising, dehazing, low-light enhancement) and mixed degradations.

4. Experimental Results

The method was evaluated on three primary tasks using standard datasets:

• Low-Light Enhancement (LOLv1, LOLv2): UP-ZeroIR achieved the highest PSNR and SSIM, outperforming the second-best zero-shot method (LD-RPS) by 0.95 dB and 0.98 dB respectively. It also surpassed supervised all-in-one methods in several metrics.
• Image Dehazing (HSTS subset of RESIDE): The method improved PSNR by 1.03 dB over the second-best zero-shot method, effectively removing haze while preserving fine details and color fidelity.
• Image Denoising (Kodak24): UP-ZeroIR achieved a 0.85 dB improvement in PSNR over the second-best method, successfully suppressing noise while retaining structural details like textures and edges.
• Mixed Degradations: In scenarios combining low-light, noise, and haze, the method demonstrated superior robustness compared to other posterior sampling methods, which often suffered performance drops with entangled degradations.

5. Significance and Claims

The paper claims that UP-ZeroIR represents a shift toward physically interpretable image restoration. By explicitly modeling degradation as a distribution-level alignment problem rather than a feature learning task, the method:

• Reduces Uncertainty: It mitigates the uncertainty inherent in fixed-trajectory diffusion by enabling controllable, degradation-aware solution exploration.
• Generalizes Effectively: The unified distribution model allows the framework to generalize to unseen and mixed degradations without task-specific retraining.
• Improves Efficiency: The dynamic quality-refinement strategy reduces inference time by approximately 30% compared to fixed-trajectory baselines while improving convergence.

The authors position this work as the first exploration of a physically interpretable restoration paradigm via explicit distribution-level controllability, offering a practical solution for zero-shot deployment in real-world scenarios where degradation types are unknown and variable.