Provably Contractive and High-Quality Denoisers for Convergent Restoration

1. Problem Statement

Image restoration (denoising, deblurring, super-resolution) is critical in fields like medical imaging and surveillance. While deep learning models (e.g., DnCNN, Restormer) have achieved state-of-the-art (SOTA) performance, they suffer from two major limitations:

• Lack of Stability: They are often sensitive to minor input perturbations (e.g., JPEG compression, chroma subsampling, sensor noise), leading to amplified artifacts and degraded quality.
• Unproven Convergence in PnP: In Plug-and-Play (PnP) algorithms, which use learned denoisers as priors for iterative restoration, convergence guarantees rely on the denoiser being non-expansive (Lipschitz constant $L \le 1$) or contractive ($L < 1$). Most SOTA denoisers lack these strict mathematical guarantees, causing PnP iterations to diverge or behave unpredictably.

The core challenge is: Can we design a denoiser that is provably contractive (ensuring stability and convergence) while maintaining reconstruction quality comparable to unconstrained SOTA models?

2. Methodology

The authors propose a novel architecture that constructs a provably contractive denoiser by composing specific layers with strict Lipschitz constraints.

A. Theoretical Foundation

• Contractivity: A function $f$ is contractive if $\|f(x) - f(y)\| \le L\|x - y\|$ with $L < 1$. This ensures that input perturbations of size $\epsilon$ result in output changes of at most $\epsilon$.
• Composition: The Lipschitz constant of a composed network is the product of the Lipschitz constants of its layers. If every layer is contractive, the entire network is contractive.

B. Architecture Design

The network is built by stacking $M$ identical Contractive Layers ($T$), each composed of three main components:

1. Gradient and Prox-Wavelet Block (FBS Step):

   • Based on the Forward-Backward Splitting (FBS) algorithm for solving inverse problems with a wavelet-based regularizer.
   • It performs a gradient step followed by a proximal operator that applies soft-thresholding only to high-frequency wavelet coefficients.
   • Proof: This block is mathematically proven to be $(1-\alpha)$-contractive (where $\alpha \in (0,1)$ is a learnable step size).

2. Scaled Convolutional Layer:

   • Standard convolutions are not inherently contractive. The authors enforce contractivity by normalizing the convolution kernel $K$.
   • The operator norm $\|V_K\|_{op}$ is computed exactly in the frequency domain (via 2D FFT).
   • The kernel is scaled by $s(K) + \epsilon$ (where $s(K)$ is the spectral norm) to ensure the Lipschitz constant is strictly less than 1.

3. Layer Composition:

   • The final layer $T$ combines the FBS block and the scaled convolution, scaled by a factor $1/((1-\alpha) + \epsilon)$.
   • Theorem: The composition of these components guarantees the layer is contractive. Stacking $M$ such layers results in a globally contractive network.

C. Implementation Details

• Wavelets: Uses orthogonal wavelet transforms (Haar, Daubechies-4, Symlet-4) cyclically across layers.
• Training: Trained on BSD500 patches using Adam optimizer. Constraints (positive thresholds, step sizes in $(0,1)$) are enforced via clipping and softplus functions.
• Inference: Uses a patch-wise strategy with overlapping windows and Tukey-window aggregation to handle arbitrary image sizes without blocking artifacts.

3. Key Contributions

1. First High-Quality Contractive Denoiser: The paper presents the first method to demonstrate that a provably contractive denoiser can achieve performance competitive with unconstrained SOTA models (like Restormer and DnCNN), closing the accuracy gap significantly.
2. Rigorous Stability Guarantees: The model provides a mathematical guarantee that input perturbations (e.g., compression artifacts) will not be amplified, unlike standard deep networks.
3. Convergent PnP Restoration: When used as a prior in Plug-and-Play algorithms (PnP-FBS), the denoiser guarantees convergence to a fixed point, whereas unconstrained denoisers often diverge.
4. Architectural Innovation: Introduces a specific composition of proximal wavelet blocks and spectrally normalized convolutions that balances expressivity with strict Lipschitz constraints.

4. Experimental Results

A. Robustness to Perturbations

• JPEG & Chroma Subsampling: Under mild JPEG compression (90% quality) and chroma subsampling, the proposed model maintains stable PSNR. In contrast, Restormer and DnCNN show significant performance drops and visual artifacts (Figures 1 & 2).
• Adversarial Perturbations: The model exhibits significantly smaller output variations under adversarial noise compared to unconstrained baselines.

B. Image Denoising (Standalone)

• Performance: On standard benchmarks (BSD68, Kodak24, Urban100), the model achieves the highest PSNR/SSIM among all provably 1-Lipschitz/contractive models.
• Gap to SOTA: The gap between the proposed contractive model and the best unconstrained model (Restormer) is small (typically 0.8–1.5 dB PSNR and 0.02–0.06 SSIM). This challenges the assumption that strict Lipschitz constraints inherently degrade quality.
• Visual Quality: Unlike other 1-Lipschitz models that tend to oversmooth, the proposed model preserves fine textures and edges comparable to DnCNN and Restormer.

C. Plug-and-Play (PnP) Restoration

• Convergence: In PnP-FBS iterations for deblurring and super-resolution, the proposed denoiser ensures monotone convergence. Unconstrained models (Restormer, DnCNN) were observed to diverge.
• Restoration Quality:
  • Deblurring: Outperforms 1-Lipschitz baselines and often surpasses unconstrained models in PSNR/SSIM, recovering sharper edges and text without ringing artifacts.
  • Super-Resolution: Achieves SOTA results among contractive models and competitive results against unconstrained models across various blur kernels (Gaussian, anisotropic, defocus).

5. Significance and Conclusion

This work bridges the gap between theoretical stability and practical performance in image restoration.

• Paradigm Shift: It challenges the literature's assumption that enforcing strict Lipschitz constraints (for stability) necessarily sacrifices reconstruction quality.
• Reliability: It enables the deployment of deep learning-based restoration in safety-critical domains (defense, medical) where robustness to input shifts and algorithmic convergence are mandatory.
• Future Direction: The authors suggest extending this approach to end-to-end contractive restoration networks, moving the field toward verifiable and stable vision models.

Code Availability: The authors have released their code and pretrained models at https://github.com/SHUBHI1553/Contractive-Denoisers.