Degradation-Aware and Structure-Preserving Diffusion for Real-World Image Super-Resolution

1. Problem Statement

Real-world image super-resolution (SR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs. While diffusion models have shown superior capability in generating high-frequency details compared to traditional regression-based methods, applying them to real-world scenarios remains challenging due to two main issues:

• Complex and Heterogeneous Degradations: Unlike synthetic benchmarks (e.g., bicubic downsampling), real-world images suffer from unknown, spatially varying degradations (blur, noise, compression artifacts). Existing diffusion methods often treat the LR input as a monolithic signal, failing to explicitly decouple degradation patterns from image content. This leads to restoration ambiguity, where models struggle to distinguish between genuine textures and artifacts, resulting in either over-smoothing or unnatural hallucinations.
• Structural Sensitivity to Noise: Standard diffusion training applies spatially uniform Gaussian noise to the latent space. This ignores the spatial heteroscedasticity of images: high-frequency structural regions (edges) are more sensitive to noise perturbations than flat areas. Uniform noise can prematurely erase critical geometric cues, forcing the model to "re-invent" geometry rather than refining existing structures.

2. Methodology

The authors propose DASP-SR, a framework that introduces two lightweight, add-on modules to a standard diffusion SR backbone (specifically based on a ControlNet/UNet architecture). These modules focus on explicit degradation conditioning and structure-aware noise modulation without fundamentally altering the diffusion formulation.

A. Degradation-aware Token Injection (DTI)

DTI addresses the lack of explicit degradation knowledge by extracting a compact, interpretable descriptor from the LR input and injecting it into the model's attention mechanism.

• Descriptor Extraction: A 6-dimensional vector $\mathbf{d}$ is computed from the grayscale LR image, capturing:
  1. Blur: Reciprocal of Laplacian variance.
  2. Noise: Mean absolute residual after pooling.
  3. JPEG Artifacts: Blocking artifacts from $8\times8$ boundaries.
  4. Edge Density: Fraction of pixels with high Sobel gradients.
  5. Brightness & Contrast: Global luminance statistics.
• Token Injection: The vector $\mathbf{d}$ is projected via a lightweight MLP (2 layers) into a degradation token $\mathbf{t}_{deg}$. In the dynamic variant, this token is further modulated by the diffusion timestep.
• Integration: The degradation token is appended to the image hidden states and injected into the image cross-attention branches of both the ControlNet and UNet. This allows the denoising network to condition its restoration process explicitly on the estimated degradation type and severity.

B. Spatially Asymmetric Noise Injection (SANI)

SANI addresses the structural fragility of edges during the training process by modulating the noise magnitude based on local image geometry.

• Edge Strength Map: An edge strength map $\mathbf{E}$ is computed from the LR input using Sobel filtering and normalized to $[0, 1]$.
• Noise Modulation: During the forward diffusion process, the standard Gaussian noise $\epsilon$ is modulated as:
  $$\epsilon' = \epsilon \odot (1 - \lambda \mathbf{E})$$
  where $\lambda$ is a hyperparameter (default 0.6).
• Effect: Regions with high edge strength receive significantly less noise amplitude (up to 60% reduction), preserving the geometric skeleton of the image. Smooth regions retain near-standard noise levels, allowing for stochastic texture generation.
• Inference Consistency: SANI is a training-only strategy. During inference, the standard reverse diffusion process is used without modification. The authors argue that the pre-trained diffusion backbone is robust enough to handle the slight distribution shift caused by this training-time modulation.

3. Key Contributions

1. Degradation-aware Token Injection (DTI): A novel mechanism that explicitly encodes physical degradation statistics (blur, noise, compression, etc.) into conditioning tokens, enabling the diffusion model to perform targeted restoration based on specific corruption types.
2. Spatially Asymmetric Noise Injection (SANI): A structure-aware training strategy that reduces noise perturbation in edge-rich regions during training, effectively anchoring the generative process to the original structural topology and preventing the destruction of geometric cues.
3. Lightweight Design: Both modules are minimal add-ons. DTI adds only ~0.2M parameters (0.01% of the total), and SANI introduces zero inference cost. They require only minor modifications to the conditioning pipeline of existing diffusion SR frameworks.

4. Experimental Results

The method was evaluated on DIV2K and RealSR benchmarks against state-of-the-art methods (e.g., Real-ESRGAN, SeeSR, StableSR, DiffBIR).

• Perceptual Quality: DASP-SR achieves State-of-the-Art (SOTA) or competitive results on no-reference perceptual metrics (NIQE, NRQM, PI, CLIP-IQA, MUSIQ). For instance, on RealSR, it achieved the best NIQE (3.709) and MUSIQ (69.597) scores.
• Perception-Distortion Trade-off: While the method yields lower PSNR/SSIM scores compared to regression-based baselines (a known trade-off for generative models that prioritize realism over pixel-perfect fidelity), it provides a superior balance for real-world applications where visual realism is paramount.
• Ablation Studies:
  • DTI alone: Improves texture realism but may struggle with structural coherence on complex degradations without SANI.
  • SANI alone: Significantly improves edge preservation and structural metrics (e.g., reducing NIQE by ~0.4 on RealSR).
  • Combined: The synergy of both modules yields the best overall perceptual performance, confirming that explicit degradation guidance and structural noise preservation are complementary.
• Efficiency: The inference time increase is negligible (+0.10s per image), and the parameter overhead is minimal.

5. Significance and Limitations

Significance:
This work bridges the gap between the generative power of diffusion models and the practical requirements of real-world SR. By explicitly modeling degradation statistics and protecting structural integrity during training, DASP-SR demonstrates that explicit, interpretable cues are crucial for guiding diffusion models in complex restoration tasks. It offers a practical path to improving diffusion SR without the need for massive architectural overhauls.

Limitations:

• Inference Cost: Like all multi-step diffusion models, it suffers from high inference latency compared to one-step or GAN-based methods.
• Descriptor Rigidity: The hand-crafted degradation descriptor (6D vector) may be insufficient for highly complex, mixed, or novel degradation types not covered by the predefined axes.
• Failure Cases: In extreme scenarios, the method can produce over-sharpened edges or hallucinated textures, as seen in the provided failure case analysis.

The authors suggest future work involving learned degradation representations and more efficient diffusion formulations to address these limitations.