Quaternion Wavelet-Conditioned Diffusion Models for Image Super-Resolution

Technical Summary: ResQu – Quaternion Wavelet-Conditioned Diffusion Models for Image Super-Resolution

Problem Statement
Image Super-Resolution (SR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs, a task critical for downstream computer vision applications such as object detection and segmentation. While deep learning has advanced SR significantly, achieving reconstructions that balance perceptual quality with structural fidelity—particularly at high upscaling factors—remains challenging. Existing diffusion-based SR methods often struggle to preserve fine-grained details and realistic textures without introducing artifacts or blurring. Furthermore, many approaches rely on predefined degradation models that limit generalizability to real-world scenarios.

Methodology: The ResQu Framework
The authors propose ResQu, a novel SR framework that integrates a quaternion wavelet preprocessing pipeline with latent diffusion models (LDMs). The architecture leverages the generative priors of pre-trained foundation models, specifically Stable Diffusion (SD), while introducing a specialized conditioning mechanism.

1. Quaternion Wavelet Transform (QWT) and QUAVE:
   Unlike standard Discrete Wavelet Transforms (DWT), which lack translation invariance and phase information, ResQu utilizes the Quaternion Wavelet Transform. QWT decomposes images into four sub-bands (scaling and three directional wavelets), yielding 16 real sub-bands that capture rich multidimensional representations of both low-frequency approximations and high-frequency details. To avoid redundancy, the framework employs QUAVE, a learning-based approach that extracts features from the most informative QWT sub-bands.

2. Quaternion Wavelet- and Time-Aware Encoder:
   The core innovation is a novel encoder that processes LR inputs to generate quaternion wavelet embeddings. These embeddings are dynamically integrated into the diffusion process alongside timestep embeddings.

   • Multi-Scale Conditioning: The encoder injects these features into the intermediate feature maps of the U-Net backbone via Spatial Feature Transformations (SFT).
   • Adaptive Guidance: The encoder is "time-aware," meaning it adjusts its conditioning strength based on the diffusion timestep. In early denoising stages (low Signal-to-Noise Ratio), it provides strong structural guidance to preserve integrity. As the latent representation refines (higher SNR), the encoder reduces its influence, allowing the diffusion model to focus on generating fine-grained textures.

3. Training Strategy:
   ResQu fine-tunes a pre-trained Stable Diffusion model rather than training from scratch. The VAE encoder and decoder are kept frozen, and the U-Net backbone is adapted with the new quaternion wavelet- and time-aware encoder. This approach significantly reduces computational requirements while leveraging the extensive generative priors learned from vast datasets.

Key Contributions

1. Novel Integration: The introduction of a latent diffusion model for image SR that uniquely integrates quaternion wavelet features with generative priors.
2. Specialized Encoder: The proposal of a quaternion wavelet- and time-aware encoder that enhances the conditioning process at multiple scales during denoising, improving the performance of pre-trained foundation models like Stable Diffusion.
3. Performance Validation: Extensive experiments demonstrating that ResQu outperforms existing methods in both perceptual quality (LPIPS, FID) and quantitative metrics (PSNR, SSIM).
4. Ablation and Domain Analysis: Rigorous ablation studies on architecture design (e.g., Controllable Feature Wrapping) and zero-shot evaluations on domain-specific datasets (ShipSpotting), proving the method's robustness and versatility.

Experimental Results
The authors evaluated ResQu on synthetic (DIV2K) and real-world (RealSR, DRealSR) benchmarks, as well as a domain-specific maritime dataset (ShipSpotting).

• Quantitative Performance: On the DIV2K validation set, ResQu achieved a PSNR of 25.21 and SSIM of 0.645, outperforming baselines like StableSR and LDM. On RealSR and DRealSR, the model demonstrated strong generalization, achieving the highest PSNR (26.45 and 29.69, respectively) and competitive SSIM scores.
• Qualitative Analysis: Visual comparisons highlight ResQu's ability to reconstruct intricate structural details (e.g., facial wrinkles, fur textures) and text elements without the blurring or aliasing artifacts common in other methods. The quaternion-based conditioning effectively preserves sharp edges and natural stochastic patterns.
• Zero-Shot Capability: In a zero-shot evaluation on the ShipSpotting dataset (without retraining), ResQu achieved performance comparable to fully trained specialized models, demonstrating its ability to generalize to unseen domains and handle complex maritime textures.
• Ablation Findings: Studies on sampling steps revealed a trade-off where fewer steps improved efficiency and pixel-wise metrics (PSNR/SSIM) but slightly reduced perceptual quality (LPIPS). Additionally, a custom-trained Controllable Feature Wrapping (CFW) module outperformed the pre-trained StableSR version in preserving structural fidelity.

Significance and Claims
The paper claims that ResQu establishes a new benchmark in image super-resolution by effectively synthesizing generative priors with advanced signal processing techniques. The authors assert that their approach successfully addresses the challenge of balancing structural fidelity with perceptual realism, a common limitation in current diffusion-based SR methods. By leveraging quaternion wavelet embeddings, the framework captures essential phase relationships and multidimensional features, enabling the recovery of fine-grained details and robust performance across diverse content types, from human portraits to complex natural textures and text. The work contributes a novel architectural paradigm that integrates quaternion wavelet features with diffusion models, offering a versatile solution for real-world image enhancement tasks.