Beyond the Ground Truth: Enhanced Supervision for Image Restoration

1. Problem Statement

Deep learning-based image restoration (e.g., deblurring, denoising) has achieved significant success but faces a critical bottleneck in real-world applications: suboptimal ground truth (GT) images.

• The Limitation: In real-world datasets, obtaining perfect reference images is practically impossible. For example, deblurring datasets often use video frames with slight camera shake, and denoising datasets often use averaged noisy captures, resulting in inherently blurred or imperfect GTs.
• The Consequence: Models trained on these imperfect GTs are forced to learn the imperfections of the reference, limiting their ability to achieve high perceptual quality and preventing them from recovering details that are lost in the "flawed" reference.
• The Challenge: Simply applying generative models (like diffusion) to enhance GTs often leads to hallucinated artifacts (details that look real but are semantically incorrect), which compromises the fidelity of the restoration.

2. Methodology

The authors propose a two-stage framework: Supervision Enhancement and Output Refinement.

A. Supervision Enhancement Framework

The goal is to generate a higher-quality "Enhanced GT" from existing suboptimal GTs without introducing semantic hallucinations.

1. Super-Resolution (SR) Variants:
   • An existing suboptimal GT ($I_{GT}^0$) is upsampled using bicubic interpolation.
   • A one-step diffusion-based Image Super-Resolution (ISR) model (OSEDiff) is applied to generate perceptually enhanced variants ($I_{GT}^i$). These variants contain richer high-frequency details but may suffer from semantic distortions.
2. Frequency-Domain Mixup:
   • Instead of fusing images in the spatial domain (which causes artifacts), the method fuses them in the frequency domain.
   • Conditional Frequency Mask Generator: A network $g$ takes the original GT and a condition parameter $\lambda$ (controlling enhancement intensity) to predict coefficients for a set of predefined ring-shaped Gaussian basis masks ($R_b$).
   • These masks are designed to control specific frequency bands (low to high) smoothly.
   • Adaptive Fusion: The generator produces adaptive masks ($M_i$) that selectively preserve low-frequency semantic structures from the original GT while incorporating high-frequency perceptual details from the SR variants.
   • Formula: The enhanced GT ($\hat{I}_{GT}$) is computed as:
     $$\hat{I}_{GT} = \mathcal{F}^{-1}\left( \sum_{i=0}^{N} M_i \odot \mathcal{F}(I_{GT}^i) \right)$$
     where $\mathcal{F}$ is the Fourier transform and $\odot$ is element-wise multiplication.
3. Optimization: The mask generator is trained using a composite loss balancing reconstruction loss (to maintain semantic consistency with the original GT) and perceptual loss (using no-reference IQA metrics like MUSIQ, MANIQA, TOPIQ).

B. Output Refinement Network (ORNet)

• Architecture: A lightweight, model-agnostic network (based on NAFBlocks and U-Net structure) designed to be plugged into any pre-trained restoration model.
• Training Strategy: ORNet is trained to map the output of a pre-trained restoration model (which approximates the original, imperfect GT) to the newly generated Enhanced GT.
• Benefit: This allows existing state-of-the-art models to be upgraded without retraining the entire backbone, effectively "refining" their output to match the higher-quality supervision.

3. Key Contributions

1. Supervision Enhancement via Frequency Mixup: A novel framework that enhances ground truth images by adaptively fusing frequency components from original and super-resolved variants. This preserves semantic fidelity while enriching perceptual details, avoiding the hallucination issues common in pure generative approaches.
2. Lightweight, Model-Agnostic Refinement: The introduction of ORNet, a plug-and-play module that improves the output of arbitrary pre-trained restoration models without architectural changes or retraining of the base model.
3. Robustness and Generalization: The approach demonstrates strong performance in Out-of-Distribution (OOD) scenarios (e.g., unseen blur or noise types) and across different tasks (deblurring, denoising, low-light enhancement).
4. Comprehensive Validation: Extensive quantitative experiments, qualitative visualizations, and user studies confirming that the enhanced GTs and refined outputs are preferred by humans and score higher on perceptual metrics.

4. Experimental Results

The method was evaluated on standard benchmarks (GoPro for deblurring, SIDD for denoising) and OOD settings.

• Quantitative Performance:
  • Perceptual Metrics: ORNet significantly outperforms state-of-the-art baselines (e.g., Restormer, NAFNet, FFTFormer) on no-reference metrics. For example, on GoPro deblurring, MUSIQ scores jumped from ~46 (FFTFormer) to 64.57 (+ORNet).
  • Fidelity: In OOD settings, the method maintains high PSNR/SSIM against the original GT while drastically improving perceptual quality, proving it removes residual degradation without destroying semantic structure.
  • Hallucination Check: Quantitative analysis using OCR and Face Recognition showed that Enhanced GTs maintain 100% accuracy in identity verification and improved text recognition, proving the absence of semantic hallucinations.
• Qualitative Results: Visual comparisons show sharper text, cleaner textures, and better structural details compared to baselines.
• Efficiency: ORNet is highly efficient (4.5M parameters, 20G MACs), adding negligible computational overhead compared to heavy restoration backbones.
• User Studies: Participants consistently preferred the Enhanced GTs and ORNet outputs over baselines in blind preference tests.

5. Significance

This paper addresses a fundamental, often overlooked limitation in image restoration: the quality of the training target. By proposing a method to "fix" the ground truth itself through frequency-domain mixing, the authors enable existing models to surpass their previous performance ceilings.

• Practical Impact: The "plug-and-play" nature of ORNet means it can be immediately applied to improve a wide range of existing restoration models without the cost of retraining large architectures.
• Theoretical Insight: It demonstrates that frequency-domain fusion is a superior strategy for balancing perceptual enhancement with semantic fidelity compared to spatial-domain mixing or pure generative hallucination.
• Future Direction: The work suggests that future progress in real-world restoration depends not just on better network architectures, but on better supervision signals derived from the data itself.