Bird-SR: Bidirectional Reward-Guided Diffusion for Real-World Image Super-Resolution

1. Problem Statement

Real-world image super-resolution (Real-ISR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs with unknown and complex degradations. While diffusion models have shown superior performance in synthesizing intricate details compared to traditional supervised methods, they face three critical challenges when applied to real-world scenarios:

• Distribution Shift: Models trained on synthetic LR-HR pairs (generated via simple degradation models like bicubic downsampling) fail to generalize to real-world LR images. Real-world degradations are far more diverse, leading to a significant domain gap where models produce over-smoothed or unrealistic textures.
• Reward Hacking: When applying reward-based preference optimization (ReFL) directly to real-world data (which lacks ground truth), models tend to "hack" the reward signal. This results in visually implausible artifacts or hallucinations that score well on the reward metric but degrade structural fidelity.
• Trajectory-Level Trade-off: Diffusion processes have distinct stages: early steps define global structure, while later steps refine high-frequency textures. Blindly applying reward optimization across all timesteps can amplify perceptual gains at the cost of structural consistency, or vice versa.

2. Methodology: Bird-SR

The authors propose Bird-SR, a bidirectional reward-guided diffusion framework that jointly optimizes the forward and reverse diffusion processes using both synthetic paired data and unpaired real-world LR data. The core mechanism involves Reward Feedback Learning (ReFL) tailored to the specific characteristics of the diffusion trajectory.

A. Forward Optimization with Relative Reward (Synthetic Data)

For synthetic LR-HR pairs, the model utilizes a noise injection strategy rather than full reverse sampling:

• Closed-Form Single-Step: Instead of multi-step denoising, noise is injected into clean HR images at a random timestep $t$. The model reconstructs the image using a closed-form single-step interpolation formula.
• Relative Reward: To prevent reward hacking, the reward is formulated as a relative advantage between the ground truth ($x_0$) and the model prediction ($\hat{x}_0$). The objective maximizes the difference: $r(x_0) - r(\hat{x}_0)$.
• Benefit: This approach provides precise structural supervision and significantly stabilizes optimization by avoiding long backpropagation chains.

B. Reverse Trajectory Reward Feedback (Real-World Data)

For unpaired real-world LR images, the model optimizes the reverse diffusion process:

• Late-Stage Optimization: Reward signals are applied only at the final timestep of the reverse trajectory. This aligns with the diffusion mechanism where early steps handle global semantics (already well-aligned) and later steps handle high-frequency perceptual details.
• Semantic Alignment Constraint: To mitigate reward hacking in the absence of ground truth, a semantic alignment loss is introduced. The model's output is constrained to match the high-level semantic features (extracted via DINO) of a reference model's prediction. This ensures structural consistency while allowing pixel-level perceptual refinement.

C. Dynamic Fidelity-Perception Weighting

To balance structural preservation and perceptual enhancement, the authors introduce a dynamic weighting strategy ($\lambda(t)$) that varies across diffusion timesteps:

• Early Timesteps: Higher weight is assigned to distortion loss (structural fidelity) to stabilize the global structure.
• Later Timesteps: The weight shifts progressively toward the reward-based objective to enhance high-frequency details and textures.
• This dynamic schedule prevents the model from over-optimizing for texture at the expense of structure during coarse reconstruction.

3. Key Contributions

1. Bidirectional Framework: A novel architecture that jointly optimizes forward (synthetic) and reverse (real-world) processes, effectively bridging the distribution gap between synthetic training data and real-world inputs.
2. Stable Reward Optimization:
   • For synthetic data: A noise-injection strategy with relative rewards ensures stability and prevents hacking.
   • For real data: Semantic alignment constraints guide the reward feedback, preserving structural integrity.
3. Trajectory-Aware Weighting: A dynamic mechanism that adaptively balances distortion and perception losses based on the diffusion timestep, addressing the inherent trade-off in diffusion models.
4. State-of-the-Art Performance: The method achieves superior perceptual quality on real-world benchmarks without sacrificing structural consistency, outperforming existing diffusion-based SR methods.

4. Experimental Results

The authors evaluated Bird-SR on four real-world benchmarks: DrealSR, RealSR, RealLR200, and RealLQ250.

• Quantitative Performance: Bird-SR consistently outperformed state-of-the-art methods (including StableSR, DiffBIR, SUPIR, SeeSR, and DiT4SR) on non-reference perceptual metrics such as MUSIQ, MANIQA, ClipIQA, and LIQE. It also maintained competitive LPIPS scores, indicating good structural fidelity.
• Qualitative Results: Visual comparisons demonstrated that Bird-SR produces more realistic textures, preserves fine structures (e.g., text, edges), and effectively suppresses artifacts compared to baselines.
• User Study: In a study with 20 volunteers, Bird-SR was preferred over competing methods in 95.2% of cases for realism and 75.2% for fidelity.
• Ablation Studies:
  • Removing the bidirectional approach (using only forward or only reverse) led to either poor adaptation to real data or unstable optimization.
  • Removing semantic alignment or structural constraints resulted in severe reward hacking and structural distortion.
  • The dynamic weighting strategy was crucial for balancing the trade-off between structure and texture.

5. Significance and Impact

Bird-SR represents a significant advancement in Real-World Image Super-Resolution by addressing the fundamental limitation of distribution shift in diffusion models.

• Robustness: By leveraging both synthetic and real data through a bidirectional reward framework, it creates a model that is robust to unknown degradations.
• Safety in Optimization: The introduction of relative rewards and semantic constraints provides a solution to the "reward hacking" problem, making reward-guided diffusion safer and more reliable for practical applications.
• Generalizability: The framework is model-agnostic and was successfully demonstrated on both large-scale models (DiT4SR based on Stable Diffusion 3.5) and lightweight models (ResShift), suggesting broad applicability.
• Applications: The method has potential applications in remote sensing, medical imaging, and low-bandwidth transmission where high-quality, realistic detail recovery is critical.

In conclusion, Bird-SR successfully aligns diffusion-based super-resolution with human perceptual preferences while maintaining structural fidelity, offering a promising direction for next-generation image restoration systems.