💻 computer science

GDPO-SR: Group Direct Preference Optimization for One-Step Generative Image Super-Resolution

This paper proposes GDPO-SR, a novel framework that integrates Group Direct Preference Optimization with a noise-aware one-step diffusion model and an attribute-aware reward function to enhance the performance of one-step generative image super-resolution by addressing limitations in stochasticity, sample diversity, and local detail preservation.

Original authors: Qiaosi Yi, Shuai Li, Rongyuan Wu, Lingchen Sun, Zhengqiang Zhang, Lei Zhang

Published 2026-03-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qiaosi Yi, Shuai Li, Rongyuan Wu, Lingchen Sun, Zhengqiang Zhang, Lei Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Fixing Blurry Photos in a Flash

Imagine you have an old, blurry, low-resolution photo of your grandmother. You want to turn it into a crisp, high-definition masterpiece. This is called Image Super-Resolution (ISR).

For a long time, computers were great at making photos sharper but not necessarily more realistic. Recently, AI models (like diffusion models) learned to "hallucinate" or imagine missing details (like the texture of skin or the weave of a shirt) to make photos look amazing.

However, there's a catch:

The Slow Way: The best AI models take many steps to "denoise" an image, like slowly chiseling a statue out of a block of marble. This takes a long time.
The Fast Way: Newer "one-step" models try to do the whole job in a single instant. But because they are so fast, they are often too rigid. They act like a photocopier: if you give them the same blurry photo, they always produce the exact same result, even if that result is slightly blurry or missing details. They lack imagination.

The Goal of this paper: Can we make the "Fast Way" (one-step) as creative and high-quality as the "Slow Way," without slowing it down?

The Problem: The "Boring" Artist

The authors realized that the current fast AI models are like a boring artist who refuses to take risks.

The Issue: If you ask this artist to paint a tree, they will paint the exact same tree every time, even if you ask them to try again.
Why this matters: To teach an AI to be better, you need to show it examples of "good" vs. "bad" outcomes so it can learn what humans prefer. But if the AI always produces the same outcome, you can't show it different variations to learn from. It's like trying to teach a chef to cook better by only letting them make the exact same sandwich every time.

The Solution: GDPO-SR (The "Group" Approach)

The authors propose a new training method called GDPO-SR. Think of it as a three-part recipe to turn that boring artist into a creative genius who works in a flash.

1. The "Noise Injector" (NAOSD)

First, they give the artist a magic shaker of glitter (noise).

How it works: Before the AI tries to fix the photo, they inject a tiny bit of random "noise" (like shaking a snow globe).
The Analogy: Imagine asking an artist to draw a tree, but you shake the table slightly differently every time. Sometimes the shake makes the tree look lush and green; other times, it looks a bit wilted.
The Trick: They use a special strategy called "Unequal Timesteps." They shake the table hard (high noise) to create variety, but then they clean up the mess gently (low noise) to make sure the final picture doesn't get ruined. This creates a group of different versions of the same photo from the same input.

2. The "Taste Test" (Attribute-Aware Reward)

Now that the AI has generated a group of different photos (some good, some bad), they need to judge them.

The Problem: A simple judge might say, "This one looks sharp," but miss that the texture looks fake. Or they might say, "This looks artistic," but miss that the building is crooked.
The Solution: They created a Smart Judge (Reward Function).
The Analogy: Imagine a food critic who tastes a dish.
- If the dish is a smooth soup (a smooth area in an image, like a blue sky), the critic cares mostly about fidelity (is it the right color?).
- If the dish is a chunky stew (a detailed area, like a brick wall or fur), the critic cares mostly about perception (does it look tasty and real?).
- The AI's judge automatically switches its criteria based on what part of the image it's looking at. It gives a high score to the "best" version in the group and a low score to the "worst" one.

3. The "Group Huddle" (GDPO Strategy)

Finally, the AI learns from this group.

Old Way (DPO): The AI was shown just one good photo and one bad photo. It learned, "Okay, I like the good one."
New Way (GDPO): The AI sees a whole group of photos (a "huddle"). It compares them all against each other.
The Analogy: Instead of just showing a student one "A" paper and one "F" paper, the teacher shows them a whole stack of essays. The teacher says, "Look at this one, it's the best. Look at that one, it's the worst. Look at the others in between. Now, try to write like the best one."
By comparing the whole group, the AI learns much faster and more effectively which details matter.

The Results: Fast, Sharp, and Real

The paper tested this new method (GDPO-SR) and found:

It's Fast: It still takes only one step to generate the image (like a flash photo), so it's super quick.
It's High Quality: It produces images that are sharper and have more realistic details (like brick textures or leaf veins) than previous fast methods.
It's Balanced: It doesn't just make things look "cool" (which often makes them look fake); it keeps the image true to the original while adding the missing details.

Summary in One Sentence

The authors taught a fast AI to "imagine" different versions of a blurry photo, used a smart judge to pick the best details from those versions, and then taught the AI to always aim for that best version—resulting in instant, high-definition, realistic photos.

1. Problem Statement

The paper addresses the limitations of One-Step Generative Image Super-Resolution (ISR) and the challenges of applying Reinforcement Learning (RL) to this specific domain.

Deterministic Nature of One-Step Models: Existing one-step ISR methods (e.g., OSEDiff, InvSR) directly map Low-Resolution (LR) inputs to High-Resolution (HR) outputs without random noise initialization. This deterministic behavior prevents the generation of diverse outputs for the same input, which is a prerequisite for preference-based RL optimization (like DPO or GRPO).
Limitations of Existing RL Methods:
- Direct Preference Optimization (DPO): Typically relies on offline-generated positive/negative pairs, limiting sample diversity and data efficiency.
- Group Relative Policy Optimization (GRPO): Generates multiple samples online but calculates the likelihood over the entire image. This global approach often ignores local details (textures, edges) crucial for ISR, potentially degrading visual quality.
The Trade-off: Multi-step diffusion models offer high generative capacity but suffer from high computational costs and hallucination. One-step models are efficient but lack generative diversity and detail.

2. Methodology

The authors propose GDPO-SR, a framework integrating Group Direct Preference Optimization (GDPO) with a novel Noise-Aware One-Step Diffusion (NAOSD) model.

A. Noise-Aware One-Step Diffusion (NAOSD)

To enable RL, the model must generate diverse outputs from a single LR input.

Mechanism: The model injects controllable Gaussian noise ( $\epsilon$ ) into the latent features of the LR image before the one-step denoising process.
Unequal-Timestep Strategy: The authors introduce a strategy where the timestep for noise addition ( $t_{add}$ $t_{a dd}$ ) is decoupled from the timestep for diffusion denoising ( $t_{diff}$ $t_{d i f f}$ ).
- $t_{add}$ is set higher to expand the sampling space and increase diversity.
- $t_{diff}$ is set lower (more conservative) to maintain reconstruction fidelity and prevent performance degradation caused by excessive noise.
Architecture: Based on Stable Diffusion (SD2.1), using LoRA for fine-tuning the VAE and UNet.

B. Group Direct Preference Optimization (GDPO)

GDPO combines the precision of DPO with the efficiency of GRPO.

Two-Stage Process:
1. Advantage Calculation:
  - For a given LR input, the model generates a group of $G$ ISR candidates by sampling different noise vectors.
  - Attribute-Aware Reward Function (ARF): A novel reward function is designed to evaluate samples. It balances Full-Reference (FR) metrics (e.g., PSNR for fidelity) and No-Reference (NR) metrics (e.g., MANIQA, MUSIQ for perceptual quality).
  - Dynamic Weighting: The weights for FR and NR metrics are adjusted dynamically based on the image content. The image is split into smooth and detailed regions (using Sobel gradients and Shannon entropy). Smooth regions favor FR metrics, while detailed regions favor NR metrics.
  - Group Relative Advantage: The absolute rewards are converted into group-relative advantages ( $A_i$ ) by normalizing against the group mean and standard deviation.
2. Policy Optimization:
  - The model is updated using a modified Diffusion-DPO loss that incorporates the group-relative advantages.
  - The loss function prioritizes samples with higher advantages (better quality) while penalizing lower-quality ones, effectively aligning the policy with human preferences without requiring offline paired data.

3. Key Contributions

GDPO Framework: The first application of Group Direct Preference Optimization to one-step generative ISR, overcoming the deterministic limitation of existing one-step models.
NAOSD Model: A noise-aware one-step diffusion architecture with an unequal-timestep strategy that successfully injects controllable stochasticity to generate diverse outputs without sacrificing reconstruction fidelity.
Attribute-Aware Reward Function (ARF): A dynamic reward mechanism that adapts to image content (smooth vs. textured areas) to balance fidelity and perceptual quality, addressing the "global likelihood" flaw of standard GRPO.
Integration of DPO and GRPO: The method leverages online group sampling (like GRPO) for efficiency while maintaining pixel-level constraints (like DPO) to preserve local details.

4. Experimental Results

The authors evaluated GDPO-SR on real-world (DRealSR, RealSR) and synthetic (DIV2K-val) datasets.

Quantitative Performance:
- vs. Base Model (NAOSD): GDPO-SR consistently outperformed the base model across all metrics, showing significant gains in both fidelity (PSNR, SSIM) and perceptual quality (MANIQA, MUSIQ).
- vs. State-of-the-Art (SOTA): GDPO-SR achieved the best or near-best results among both multi-step (StableSR, SeeSR) and one-step (OSEDiff, InvSR) methods.
  - On DRealSR, it achieved the highest PSNR (28.18 dB) and SSIM (0.7839) while maintaining low LPIPS (0.2851).
  - It outperformed the RL-based multi-step method DP2OSR in fidelity metrics while maintaining competitive perceptual scores.
Qualitative Performance: Visual comparisons showed that GDPO-SR reconstructs clearer textures (e.g., brick walls, plant veins) and avoids the over-smoothing or hallucination artifacts common in competing methods.
Efficiency: GDPO-SR maintains the inference speed of one-step models (~0.11s per image), comparable to OSEDiff and InvSR, with no additional computational overhead during inference.
Ablation Studies:
- Confirmed that the unequal-timestep strategy is crucial for balancing diversity and fidelity.
- Demonstrated that the ARF (specifically the adaptive weighting) is superior to using fixed weights or single metric types.
- Showed that a group size ( $G$ ) of 6 provides an optimal balance between diversity and training stability.

5. Significance

This paper bridges the gap between the efficiency of one-step diffusion models and the high-quality generation capabilities of RL-based preference optimization.

Paradigm Shift: It proves that one-step models can be trained with RL to produce diverse, high-quality outputs, challenging the notion that multi-step processes are necessary for generative ISR.
Practicality: By maintaining a single-step inference time, the method is highly suitable for real-time applications while delivering quality comparable to computationally expensive multi-step diffusion models.
Generalizability: The proposed GDPO strategy and attribute-aware reward function offer a new direction for improving other one-step generative tasks where local detail preservation is critical.