💻 computer science

Preserve More Details: Mitigating Content Drift in Real-World Image Super-Resolution

The paper introduces FSP-Diff, a novel one-step diffusion model with a dual-pathway architecture designed to mitigate content drift and preserve fine details in real-world image super-resolution by effectively balancing structured detail recovery with semantic guidance.

Original authors: Chunxiao Liu, Wei Liu, Anbin Xiong, Erli Meng

Published 2026-08-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chunxiao Liu, Wei Liu, Anbin Xiong, Erli Meng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to fix a blurry, pixelated photo of your favorite city street. You want it to look crisp and clear, like a high-definition movie still. This is the world of Image Super-Resolution, a branch of computer science dedicated to taking low-quality, grainy pictures and magically turning them into high-quality masterpieces. In the past, computers struggled with "real-world" photos because they didn't know how to handle the messy, unpredictable blurs of the real world (like shaky hands or bad weather). Recently, scientists have started using powerful AI tools called Diffusion Models. Think of these models as artists who have seen billions of pictures and learned how nature usually looks. They can guess what a blurry fence should look like by remembering what fences generally look like. However, there's a catch: sometimes these AI artists get too creative. Because the original photo is so blurry, the AI might guess the wrong details entirely—turning a house into a pile of snow or a fence into a solid wall. This paper tackles that specific problem: how to let the AI use its imagination without letting it daydream away the truth.

The researchers behind this study, Chunxiao Liu and his team from Xiaomi Corporation, noticed that when AI tries to fix a blurry photo, it often loses the tiny, important details and changes the meaning of what's in the picture. They call this "content drift." It's like trying to copy a sketch from a distance; if you squint too hard, you might accidentally turn a cat into a dog because the details are too fuzzy to see. The team found that existing methods rely too much on the AI's "memory" of what things should look like, rather than the actual, faint clues left in the blurry photo. This causes two main problems: visual detail degradation (the fine lines get mushy or disappear) and textual semantic shift (the AI changes the story, like turning a "house" into "snow" because it can't see the roof clearly).

To fix this, the team built a new system called FSP-Diff. Imagine the AI as a painter who usually just guesses the whole picture based on a vague description. FSP-Diff gives this painter two special tools. The first tool is a "Detail-Injector." Before the painter starts, this tool scans the blurry photo with a super-sensitive eye (a specific type of AI called a Vision Transformer) to find the hidden, sharp edges and lines that the main AI usually misses. It then hands these "structured details" to the painter, forcing them to stick to the real shapes in the photo. The second tool is a "Semantic Inspector." Sometimes the AI gets confused about what it's looking at (thinking a house is a snow pile). This tool checks the "story" (the text description the AI generates) against the real details it just found. If the story doesn't match the shapes, the tool corrects the story so the painter doesn't get led astray.

The paper shows that this two-tool approach works incredibly well. By using a "dual-pathway" design—one path for injecting the hard facts of the image and another for checking the story—the new model manages to keep the fine details sharp and the meaning accurate. In tests, FSP-Diff was able to do this in just one step, which is much faster than other methods that take many steps to refine the image. The results showed that their method produced clearer images with fewer weird mistakes compared to other top-tier AI models. For instance, on a test called DIV2K-Val, their model achieved a score of 24.44 for image clarity (PSNR), beating the previous best one-step model, OSEDiff, which scored 23.72. They also found that their method reduced the "drift" where a house might turn into snow, keeping the reconstructed images looking much more like the original scene.

The authors are careful to note that while their method is a significant improvement, it isn't a magic wand that solves every problem instantly. They tested it on standard datasets and found it consistently outperformed other one-step diffusion methods in both numbers and how the images looked to human eyes. However, they also observed that some other models, which used more complex training or bigger datasets, sometimes scored higher on specific "no-reference" metrics (tests that judge quality without a perfect original to compare against). Despite this, FSP-Diff managed to strike a great balance, preserving more of the original structure without needing a massive, multi-stage training process. The team suggests that by focusing on preserving those tiny, structured details and making sure the AI's "story" matches the "shapes," they can stop the content drift that has plagued real-world image restoration. In short, they taught the AI to look closer at the clues before it starts guessing, ensuring the final picture is not just pretty, but actually true to the original.

Technical Summary: Preserve More Details: Mitigating Content Drift in Real-World Image Super-Resolution

Problem Statement
Real-world image super-resolution (Real-ISR) aims to reconstruct high-quality (HQ) images from low-quality (LQ) inputs captured in unconstrained scenarios. While recent methods leveraging Stable Diffusion (SD) models have achieved impressive results by injecting natural image priors, they suffer from a critical limitation: content drift. The authors identify that the insufficient clarity of LQ inputs, combined with the highly compressed latent space of SD's Variational Autoencoder (VAE), leads to two specific failure modes:

Visual Detail Degradation: Fine structures (e.g., fences, textures) are discarded during latent compression, causing the model to rely too heavily on pre-trained priors rather than the intrinsic content of the LQ image.
Textual Semantic Shift: Existing methods often generate captions from LQ images to guide synthesis. However, unclear details in LQ inputs lead to inaccurate captions, resulting in severe misreconstructions (e.g., a house rendered as a snow pile).

Methodology: FSP-Diff
To address these issues, the authors propose FSP-Diff, a novel one-step diffusion model designed to preserve fine structures and provide robust semantic guidance. The framework is built upon a pre-trained text-to-image (T2I) diffusion model (specifically SD 2.1-base) and introduces a dual-pathway architecture guided by structured details extracted from the LQ input.

Structured Details Extraction:
Instead of relying solely on the compressed latent representation ( $Z_L$ ), the model extracts structured details ( $X_S$ ) using a frozen, pre-trained Vision Transformer (ViT) encoder. The first block of the ViT is utilized to capture low-level edge and boundary information before significant semantic abstraction occurs.
Dual-Pathway Architecture:
- Detail-Conditioned Pathway (Detail Injector): This pathway is responsible for recovering fine structures. It employs a Detail-Aware Token Compression (DATC) module to compress the high-resolution feature map $X_S$ into a compact set of tokens ( $X_H$ ) to reduce computational cost. These tokens are then integrated into the diffusion process via a Structured Detail Conditioned Attention (SDCA) module. SDCA allows the diffusion latents to attend to the structured details, explicitly guiding the generation of fine-grained structures.
- Detail-Modulated Semantic Pathway (Semantic Inspector): This pathway refines the semantic guidance to prevent textual drift. It uses a Semantic–Detail Modulator (SDM) to adaptively adjust the text embeddings ( $X_C$ ) derived from the LQ image. By computing attention weights between the semantic embeddings and the structured details ( $X_S$ ), the SDM rectifies potential semantic misalignments, ensuring the text condition accurately reflects the actual content of the LQ image.
Training Strategy:
The model is trained using a reconstruction loss ( $L_{rec}$ ) combining $L_1$ and LPIPS, alongside a Variational Score Distillation (VSD) regularization loss ( $L_{reg}$ ) to align outputs with real-world HQ distributions. A timestep-shifted training strategy is applied to the noise scheduler to stabilize training, with a scaling factor $s=0.9$ found to be optimal.

Key Contributions
The paper outlines three primary contributions:

Problem Identification: The authors explicitly characterize "content drift" in Real-ISR as a dual problem of visual detail degradation and textual semantic shift caused by unreliable LQ inputs and compact latent spaces.
Architectural Innovation: They propose FSP-Diff, a one-step diffusion framework featuring a dual-pathway design. This design injects structured details to recover fine structures and modulates semantic guidance to mitigate deviations, jointly improving fidelity and reliability.
Performance: Extensive experiments demonstrate that FSP-Diff achieves significant improvements over existing one-step diffusion methods, preserving more structured details in a single inference step without requiring complex multi-stage training or larger external datasets.

Experimental Results
The authors evaluated FSP-Diff on standard benchmarks (DIV2K-val, RealSR, DRealSR) against state-of-the-art one-step methods (SinSR, OSEDiff, TSD-SR, TVT) and multi-step methods.

Quantitative Performance: FSP-Diff consistently outperforms OSEDiff on approximately 80% of metrics. It achieves the best or second-best scores in reference-based metrics (PSNR, SSIM, LPIPS, DISTS) and competitive results in no-reference metrics (NIQE, MANIQA). Notably, it achieves the lowest FID scores across all three benchmarks, indicating strong distributional alignment with real HQ images.
Qualitative Performance: Visual comparisons show that FSP-Diff effectively recovers fine details (e.g., fur, textures) that other methods blur or hallucinate. It successfully avoids the "semantic shift" errors (e.g., misidentifying objects) seen in baselines. Compared to TVT, which preserves details but introduces rigid polygonal artifacts, FSP-Diff produces more natural and faithful reconstructions.
Efficiency: Despite the added complexity of the dual pathways, FSP-Diff maintains a low inference time (0.15s on A100 for 512x512 images), significantly faster than multi-step diffusion models (e.g., StableSR at 12.42s) and comparable to other one-step methods.

Significance
The paper claims that FSP-Diff represents a significant step forward in Real-ISR by directly addressing the root causes of content drift. By explicitly extracting and utilizing structured details from the input to guide both the generative process and the semantic conditioning, the framework achieves a better balance between fidelity (preserving input content) and perceptual quality (generating realistic details). The work demonstrates that a carefully designed one-step diffusion model can surpass more complex multi-step approaches and other one-step baselines without relying on extensive multi-stage training or larger foundational models, offering a practical and efficient solution for real-world super-resolution tasks.

Technical Summary: Preserve More Details: Mitigating Content Drift in Real-World Image Super-Resolution

More like this