⚡ electrical engineering

A Noise Constrained Diffusion (NC-Diffusion) Framework for High Fidelity Image Compression

This paper proposes a Noise Constrained Diffusion (NC-Diffusion) framework for high-fidelity image compression that reformulates quantization noise as the diffusion process noise to resolve noise mismatch and improve efficiency, while incorporating adaptive frequency-domain filtering and zero-shot sample-guided enhancement to achieve state-of-the-art reconstruction quality.

Original authors: Zhenyu Du, Yanbo Gao, Shuai Li, Yiyang Li, Hui Yuan, Mao Ye

Published 2026-04-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhenyu Du, Yanbo Gao, Shuai Li, Yiyang Li, Hui Yuan, Mao Ye

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a high-definition photo of your favorite landscape to a friend, but your internet connection is very slow. To make the file small enough to send quickly, you have to "compress" it.

The Problem: The "Blurry Photo" Dilemma
Traditionally, compression works like a strict editor who throws away details to save space. The result is often a blurry, blocky image.

Recently, scientists tried using AI "Generators" (specifically called Diffusion Models) to fix this. Think of these AI generators like a talented artist who has seen millions of photos. If you give them a blurry sketch, they can "imagine" and paint in the missing details to make it look sharp again.

However, there was a big catch with the old way of using these artists:

The Old Way: The AI started with a blank, static-filled canvas (pure random noise) and tried to guess what the original photo looked like, using the blurry sketch as a hint.
The Result: Because the AI started from random noise, it sometimes "hallucinated." It might add a tree that wasn't there, change the color of a car, or invent a letter "E" that was actually missing in the original. It was great at making pretty pictures, but bad at making accurate copies of your specific photo.

The Solution: The "NC-Diffusion" Framework
The authors of this paper, Zhenyu Du and his team, came up with a smarter way to use the AI artist. They call it Noise Constrained Diffusion (NC-Diffusion).

Here is how it works, using a simple analogy:

1. The "Noise Mismatch" Problem

Imagine the compression process creates a specific type of "dust" on your photo (called Quantization Noise). It's a predictable, structured dust caused by the math of shrinking the file.

Old AI: The AI was trained to remove "random static" (like TV snow). When you gave it the "structured dust" from compression, it got confused. It tried to remove the dust but ended up smearing the picture or inventing new things because the "dust" didn't look like the "static" it was used to.
New AI (NC-Diffusion): The authors realized, "Hey, let's teach the AI to remove this specific type of dust." Instead of starting with random static, they teach the AI to understand exactly how the compression dust forms.

2. The "Reverse Journey"

Instead of the AI starting from a blank, noisy canvas and guessing the image, the new method starts with the already compressed, blurry image.

Analogy: Imagine you have a muddy footprint.
- Old Method: You try to guess what the shoe looked like by staring at a pile of random mud.
- New Method: You take the muddy footprint, and the AI acts like a master restorer who knows exactly how the mud got there. It gently wipes away only the mud that belongs to the compression process, revealing the original shoe print underneath without changing the shape of the shoe.

3. The "High-Frequency" Superpower

Even after cleaning the mud, some fine details (like the texture of a brick wall or the strands of hair) might still be fuzzy.

The Fix: The authors added a special "Frequency Filter." Think of this as a pair of glasses that only lets the AI see the fine, sharp details (high frequencies). It forces the AI to focus on sharpening those tiny edges without messing up the big, smooth parts of the image (like the sky).

4. The "Zero-Shot" Guide

Finally, to make sure the AI doesn't get too creative and change the meaning of the photo, they added a "Guide."

Analogy: Imagine the AI is painting, but you want to make sure it doesn't turn a red apple into a green one. The "Guide" (using a tool called CLIP) constantly checks: "Does this new detail look like the original blurry sketch?" If the AI tries to add something weird, the Guide nudges it back to the truth.

Why is this a Big Deal?

Speed: Because the AI starts with the blurry image instead of random noise, it doesn't have to guess as much. It finishes the job much faster (like cleaning a specific stain vs. washing a whole dirty shirt).
Accuracy: It keeps the image looking exactly like the original (faithful) while making it look sharp and clear (high fidelity). It doesn't invent fake trees or change letters.
Efficiency: It saves more data (bits) while looking better than any previous method.

In Summary:
This paper teaches an AI artist how to fix a compressed photo by understanding the specific errors made during compression, rather than guessing from scratch. It's like giving the artist a map of exactly where the dirt is, so they can clean it perfectly without accidentally painting over the original picture.

1. Problem Statement

While diffusion models have revolutionized image generation, their application to image compression faces a critical challenge: Noise Mismatch.

The Conflict: Traditional diffusion-based compression methods start the reconstruction process from random Gaussian noise. However, the actual noise introduced in learned image compression (during the quantization step) is quantization noise, which follows a specific, non-Gaussian distribution determined by the encoder/decoder architecture and the input image content.
The Consequence: Using random Gaussian noise to model quantization noise introduces inherent uncertainty and randomness. This leads to:
- Deviation from the original image: Reconstructions often hallucinate details or lose specific textures (e.g., missing letters or patterns) to satisfy perceptual metrics, failing the "fidelity" requirement of compression.
- Inefficiency: Starting from pure noise requires many inference steps to converge, slowing down the decoding process.
- Suboptimal Quality: The mismatch between the assumed noise distribution (Gaussian) and the actual noise (quantization) results in artifacts and poor rate-distortion/perception trade-offs.

2. Methodology: NC-Diffusion Framework

The authors propose the Noise Constrained Diffusion (NC-Diffusion) framework, which fundamentally rethinks the relationship between compression and diffusion by aligning their noise processes.

A. Core Concept: Noise Constraint

Instead of adding random Gaussian noise, the framework treats the quantization noise itself as the noise source for the diffusion process.

Forward Process: The ground-truth image ( $I_0$ ) is diffused into the initial compressed result ( $I_T$ ) by adding the specific quantization noise ( $\epsilon_n$ ) derived from the compression pipeline ( $I_T = D\{Q[E(I_0)]\}$ ).
Reverse Process: The model learns to reverse this specific process. During inference, it starts directly from the initial compressed image ( $I_T$ ) and iteratively removes the quantization noise to recover the ground truth ( $I_0$ ).
Benefit: This eliminates the "randomness" gap. The diffusion process is constrained to the actual noise distribution of the compression system, ensuring the reconstruction remains faithful to the original image while removing compression artifacts.

B. Adaptive Frequency-Domain Filtering (AFF)

To address the loss of high-frequency details (textures) common in compression:

Mechanism: An Adaptive Frequency-Domain Filtering module is inserted into the skip connections of the U-Net backbone.
Operation: It transforms skip features into the Fourier domain, applies a learnable spectral modulation mask ( $\alpha_k$ ) to selectively enhance high-frequency components, and transforms them back.
Goal: Explicitly boost high-frequency details without distorting low-frequency structural information.

C. High-Frequency Detail Preservation Loss

A specialized loss function ( $L_{high}$ ) is introduced to supervise the reconstruction of high-frequency details.

Implementation: Both the ground truth and predicted images are decomposed using a 2D Discrete Wavelet Transform (DWT).
Objective: The Mean Squared Error (MSE) is calculated specifically on the high-frequency sub-bands (horizontal, vertical, and diagonal) to force the model to learn fine textures.

D. Zero-Shot Sample-Guided Enhancement

To further improve perceptual fidelity during inference without retraining:

Mechanism: A CLIP-based perceptual loss is calculated between the initial compressed image and the current denoised estimate.
Guidance: The gradient of this loss is backpropagated to adjust the sampling trajectory. This guides the diffusion process to retain the semantic and perceptual content of the initial reconstruction, preventing the model from drifting too far from the original image content.

3. Key Contributions

Noise-Constrained Diffusion Framework: A novel approach that equates quantization noise with diffusion noise, solving the distribution mismatch problem. This allows for direct inference from the compressed image, significantly improving inference efficiency and reconstruction fidelity.
Plug-and-Play AFF Module: A frequency-domain filtering module that enhances skip connections in U-Net architectures, specifically targeting the recovery of lost high-frequency details.
Zero-Shot Enhancement: A test-time strategy using CLIP guidance to balance distortion and perception without requiring additional training data.
State-of-the-Art Performance: The method achieves superior results in both Rate-Distortion (PSNR) and Rate-Perception (FID, LPIPS, MS-SSIM) metrics compared to existing generative compression methods.

4. Experimental Results

The method was evaluated on CLIC2020 and Kodak datasets against state-of-the-art methods (e.g., HiFiC, MS-ILLM, DiffEIC, CDC).

Performance Metrics:
- BD-Rate Savings: On the Kodak dataset, the proposed method achieved a 28.01% BD-rate saving compared to the baseline (ELIC) when considering all components (NC-Diffusion + AFF + $L_{high}$ ).
- Fidelity vs. Perception: Unlike other generative methods that sacrifice fidelity for perceptual scores (often hallucinating details), NC-Diffusion maintains high PSNR (objective quality) while achieving excellent FID (distribution alignment).
- Visual Quality: Visual comparisons show that NC-Diffusion preserves specific textures (e.g., text, fine patterns) that other methods often blur or hallucinate.
Efficiency:
- Decoding Speed: The method is significantly faster than other diffusion-based compressors (e.g., CDC, DiffEIC). Because it starts from a reconstructed image rather than random noise, it can produce high-quality results in fewer inference steps (even 1-step is viable, though 10 steps yield better results).
- Table I Results: Decoding speed for NC-Diffusion is 0.102s, compared to 1.02s for CDC and 1.837s for DiffEIC.

5. Significance

This paper represents a significant step forward in learned image compression by bridging the gap between generative AI and data compression.

Paradigm Shift: It moves away from the "generate from noise" paradigm for compression, which is inherently lossy in terms of fidelity, to a "denoise from compression artifacts" paradigm.
Practicality: By improving inference speed and ensuring the reconstructed image is a faithful representation of the source (rather than a plausible hallucination), NC-Diffusion makes diffusion models viable for real-world high-fidelity compression applications where data integrity is crucial.
Generalizability: The proposed modules (AFF, sample-guided enhancement) are modular and can be applied to other diffusion-based low-level vision tasks.