💻 computer science

Pixel-Space Diffusion via Observation Operators

This paper introduces Observation Operator Diffusion, a framework that resolves the scale-time mismatch in pixel-space models by replacing fixed full-image supervision with a time-indexed, coarse-to-fine observation trajectory using Gaussian-Lanczos operators, thereby accelerating convergence and achieving state-of-the-art generation quality.

Original authors: Shaojie Guo, Lichen Ma, Haoyang Tong, Yu He, Zipeng Guo, Xiaoan Liu, Feng Yan, Yu Guo, Fei Wang, Junshi Huang, Yan Wang

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shaojie Guo, Lichen Ma, Haoyang Tong, Yu He, Zipeng Guo, Xiaoan Liu, Feng Yan, Yu Guo, Fei Wang, Junshi Huang, Yan Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, computers are learning to paint pictures from nothing but a list of words. For years, the most successful artists in this digital realm have worked in a compressed, abstract space, much like a sculptor who first carves a rough block of stone before refining the details. They build a simplified version of an image, then use a separate tool to translate that sketch into a high-resolution photograph. While this method works well, it inevitably loses some of the original texture and sharpness in the translation. A newer, more direct approach attempts to create the final image pixel by pixel, right from the start, promising a level of clarity that the older methods cannot match. However, this direct route has been notoriously difficult to master, often resulting in blurry or unstable creations that take an immense amount of time to perfect.

The core problem lies in how these models learn to see. Imagine trying to describe a distant mountain range to someone while standing in a thick fog. At first, you can only make out the broad, sweeping shapes of the peaks. As the fog lifts, the details of the trees and rocks become visible. If you tried to describe every single leaf and stone while the fog was still thick, you would be guessing wildly, and your description would be full of errors. This is exactly what happens inside the current generation of pixel-based image generators. They are forced to guess the finest details of an image even when the "fog" of noise is still heavy, trying to predict the entire picture at once. This mismatch between what the computer can actually see at a given moment and what it is being asked to predict creates a confusing signal that slows down learning and degrades the final quality.

Researchers at East China Normal University and JD.com have identified this specific confusion as a fundamental flaw in how these models are trained. They found that image structures naturally emerge from the blurry to the sharp, a process that takes time. Existing models, however, ignore this natural order. They demand that the computer predict the full, crisp image at every single step of the process, even when the input is still mostly random noise. This forces the model to struggle with details that are simply not recoverable yet, leading to a chaotic learning experience. To fix this, the team developed a new training method that respects the natural timeline of image recovery. Instead of asking the model to see everything at once, they teach it to look at the image through a series of lenses that change over time.

The researchers introduced a system where the target the computer tries to predict evolves alongside the noise. At the beginning, when the image is very noisy, the model is only asked to predict the broad, coarse shapes of the scene. As the noise is gradually removed, the target shifts, asking the model to add in slightly more detail, then even more, until finally, at the very end, it is asked to predict the complete, high-definition image. This is achieved using a family of mathematical filters that act like adjustable lenses, smoothing out the image to show only the large structures first, and then progressively revealing the fine textures. By aligning the difficulty of the prediction with what is actually visible at that moment, the computer receives a much clearer signal, allowing it to learn much faster and more effectively.

To ensure this principle works not just in time but also within the computer's internal structure, the team built a new decoder, a component responsible for turning the model's internal thoughts into a final picture. This new decoder is designed to handle information in layers, starting with the big picture and refining it into fine details as the data moves through the network. It injects specific structural information at each stage, ensuring that the model focuses on the right level of detail at the right depth. This creates a cohesive flow where the global layout is established first, followed by the gradual addition of textures and edges, mirroring the way the image itself is revealed during the denoising process.

The results of this approach are striking. When tested on a standard set of images featuring thousands of different objects, the new method produced significantly sharper and more realistic images than previous attempts at direct pixel generation. In one key test, the model reached a state-of-the-art quality score in just eighty training cycles, a speed that is substantially faster than its predecessors. With further training, it achieved a quality score of 1.52, a level of fidelity that surpasses all other direct pixel-based methods currently available. The images generated show a remarkable ability to capture both the overall composition and the intricate details, such as the texture of animal fur or the delicate structure of a flower, without the blurriness that often plagues this type of artificial intelligence.

This work suggests that the path to better image generation is not just about building larger or more complex computers, but about understanding the natural order in which information becomes clear. By respecting the timeline of how details emerge from noise, the researchers have created a system that learns more efficiently and produces higher-quality results. The method successfully bridges the gap between the abstract world of mathematical noise and the concrete reality of a high-definition photograph, proving that sometimes the best way to see the whole picture is to start by looking at the big shapes.

Technical Summary: Pixel-Space Diffusion via Observation Operators

Problem Statement

Pixel-space diffusion models offer a direct route to high-fidelity image generation by avoiding the information loss and reconstruction errors inherent in latent-space autoencoders. However, these models face significant optimization difficulties. Recent work has attempted to alleviate this through target reparameterization (e.g., predicting clean images, noise, or velocity), yet these methods retain a fundamental flaw: they rely on fixed full-image supervision throughout the entire denoising process.

The authors identify a "scale–time mismatch" as a primary source of this optimization difficulty. Empirical analysis reveals that image structures follow an intrinsic coarse-to-fine recovery order: global layouts become predictable from noisy inputs early in the denoising process, while fine details only become recoverable as noise decreases. Existing models force the network to predict these fine details even at high noise levels where they are ambiguous. This forces the model to fit unrecoverable information, resulting in low-SNR (Signal-to-Noise Ratio) gradients that hinder efficient optimization and slow convergence.

Methodology

To resolve the scale–time mismatch, the paper proposes Observation Operator Diffusion, a unified framework that aligns the supervision trajectory and feature refinement with the intrinsic recovery order of image structures.

1. Time-Indexed Observation Trajectory

Instead of using a fixed clean image $x_0$ as the target, the authors replace it with a time-indexed observation trajectory $y_t = \mathcal{A}_t(x_0)$ .

Mechanism: A family of operators $\{\mathcal{A}_t\}$ maps the clean image to observations that evolve from coarse structures to the full image as the flow time $t$ increases from 0 to 1.
Gaussian–Lanczos Operators: The trajectory is instantiated using a separable Gaussian–Lanczos kernel.
- The Lanczos component provides sharp scale selectivity.
- The Gaussian component suppresses ringing artifacts associated with Lanczos filtering.
- The kernel scales are annealed over time: at early stages (high noise), the operator acts as a low-pass filter (coarse observation); as $t \to 1$ , the scales shrink to identity, recovering the full image.
Path-Consistent Objective: The framework derives a path-consistent velocity target $v^*_t = \dot{z}_t = y_t - \epsilon + t\dot{y}_t$ . This objective explicitly couples the supervision scale with the flow time, ensuring the model is only trained on image structures recoverable at the current noise level. This eliminates the low-SNR gradients caused by forcing the prediction of fine details at high noise.

2. GL-CoDA Decoder (Coarse-to-Fine Feature Refinement)

The coarse-to-fine principle is extended from the temporal evolution of supervision to the spatial refinement of network features via the GL-CoDA (Gaussian–Lanczos Coarse-to-Detail Aggregation) decoder.

Architecture: The backbone follows a two-stage Patch–Pixel design (Patch-DiT encoder + GL-CoDA decoder).
Mechanism: The decoder contains a parallel GL Structure Branch that injects scale-specific structural responses across decoding stages.
- Shallow blocks receive coarse-scale structural responses.
- Deeper blocks receive progressively finer structural increments (computed as differences between adjacent operator responses).
Integration: These responses are upsampled and injected into the main pixel representation stream, allowing the decoder to refine features from global structure to fine-grained details in a scale-ordered manner.

Key Contributions

Identification of Scale–Time Mismatch: The paper reveals that the discrepancy between the fixed full-image supervision and the time-dependent recoverability of image structures is a fundamental, overlooked source of optimization difficulty in pixel-space diffusion.
Observation Operator Diffusion Framework: The authors propose replacing fixed supervision with a time-indexed trajectory of Gaussian–Lanczos observations, coupled with a path-consistent training objective that aligns the supervised scale with the recoverable structure at each step.
GL-CoDA Decoder: A novel decoder architecture that injects scale-specific Gaussian–Lanczos responses across network depth, extending the coarse-to-fine principle to feature refinement.
Empirical Performance: Extensive experiments demonstrate that the proposed approach converges substantially faster and improves generation quality.

Experimental Results

The method was evaluated on class-conditional generation for ImageNet-256 and ImageNet-512.

ImageNet-256:
- The model achieves a gFID of 1.52 after 260 training epochs.
- This establishes a new state of the art among pixel-space models, surpassing previous bests (e.g., PixelDiT-XL at 1.61) while using 60 fewer training epochs than the previous best to reach this performance level.
- At 80 epochs, the model already achieves a gFID of 1.95, demonstrating significantly faster convergence than baselines.
ImageNet-512:
- Achieves a gFID of 1.58, improving upon the previous best pixel-space result (1.81) by 12.7%.
- Shows improved distributional coverage (Recall increased from 0.67 to 0.69).
Ablation Studies:
- Kernel Design: The combination of Gaussian and Lanczos filtering outperforms using either alone or an identity operator, confirming the benefit of sharp yet artifact-free scale selectivity.
- Depth Assignment: Assigning coarse responses to shallow blocks and fine responses to deep blocks is critical; reversing this order degrades performance.
- Architecture Agnosticism: The observation path improves performance across different backbone architectures (JiT, PixelDiT), indicating the benefit is general to the supervision strategy.

Significance and Claims

The paper claims that the proposed framework addresses a fundamental limitation in pixel-space diffusion: the misalignment between the fixed supervision target and the dynamic recoverability of image structures. By aligning the supervised scale with the structure reliably recoverable at each denoising step, the method produces higher-SNR gradients, leading to:

Substantially faster convergence (reaching SOTA in fewer epochs).
Consistently improved generation quality (lower FID, better sFID, and higher recall).
Scalability to higher resolutions (512x512) without loss of performance.

The authors position this work as a reformulation of the supervision target itself, moving from a static endpoint to a dynamic, time-indexed observation trajectory that respects the physics of image recovery. They note that while the current focus is on natural images, the scale-aware framework holds promise for specialized domains like medical imaging.