💻 computer science

Unifying Deep Stochastic Processes for Image Enhancement

This paper unifies diverse deep stochastic processes for image enhancement under a common stochastic differential equation framework, demonstrating through controlled experiments that performance depends on specific design choices rather than a single superior method, and releases ItoVision to facilitate fair comparison and rapid prototyping.

Original authors: Wojciech Kozłowski, Radosław Kuczbański, Kamil Adamczewski, Karol Szczypkowski, Maciej Zięba

Published 2026-08-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wojciech Kozłowski, Radosław Kuczbański, Kamil Adamczewski, Karol Szczypkowski, Maciej Zięba

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of digital photography, a blurry, dark, or rain-streaked image is often just a starting point, not a dead end. For decades, scientists have tried to build computer programs that can look at a damaged photo and imagine what the original, perfect scene looked like. This is a difficult puzzle because a single bad photo could have come from many different good ones. To solve this, modern researchers have turned to a powerful idea called generative modeling. Instead of trying to guess a single answer, these programs learn to create a whole range of possible high-quality images, slowly refining a random cloud of static until a clear picture emerges. Recently, a specific type of these programs, known as diffusion models, has become the standard for making photos look better. However, as the field has grown, a new wave of methods has appeared that tries to guide this process more directly by using the bad photo as a constant map.

A team of researchers set out to untangle this growing confusion. They noticed that while many new methods claimed to be fundamentally different, they were all built on very similar mathematical foundations. The team decided to stop treating these methods as separate inventions and instead viewed them as variations of the same underlying process. They organized the landscape of image enhancement into three main families: standard models that start from pure randomness, methods that pull the image toward a specific target like a magnet, and methods that act as a bridge, forcing the process to start at the bad image and end exactly at the good one. By rewriting all these approaches into a single, unified language, the researchers could strip away the extra layers of complexity that had made fair comparison impossible. They removed the differences in how the programs were timed, how they were sampled, and how they were built, leaving only the core logic of each method to be tested.

What they found challenged a popular belief in the field. For some time, the assumption had been that these newer, more guided methods would naturally produce better results because they kept the bad image in mind throughout the entire process. The researchers ran a massive, controlled experiment across four different tasks: making low-resolution faces sharp, brightening dark photos, adding color to black-and-white images, and removing rain streaks. They trained every single method using the exact same computer architecture and the same rules. The results showed that there was no single champion. In fact, the standard, unguided models often performed just as well as, or even better than, the specialized guided methods. The study revealed that the perceived superiority of the newer techniques was often an illusion created by how they were implemented, rather than a true advantage of their design.

The researchers dug deeper to understand why some methods failed while others succeeded. They discovered that a specific setting, known as temperature, played a critical role. In the methods that tried to guide the image toward a target, setting this temperature too low caused the process to become too rigid. The computer would get stuck following a straight line between the bad input and the expected output, losing the ability to invent the missing details that make a photo look real. This resulted in images that were blurry and lacked texture. Conversely, when the temperature was set correctly, the methods performed much better. They also found that the way the computer took its steps mattered greatly. Using a rigid, predictable path to generate the final image caused the results to become oversmoothed, washing out fine details. The most consistent results came from allowing a bit of randomness in the final steps, which helped the model recover the intricate textures of the original scene.

To ensure that other scientists could verify these findings and build upon them, the team released a new software library called ItoVision. This tool puts all the different methods into a single, modular framework, allowing researchers to swap out the core process without changing the rest of the system. This transparency means that future comparisons will be fair and that progress will be measured by genuine improvements rather than by tweaking the settings of a specific program. The work suggests that the future of image enhancement lies not in inventing entirely new types of processes, but in carefully tuning the existing ones. By understanding that the differences between these methods are often just choices about how to steer the process, rather than fundamental differences in how they work, the field can move forward with a clearer, more unified approach to restoring the world's images.

Technical Summary: Unifying Deep Stochastic Processes for Image Enhancement

Problem Statement

Image enhancement encompasses inverse problems such as super-resolution, low-light enhancement, colorization, and deraining, where the goal is to recover a high-quality image ( $x_0$ ) from a degraded observation ( $y$ ). These problems are inherently ill-posed, as a single low-quality input corresponds to a distribution of plausible high-quality reconstructions. Consequently, modern systems utilize generative modeling to capture complex conditional distributions rather than relying on point estimation.

While diffusion-based models have become a strong baseline, a growing body of work proposes alternative stochastic processes that explicitly condition the generative trajectory on the degraded input $y$ (e.g., ResShift, IR-SDE, BBDM). These methods aim to interpolate between low- and high-quality images rather than evolving from unconditional noise. However, the relationship between these conditional processes and standard diffusion models remains unclear. Existing methods differ not only in their stochastic definitions but also in schedulers, samplers, discretization strategies, and architectures. This entanglement makes it difficult to isolate the effect of the underlying process itself, leading to a lack of fair, direct comparisons and a poor understanding of why certain conditional approaches succeed or fail.

Methodology

The authors introduce a unified perspective by classifying recent image enhancement methods into three families of continuous-time processes: unconditional diffusion models, Ornstein–Uhlenbeck (OU) processes, and diffusion bridges. The core contribution is demonstrating that all these approaches arise from a single Stochastic Differential Equation (SDE) formulation:

$d\mathbf{x}_t = f(\mathbf{x}_t, t, \mathbf{y}) dt + g(t) d\mathbf{w}_t$

where $\mathbf{x}_t$ is the state at time $t$ , $\mathbf{y}$ is the conditioning variable (degraded image), and $\mathbf{w}_t$ is a Wiener process. The authors show that seemingly disparate methods differ primarily in their:

Drift and diffusion terms ( $f$ and $g$ ).
Terminal distributions (base distributions at $t=1$ ).
Boundary conditions (how the process starts and ends).

Crucially, the framework decouples the definition of the stochastic process from implementation choices such as schedulers, samplers, and discretization strategies, treating the latter as orthogonal design choices.

Theoretical Unification

Unconditional Processes: Standard Diffusion Models (DM-VE, DM-VP) and Flow Matching (FM) are shown to be instances where the input $\mathbf{y}$ does not appear in the forward process definition but is provided as an additional input to the neural network. Flow Matching is reformulated as a continuous stochastic process whose reverse-time ODE recovers the original flow.
Ornstein–Uhlenbeck (OU) Processes: Methods like IR-SDE, ResShift, and InDI are identified as generalized OU processes where the stationary distribution is centered at $\mathbf{y}$ . The authors derive closed-form analytical solutions for these processes (e.g., for ResShift and InDI), replacing the discrete, numerically computed schedulers used in original implementations with continuous-time formulations.
Diffusion Bridges: Methods such as BBDM, DDBM, I2SB, GOUB, and UniDB are unified through Doob's h-transform. The authors demonstrate that BBDM, DDBM-VE, and I2SB are mathematically equivalent, differing only in their choice of noise scheduler. UniDB is shown to be a generalization that introduces a terminal cost parameter ( $\gamma$ ) to trade off endpoint accuracy against path smoothness.

Key Contributions

Unified SDE Framework: A continuous-time formulation that generalizes unconditional models, flow-based methods, OU processes, and diffusion bridges, explicitly separating method definitions from schedulers and samplers.
Controlled Empirical Study: A large-scale evaluation across four tasks (super-resolution, low-light enhancement, colorization, deraining) using identical backbone architectures (UNet and DiT), training protocols, and loss functions (MSE).
Identification of Performance Factors:
- Temperature: The authors identify low-temperature settings in OU-based methods as a key failure factor. Low temperature induces strong collinearity between intermediate states and the input, causing the network to learn simple extrapolation rather than modeling the true score, leading to poor detail recovery.
- Deterministic Sampling: The study reveals that deterministic samplers (e.g., Euler ODE) applied to diffusion bridges systematically cause oversmoothing. By eliminating the only source of stochasticity (Brownian motion), these samplers force trajectories to follow straight lines between $\mathbf{y}$ and $\mathbf{x}_0$ , converging to an averaged ground truth.
ItoVision Library: The release of a modular PyTorch library that implements the unified framework, enabling reproducible comparisons and rapid prototyping of stochastic image enhancement methods.

Results

The controlled experiments reveal no consistently dominant method across all tasks.

Performance Parity: When trained under identical protocols, most methods achieve remarkably similar performance. In many cases, plain unconditional diffusion models match or even outperform specialized OU- or bridge-based approaches.
Task-Specific Trends:
- Super-resolution & Low-light: Unconditional processes (DM-VP) showed slight advantages.
- Colorization: Diffusion bridges were most effective.
- Deraining: Both unconditional and bridge-based methods were competitive.
Specific Findings:
- ResShift performed slightly better than standard diffusion in aggregated results, but this was sensitive to temperature settings.
- InDI's poor performance was directly linked to its default low-temperature setting ( $\tau=0.06$ ); increasing the temperature improved its results significantly.
- UniDB outperformed GOUB in almost all tasks, attributed to its regularization via the parameter $\gamma$ , which prevents overshooting and keeps trajectories within well-modeled regions of the data manifold.
- Ancestral sampling (stochastic) was found to be the most consistent sampler, while deterministic samplers yielded high LPIPS scores (indicating oversmoothing) for diffusion bridges.

Significance and Claims

The paper claims to establish a new baseline for fair and interpretable evaluation of stochastic image enhancement methods. By unifying the theoretical underpinnings, the authors demonstrate that the perceived superiority of conditional stochastic processes is often an artifact of implementation choices (such as schedulers, temperature, and samplers) rather than the intrinsic nature of the process itself.

The work challenges the common assumption that conditioning the stochastic trajectory inherently leads to better restoration quality. Instead, it suggests that architectural and training choices often dominate method-specific differences. The release of ItoVision is intended to support future research by providing a standardized, extensible foundation that decouples theoretical definitions from numerical implementations, facilitating the development of novel approaches without the confounding variables of disparate experimental setups.