💻 computer science

Severity-Conditioned Pixel-Wise Asynchronous Diffusion for Semi-Transparent Mural Restoration

This paper proposes Severity-Conditioned Pixel-Wise Asynchronous Diffusion (SC-PAD), a novel framework that enhances semi-transparent mural restoration by estimating a continuous severity field to drive pixel-wise asynchronous diffusion progress, thereby overcoming the limitations of binary masks and uniform temporal modeling in handling spatially non-uniform degradations.

Original authors: Yu Wu, Erchao Li, Furui Jia, Jinjiang Zhao, Lei Zhang

Published 2026-10-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yu Wu, Erchao Li, Furui Jia, Jinjiang Zhao, Lei Zhang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Ancient murals are fragile windows into the past, preserving the beliefs, art, and daily lives of civilizations long gone. However, time is a relentless enemy. Over centuries, fluctuations in temperature, humidity, and exposure to light and dust cause these painted surfaces to fade, discolor, crack, and develop mold. Unlike a simple scratch that removes a piece of a painting, many of these damages are semi-transparent. Think of a layer of grime or a stain that darkens the colors underneath but does not completely hide them. The original pigment, texture, and outlines often remain visible, just obscured. Restoring these images is a delicate task for digital conservators. If a computer tries to simply fill in the damaged areas as if they were missing, it risks inventing details that never existed or erasing the faint, original traces that are still there. The challenge lies in distinguishing between what is lost and what is merely hidden, and then carefully revealing the original image without overwriting the subtle evidence that remains.

For years, digital restoration tools have relied on a binary approach: they identify a damaged area with a simple mask, treating every pixel inside that mask as equally broken. This method works well for holes or missing pieces, but it fails when the damage varies in intensity across a single region. A patch of mold might be thick in the center and thin at the edges, yet a standard mask treats the whole patch the same way. Furthermore, most modern restoration algorithms, which use advanced generative models to imagine missing details, apply a single, uniform timeline to the entire image. They assume that every part of the picture needs the same amount of "cleaning" time to reach a restored state. This one-size-fits-all approach struggles with murals where degradation is uneven, often leading to results that look either too muddy or too artificially sharp.

To address these limitations, a team of researchers from Lanzhou University of Technology and the Gansu Academy of Sciences has developed a new method called Severity-Conditioned Pixel-Wise Asynchronous Diffusion. Instead of treating a damaged mural as a collection of broken and unbroken parts, this system views the damage as a continuous field of varying intensity. The researchers first teach the computer to estimate exactly how severe the degradation is at every single point on the image, creating a detailed map of the damage's strength rather than just a simple outline. This map acts as a guide, telling the restoration engine how much of the original image is still visible beneath the grime at any given spot.

The core innovation of this work is the shift from a synchronized process to an asynchronous one. In traditional methods, the computer works on the entire image at the same speed, moving from a noisy, damaged state to a clean state in lockstep. The new method allows each pixel to move at its own pace. If a specific area is only slightly faded, the system recognizes that it needs very little help and restores it quickly. If another area is heavily obscured, the system dedicates more computational effort to carefully reconstructing it. This is managed by a dynamic system that adjusts the "restoration progress" for every pixel individually, ensuring that the final result respects the unique history of damage in every corner of the mural. The process also uses a specific mathematical rule, similar to how layers of transparent paint mix, to ensure that the restored image blends naturally with the surrounding healthy areas, preserving the original contours and textures.

The researchers tested their system on a dataset of Dunhuang murals, which are famous for their historical and artistic significance. They compared their new method against several existing techniques, including those that use standard masks and other advanced generative models. The results showed that the new approach produced significantly clearer and more accurate restorations. In terms of pixel-level accuracy, the new method achieved a score of 30.61, outperforming the next best method which scored 29.7. More importantly, the visual results demonstrated that the system could handle the subtle transitions of semi-transparent damage much better than previous tools. It preserved the fine lines and structural details of the murals without introducing the blurring or artificial sharpness often seen in other restorations.

However, the study also highlighted a critical bottleneck. While the system works exceptionally well when it is given the exact, true severity of the damage, its performance drops when it has to estimate that severity on its own. The gap between the results achieved with perfect knowledge of the damage and the results achieved with the system's own estimates suggests that the ability to accurately measure the depth of the damage is the next major hurdle. The researchers found that while their continuous severity map was a vast improvement over simple binary masks, there is still room for improvement in how the computer interprets the visual clues of degradation.

Ultimately, this work provides a more nuanced framework for digital conservation. By moving away from rigid, uniform processes and embracing the complex, uneven reality of how murals age, the researchers have created a tool that can better respect the integrity of these cultural treasures. The method does not just fill in the blanks; it listens to the subtle variations in the damaged surface, allowing the original art to emerge with greater fidelity. While the technology is not yet perfect, particularly in how it gauges the severity of the damage without human input, it represents a significant step forward in treating digital restoration as a process of careful revelation rather than simple reconstruction. The findings suggest that future advancements will likely focus on refining the system's ability to "see" the depth of the damage, ensuring that the digital guardians of our heritage can protect these fragile stories with even greater precision.

Technical Summary: Severity-Conditioned Pixel-Wise Asynchronous Diffusion for Semi-Transparent Mural Restoration

Problem Statement
Digital mural restoration faces a specific challenge with semi-transparent degradations (e.g., fading, discoloration, mold, soot) where original pigment, texture, and contour information remain partially visible. Existing deep learning approaches, particularly those based on Diffusion Probabilistic Models, typically rely on binary damage masks. These masks identify the spatial support of degradation but fail to encode the continuously varying severity within that region. Furthermore, standard diffusion models assign a shared temporal progress ( $t$ ) to the entire image, treating all pixels as being at the same diffusion state. This global synchronization is ill-suited for spatially heterogeneous degradation, where different pixels possess different admissible state ranges and require distinct restoration dynamics. Current methods often treat severity merely as an auxiliary conditioning channel rather than an explicit state variable governing the diffusion trajectory.

Methodology: SC-PAD
The authors propose Severity-Conditioned Pixel-Wise Asynchronous Diffusion (SC-PAD), a framework that explicitly models continuous terminal severity and decouples the temporal progress of individual pixels while maintaining a unified global schedule. The method consists of three core components:

Continuous Terminal Severity Estimation:
Instead of relying solely on a binary mask, SC-PAD estimates a continuous severity field $A \in [0,1]^{H \times W}$ from the degraded observation and a coarse binary support. This is achieved via a mask-guided estimator with a bounded residual-logit refinement branch. To address the issue where easy pixels dominate the loss and obscure large local errors, the authors introduce risk-oriented supervision, comprising:
- A top- $q$ tail loss for the largest absolute errors.
- A one-sided penalty for positive severity errors.
- A neighborhood spike penalty to suppress isolated high-severity predictions.
Pixel-Wise Asynchronous Trajectory Reparameterization:
The framework introduces a temporal-warping exponent $\gamma(u)$ , predicted by a trajectory predictor conditioned on the degraded image and the estimated terminal severity. This exponent maps the shared global reference progress $m_t$ to a pixel-wise local restoration progress $g_t(u) = m_t^{\gamma(u)}$ .
- Constraints: $\gamma$ is bounded (e.g., $[0.35, 3.0]$ ) to ensure the process remains within a unified global temporal range.
- Properties: This reparameterization preserves common start ( $t=0$ ) and end ( $t=T$ ) points for all pixels while allowing strictly increasing local clocks. It enables spatially differentiated restoration trajectories without altering the fundamental boundary conditions of the diffusion process.
Dynamic Alpha State and Residual Diffusion:
The local progress $g_t$ is mapped to a dynamic Alpha state $\alpha_t$ that satisfies endpoint consistency, temporal monotonicity, and the Alpha composition law.
- Formulation: $\alpha_t = 1 - (1-A)^{g_t}$ . This follows the multiplicative structure of Alpha composition in the transmittance domain rather than arbitrary linear interpolation.
- Residual Activation: The terminal image-space residual $R_M = Y - X_0$ is progressively activated using a ratio $\rho_t$ derived from $\alpha_t$ . The noisy intermediate state is constructed as $X_t = X_0 + \rho_t \odot R_M + \sigma_t \epsilon$ .
- Coupling: This formulation explicitly links terminal severity, local time, and the intermediate diffusion state, ensuring that the restoration trajectory is constrained by the estimated severity.

Key Contributions
The paper outlines three primary contributions:

Explicit Severity State: Formulating continuous terminal severity as an explicit state variable rather than an auxiliary condition. A specialized estimator with bounded refinement and risk-oriented supervision reduces local mismatch, allowing severity to act as a terminal-state constraint.
Asynchronous Temporal Reparameterization: Introducing a severity-conditioned pixel-wise temporal warping mechanism that maps global progress to local progress. This enables spatially differentiated restoration trajectories within a single diffusion schedule while preserving temporal ordering and common endpoints.
Composition-Consistent Dynamic Alpha: Constructing a terminal-severity-constrained dynamic Alpha state that satisfies the Alpha composition law. This controls the progressive activation of the image-space residual, establishing an explicit mathematical link between severity, local time, and the diffusion state.

Experimental Results
Experiments were conducted on the Dunhuang mural dataset (400 training, 100 test images) covering ten degradation categories (salt crystallization, discoloration, cracking, etc.). SC-PAD was compared against state-of-the-art methods including LRDiff, StrDiffusion, RDDM, and PGRDiff.

Quantitative Performance: SC-PAD achieved the highest scores among all methods: PSNR 30.61 dB, SSIM 0.925, and LPIPS 0.071. It outperformed the closest baseline (PGRDiff) by 0.91 dB in PSNR and 0.0073 in SSIM.
Ablation on Severity Representation: Replacing the binary mask with the predicted soft Alpha improved PSNR by 3.27 dB (from 27.34 to 30.61). Using ground-truth Alpha (oracle) further increased PSNR to 33.87 dB, indicating that while continuous severity modeling is highly effective, the accuracy of severity estimation remains a performance bottleneck.
Trajectory Sensitivity: The study evaluated synchronous vs. asynchronous time and linear vs. composition-consistent Alpha states. Pixel-wise asynchronous time consistently improved performance across both Alpha formulations. The composition-consistent Alpha state yielded the best results when combined with asynchronous time (PSNR 31.52), outperforming the linear formulation in that specific configuration.

Significance and Claims
The paper claims that SC-PAD provides a controllable diffusion framework specifically designed for spatially non-uniform, semi-transparent mural restoration. Its significance lies in:

Moving beyond binary "missing-region" completion to modeling degradation as a continuous, spatially varying process.
Decoupling the temporal progress of individual pixels to better match the heterogeneous nature of mural deterioration.
Demonstrating that continuous severity representation and pixel-wise asynchronous progress are the two most critical components supported by current results.

The authors modestly note that while the proposed formulation improves pixel fidelity and structural consistency, severity estimation accuracy remains a significant bottleneck, as evidenced by the gap between predicted and ground-truth Alpha performance. Future work is directed toward improving severity estimation under complex real-world conditions and integrating multi-modal evidence (e.g., multispectral imaging, material analysis).

Technical Summary: Severity-Conditioned Pixel-Wise Asynchronous Diffusion for Semi-Transparent Mural Restoration

More like this