📄 other

Identification and Graded Restoration of Composite Damage in Silk Paintings under Semantic Constraints

This paper proposes a semantic-constrained, multi-stage digital restoration framework for traditional silk paintings that prioritizes the protection of cultural heritage elements by integrating interpretable damage detection, context-aware restoration routing, and quality-gated rollback mechanisms to balance effective damage removal with the preservation of historical evidence and material texture.

Original authors: Zhihao Zhang, Ying lLiu, Xing Zhang, Jiayuan Liu, Jialiang He

Published 2026-07-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhihao Zhang, Ying lLiu, Xing Zhang, Jiayuan Liu, Jialiang He

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but the crime scene is a beautiful, ancient silk painting that has been left out in the rain for a thousand years. The silk is yellowed, the paint is flaking off, and there are cracks running through the faces of the people painted on it. Now, imagine you have a super-smart robot assistant that can "fix" the painting by filling in the missing parts. The problem? This robot is a bit too eager. If you ask it to fix a crack, it might accidentally erase the person's eyebrow or rewrite a historical poem because it thinks it knows what should be there better than the original artist. This is the tricky world of digital image restoration. It's a branch of computer science where algorithms try to repair damaged photos or art. The core challenge is balancing two opposing goals: removing the damage (like cracks and stains) while protecting the original history (like the artist's brushstrokes, seals, and text). If you fix too much, you destroy the evidence; if you fix too little, the damage remains. This paper tackles that exact tension, asking: How do we let a computer fix a painting without letting it rewrite history?

The authors of this study, Zhihao Zhang and his team, propose a new way to fix traditional Chinese silk paintings that acts more like a cautious museum curator than a flashy art restorer. Instead of letting a computer guess and fill in the blanks all at once, they built a "smart traffic system" for the image. Think of the painting as a busy city. Some areas are no-go zones (like the text, the red collector's seals, and the delicate lines of the figures' faces); these are the "VIPs" that must never be touched. Other areas are construction zones (the cracks and stains). The computer's job is to navigate these zones carefully.

First, the system draws a map of the city, marking the VIPs with an absolute "Do Not Enter" sign. Then, it scans the rest of the painting to find the damage. But here's the clever part: it doesn't just use one tool for everything. It sorts the damage into different categories based on how big and complicated it is.

Tiny cracks get a gentle, mathematical "harmonious" fix that just smooths them out without changing the texture.
Big, messy holes where paint has fallen off get a powerful "deep learning" tool (a type of AI) that can imagine what the missing texture might look like.
Dangerous areas right next to the VIPs? The system skips them entirely or treats them with extreme caution, refusing to let the AI guess.

But the real magic happens at the end with a "Quality Gate." Imagine a strict inspector standing at the exit. Every time the computer finishes a repair, the inspector checks it. Did the repair look too smooth? Did it leave a weird seam? Did it accidentally blur a seal? If the answer is yes, the inspector hits a "soft rollback" button, undoing the change and leaving the original damage alone rather than a bad fix. This ensures that if the computer makes a mistake, it doesn't become permanent.

The team tested this method on twelve real silk paintings that were actually damaged, not just made-up computer images. They found that the paintings had a lot of damage, with the figures (the people) taking up about 36.77% of the image and the damaged areas they wanted to fix covering about 24.32%. The results were promising: the system managed to keep the original "energy" of the painting's lines and textures very stable, with a gradient preservation ratio of 0.9824 (a score very close to perfect, meaning the original details weren't blurred). Crucially, the changes the computer made stayed mostly inside the damaged areas, and it didn't accidentally mess up the protected text or seals.

The paper suggests that this method is not about making the painting look "perfect" or brand new in one click. Instead, it offers a safe, step-by-step process where experts can watch the computer work, see why it made a decision, and even hit the undo button if they don't like the result. It's a tool for "expert-in-the-loop" conservation, meaning the human expert stays in charge, using the computer as a helpful, but strictly supervised, assistant. The authors admit that while this works well for the specific paintings they tested, the system still needs human review and might need tweaking for different types of silk or lighting. But for now, it offers a way to heal these fragile pieces of history without letting the robot rewrite the story.

Technical Summary: Identification and Graded Restoration of Composite Damage in Silk Paintings under Semantic Constraints

Problem Definition
Traditional Chinese silk paintings face complex degradation issues, including reticular cracks, pigment exfoliation, sedimentary stains, vertical aging stripes, and background oxidation. A critical challenge in digital restoration is the entanglement of damage features with original artistic details. Dark cracks often mimic ink strokes; pigment loss resembles pale clothing; and red seal boundaries can be confused with red pigment degradation. Existing methods face three primary bottlenecks:

Semantic Entanglement: Generic generative models often overwrite authentic brushwork, inscriptions, and seals in pursuit of visual continuity.
Scale and Heterogeneity: A single image may contain mixed degradation types (fine cracks, large stains, oxidation) requiring different restoration strategies, which unified approaches fail to address adaptively.
Lack of Reversibility: Most end-to-end pipelines lack post-restoration quality auditing or rollback mechanisms, conflicting with conservation principles of minimal intervention and reversibility.

Methodology
The authors propose a semantic-protection and quality-gated multi-stage digital restoration framework. The process is designed as an interpretable, auditable workflow rather than a single-step automatic beautification tool.

Semantic Protection Construction:
Before damage detection, the system establishes hierarchical protection masks.
- Absolute Protection ( $P_{abs}$ ): Text and collector seals are strictly protected and excluded from automatic restoration.
- Soft Protection ( $P_{soft}$ ): Figure boundaries, fine line drawings, and high-risk pigment areas are assigned protection penalties.
- Conflict Resolution: When damage candidates overlap with protected regions, the protection priority (Text/Seal > Line Drawing/Pigment > Background) determines the outcome, ensuring cultural semantic safety precedes image generation.
Interpretable Multi-Source Feature Extraction:
Instead of relying on black-box candidate generation, the method uses an interpretable feature set including grayscale, HSV, LAB, local statistics, gradients, and multi-scale morphological responses (BlackHat and TopHat). This allows for the identification of specific damage types (e.g., dark cracks vs. bright pigment loss) and provides diagnostic evidence for algorithmic failures.
Zoned Damage Candidate Generation:
- Background: Targets cracks, stripes, and oxidation spots using color modeling and morphological filtering, with specific constraints to avoid treating natural silk fiber textures as damage.
- Figure Regions: Employs a conservative scoring system for stains and pigment loss, balancing positive evidence (darkening, low saturation) against protection penalties (stroke response, line-drawing density) to prevent misclassifying original artistic details as damage.
Connected-Component Adaptive Routing:
Detected damage candidates are decomposed into connected components and routed to one of three paths based on geometric scale (area, mean width), detail density, and semantic location:
- Skip/Review: Triggered if overlap with protection regions is high or detail density is excessive. These regions remain unaltered for manual review.
- Harmonic Restoration: Applied to narrow cracks with clear boundaries to preserve edge stability without texture drift.
- Deep Generative Inpainting: Applied to large-area, low-detail losses (e.g., background oxidation) using a pre-trained LaMa model. Crucially, the model is not fine-tuned on silk data; its generative freedom is strictly bounded by the routing admission criteria and subsequent constraints.
Constrained Fusion and Quality Gating:
- Fusion: Restoration results are blended with the original image using spatial weights and local color-statistical alignment to prevent luminance steps and color drift. High-frequency texture from trusted regions is re-injected to maintain the material feel of silk.
- Quality Gate: A post-restoration audit evaluates boundary luminance discontinuity, high-frequency energy ratios, and gradient residuals.
- Soft Rollback: If a component fails quality checks (e.g., seam detection or over-smoothing), the system triggers a "soft rollback," reducing the fusion weight or reverting the component to a conservative path. This ensures the process is reversible and auditable.

Key Contributions

Semantic-First Paradigm: Establishes absolute and soft protection masks for high-value semantic regions (text, seals, line drawings) before damage detection, prioritizing cultural heritage safety over visual completion.
Interpretable Feature System: Develops a multi-source feature set that converts algorithmic decisions into verifiable evidence, allowing for iterative calibration in small-sample scenarios.
Adaptive Routing Mechanism: Introduces a component-wise routing strategy that dynamically assigns restoration paths (skip, harmonic, or deep inpainting) based on geometric and semantic attributes, avoiding a "one-size-fits-all" approach.
Quality-Gated Rollback: Implements a risk-alert mechanism that supports local soft rollback, aligning the digital workflow with the conservation principles of reversibility and expert-in-the-loop review.

Experimental Results
The framework was validated on 12 real silk figure-painting images covering diverse materials (gold grounds, light silk) and damage types.

Coverage Statistics: Figure regions accounted for an average of 36.77% of the image area. The final candidate restoration regions averaged 24.32%, indicating substantial damage coverage.
Quality Metrics: In a no-reference evaluation of three samples, the mean gradient preservation ratio was 0.9824, indicating stable structural constraints. The ratio of intra-mask variation to extra-mask leakage was approximately 5.4:1, confirming that modifications were concentrated within candidate regions with minimal disturbance to protected areas.
Ablation Studies: Removing the text/seal protection module (A1) caused text/seal disturbance to increase by 9.0 times. Removing the large-area pigment loss detection (A4) resulted in zero detections for that damage type in 10/11 images, proving functional completeness. The quality gate (A6) was shown to intercept low-probability but unacceptable failures, even if average metrics showed limited change.
Comparative Analysis: Compared against recent deep inpainting baselines (LaMa, LRDiff, BrushNet, PowerPaint) under a "given-mask" setting, the proposed method demonstrated superior preservation of seals, inscriptions, and fine line drawings, avoiding the hallucination of strokes and over-smoothing seen in generative models.

Significance and Claims
The paper claims that the proposed method offers a controllable, interpretable, and rollback-enabled restoration paradigm suitable for expert-in-the-loop cultural heritage conservation.

It addresses the tension between effective damage removal and the preservation of historical evidence.
It does not claim to replace manual restoration but to provide a digital support tool for condition recording, damage analysis, and decision-making.
The authors explicitly state that the method is not intended as a one-step automatic beautification tool. Instead, it provides a structured workflow where decisions are traceable, and risks are mitigated through semantic constraints and quality gating.
The study acknowledges limitations, including the lack of expert-annotated ground truth for training and the reliance on fixed thresholds for routing and quality gating, which may require recalibration for different materials or acquisition conditions. Future work is directed toward building expert-annotated datasets and developing interactive review interfaces.

Technical Summary: Identification and Graded Restoration of Composite Damage in Silk Paintings under Semantic Constraints

More like this