Technical Summary: Digital Reconstruction of Lacquer Art Surfaces Using Computer Vision and Generative Models

1. Problem Statement

The digital preservation of Chinese lacquer art faces a critical challenge: existing restoration methods often fail to capture the unique materiality of lacquerware. Traditional physics-based simulations are computationally expensive and lack the flexibility to model the organic, intuitive responses of master craftsmen. Conversely, deep learning-based generative methods (e.g., GANs, standard diffusion models) tend to prioritize global statistical style transfer, resulting in smooth textures that lose crucial micro-details such as sharp crackle patterns (duanwen), layered material thickness, and variable gloss.

Current approaches struggle to simultaneously satisfy three core requirements for high-fidelity cultural restoration:

1. Structural Consistency: Precise retention of crack networks, contour edges, and geometric layout.
2. Texture Authenticity: Reproduction of complex, multi-scale textures and the stochastic aging features inherent to lacquer.
3. Stylistic Coherence: Adherence to the specific aesthetic and cultural essence of lacquer art without introducing "hallucinated" artifacts.

Furthermore, there is a lack of a quantifiable evaluation system for "cultural authenticity," leading to results that may look visually plausible but lack historical or craft accuracy.

2. Methodology: Structure-Aware Collaborative Generative Framework (SACGF)

The paper proposes the Structure-Aware Collaborative Generative Framework (SACGF), a task-oriented pipeline built upon Stable Diffusion (SD) that integrates three distinct modules to inject explicit priors into the generative process. The framework treats lacquer surface reconstruction as a multi-objective optimization problem balancing structural consistency, texture fidelity, and style coherence.

2.1 Data Construction

A specialized dataset of 2,194 high-resolution lacquer art images was constructed, covering 8 major categories and 32 subcategories (e.g., carved lacquer, gold inlay, crack patterns). To address the scarcity of paired damaged-intact data, the authors generated paired samples via degradation simulation, creating synthetic training pairs that mimic structural damage (flaking, cracks) and surface degradation (fading, gloss loss).

2.2 Framework Architecture

The SACGF pipeline operates through a six-stage process:

1. Structure-Aware Feature Extraction:

   • Edge/Crack Priors: Multi-scale Canny edge detection and crack-density estimation extract geometric contours and crack networks.
   • Texture Priors: Multi-scale texture descriptors are extracted using VGG-19 and Gray-Level Co-occurrence Matrix (GLCM) statistics.
   • Semantic Priors: The Segment Anything Model (SAM) generates fine-grained semantic masks (e.g., separating gold inlay from crack regions). These masks undergo manual refinement by an expert to ensure accuracy at material boundaries.

2. Condition Encoding & Spatial Transformation:

   • Extracted priors are encoded into condition tensors.
   • A self-trained, multi-condition ControlNet (trained specifically on the lacquer dataset) processes edge maps, depth maps, and color maps.
   • SAM masks are transformed into spatial constraints in the latent space.

3. Guided Restoration (Generative Core):

   • LoRA (Low-Rank Adaptation): A LoRA adapter is fine-tuned on the lacquer dataset to inject domain-specific style parameters into the Stable Diffusion backbone, ensuring the generated texture matches the "lacquer" aesthetic.
   • ControlNet Guidance: The trained ControlNet provides structural constraints (edge, depth, color) to guide the denoising process, preventing structural distortion.
   • SAM Semantic Constraints: Semantic masks act as "fences," strictly confining restoration updates to damaged regions and preventing texture/color leakage across different material boundaries (e.g., preventing gold texture from bleeding into black lacquer).

4. Optimization Objective:
   The framework minimizes a composite loss function:

   • Structure Loss: Combines SSIM and edge fidelity (Canny) to preserve contours.
   • Texture Loss: Combines LPIPS and GLCM statistics to ensure realistic material texture.
   • Style Loss: Uses Gram matrix matching to align color distribution and stylistic features.

2.3 Evaluation Metric: Digital Lacquer Fidelity Score (DLFS)

To move beyond subjective assessment, the paper introduces the Digital Lacquer Fidelity Score (DLFS). This composite metric aggregates:

• Texture Conformity: LPIPS and GLCM statistics (contrast, correlation, energy, homogeneity).
• Color Fidelity: CIEDE2000 color difference and Lab color space consistency.
• Structural Similarity: SSIM, Multi-scale SSIM, and Edge Fidelity.
  These metrics are normalized and aggregated into a single score to provide an objective, reproducible standard for cultural fidelity.

3. Key Contributions

1. SACGF Framework: A novel collaborative system integrating LoRA, ControlNet, and SAM to achieve high-fidelity restoration that balances structural, textural, and stylistic requirements.
2. Specialized Dataset & Mechanism: The construction of a 2,194-image dataset with simulated damaged-intact pairs and a human-in-the-loop refinement process for semantic masks, tailored specifically for lacquer restoration.
3. Quantifiable Cultural Fidelity: The proposal of the DLFS evaluation system, which operationalizes "cultural authenticity" into measurable physical statistics, offering a pathway to evaluate AI restoration beyond visual resemblance.

4. Experimental Results

Experiments were conducted on a test set of 220 images (including 47 real-world degraded artifacts).

• Quantitative Performance: SACGF achieved a DLFS of 0.87, significantly outperforming baselines:
  • CycleGAN: DLFS 0.31
  • SD+LoRA: DLFS 0.52
  • SD+LoRA+Canny+Depth: DLFS 0.76
  • SACGF also achieved the highest PSNR (27.42), SSIM (0.905), and lowest LPIPS (0.062) and CIEDE2000 (4.18).
• Ablation Studies:
  • Edge Conditions (Canny): Primarily strengthened structural preservation (SSIM +0.037, EdgeF +0.088).
  • Depth Maps: Improved layer consistency and spatial hierarchy, particularly for carved lacquer.
  • SAM Masks: Crucially reduced cross-region texture leakage and color bleeding, significantly improving LPIPS and color fidelity.
• Visual Analysis: The framework successfully reconstructed complex crack networks (e.g., snake-belly, ox-hair cracks) and restored gold inlay details without distorting surrounding intact areas, whereas baselines often produced blurred edges or "hallucinated" decorative patterns.

5. Significance and Claims

The paper claims that high-fidelity cultural reconstruction cannot rely on a single "black-box" model. Instead, it requires a collaborative system where structure, texture, and semantics are injected as explicit priors.

• Paradigm Shift: The study argues for a shift from "visual resemblance" (generating an image that looks like lacquer) to "cultural fidelity" (reconstructing a surface that feels like a lacquer object, respecting material logic and craft history).
• Practical Utility: The framework offers a reversible, virtual restoration tool for conservators to evaluate intervention strategies before physical work, and provides a quantifiable pathway to assess the authenticity of AI-generated heritage content.
• Limitations Acknowledged: The authors modestly note that the system struggles with:
  • Overlapping Decorations: Difficulty distinguishing 3D stacking orders (e.g., gold foil vs. cracks) in 2D images.
  • Severe Degradation: Risk of "hallucinating" historically inaccurate patterns when damage exceeds 40% and structural cues are sparse.
  • Domain Gap: Performance drops slightly when moving from synthetic training data to real-world artifact images, indicating a need for future domain adaptation.

In conclusion, the paper presents SACGF as a reproducible, structure-aware approach that bridges the gap between generative AI and the rigorous demands of cultural heritage preservation, providing both a technical framework and an evaluation standard for future research.