Multiband 3D Texture-based Change Detection for Monitoring Cultural Heritage

Technical Summary: Multiband 3D Texture-based Change Detection for Monitoring Cultural Heritage

Problem Statement
While multiband imaging (MBI) has become a standard non-invasive tool for analyzing Cultural Heritage (CH) materials, its application to active, long-term monitoring of conservation status remains under-explored. Current monitoring practices typically rely on 2D orthophotos or photographs acquired at different epochs (T1 and T2). These 2D approaches suffer from significant limitations when applied to objects with complex three-dimensional geometries (e.g., statuary), as they require independent generation of orthophotos for each epoch followed by a posteriori co-registration. This process often introduces spatial misregistration and geometric distortions, compromising the reliability of pixel-to-pixel comparisons. Furthermore, existing multiband alignment pipelines often focus on aligning multiple spectral bands for a single temporal state, failing to extend this precision to multi-epoch datasets for temporal change detection. There is a methodological gap in bridging high-resolution 2D documentation with long-term 3D diagnostics to objectively detect and track surface variations.

Methodology
The study proposes a semi-automated workflow that integrates multi-temporal, multiband photogrammetry with automated change detection. The core of the methodology involves the following steps:

1. Data Acquisition: Two fir wood mock-ups (Test-objects A and B) were subjected to controlled mechanical and chemical degradation. Data was acquired at two epochs (T1 and T2) using a full-spectrum Nikon D810 DSLR under uncontrolled ambient lighting. Four spectral bands were captured: Visible Light (VIS), Ultraviolet Reflectance (UVR), Ultraviolet-induced Luminescence (UVL), and Near-Infrared Reflectography (NIR).

   • Signal Optimization: To address low signal-to-noise ratios in UV bands under ambient light, the authors employed a high-power strobe system and the camera's "Multiple Exposure" function, averaging ten consecutive RAW frames to reduce noise without increasing ISO-induced grain.
   • Acquisition Strategy: The workflow decouples the acquisition into two datasets: a dense "Morphology" set (VIS only) for 3D mesh reconstruction and a sparse "MBI" set (all bands) strictly for texturing. This reduces acquisition time while maintaining geometric accuracy.

2. 3D Co-registration and Texture Projection:

   • A single reference 3D mesh is generated from the T1 morphological dataset.
   • The T2 VIS dataset is aligned to this T1 reference.
   • All multiband images (UVR, UVL, VIS, NIR) from both epochs are co-registered to their respective VIS counterparts using the "MIRA" (Multiband Image Registration Assistant) software.
   • Crucially, all spectral textures from both epochs are projected onto the single shared 3D mesh using a unified UV map. This ensures intrinsic pixel-to-pixel alignment across time and spectral bands, eliminating the registration errors associated with 2D overlay methods.

3. Automated Change Detection Pipeline:

   • Preprocessing: Radiometric normalization is applied using reference targets (ColorChecker, white chalk, UV target).
   • Feature Extraction: The four spectral bands (RGB channels each) are stacked into a 12-dimensional data matrix. Principal Component Analysis (PCA) is applied to the T1 (baseline) dataset to capture 97.5% of the variance.
   • Difference Calculation: The T2 dataset is projected onto the T1 PCA baseline, and a multi-temporal difference matrix is computed via pixel-wise subtraction.
   • Clustering: Unsupervised K-means clustering is applied to the difference data. An interactive step allows conservators to validate and select specific clusters corresponding to alterations, discarding the baseline cluster.
   • Spectral Classification: Remaining pixels are classified based on dominant spectral responses (NIR, VIS, UVL, UVR) and color-coded to create a diagnostic map.

4. Validation: The pipeline was validated against manual condition reports (ground truth) using two modalities:

   • Orthophotos: Continuous planar projections for initial benchmarking.
   • 3D Textures: Unwrapped maps re-projected onto the 3D mesh to evaluate performance on the actual 3D geometry.

Key Results

• Orthophoto Performance: The automated pipeline demonstrated high global accuracy, achieving 91.96% for Test-object A and 92.37% for Test-object B. Specificity was exceptionally high (>95%), indicating a low rate of false positives on stable areas.
• Texture-Based Performance: When applied to the 3D-extracted textures, Test-object A maintained high performance (92.32% accuracy). However, Test-object B showed a decline in localized metrics (precision dropped to 54.02%, recall to 42.68%).
• Analysis of Discrepancies: The authors attribute the performance drop in Test-object B to the limitations of the 3D mesh topology. The algorithm evaluates the entire mesh, including lateral facets where data quality was lower due to the acquisition focus on the top plane. This introduced spurious variance that skewed the clustering logic, generating false positives on the sides and affecting the primary top plane.
• Detection Capabilities: The system successfully detected structural alterations with distinct boundaries (lacunae, impact fractures) and chemical changes with high contrast (superficial whitening, fluid drips visible in UVL). It struggled with gradual, low-contrast changes like pigment fading and thin linear features like scratches, which blended into the baseline.

Significance and Claims
The paper claims that this workflow successfully bridges the gap between qualitative condition reporting and objective analytical assessment for Cultural Heritage monitoring. Its primary significance lies in:

1. Robust 3D Alignment: By projecting multi-epoch multiband data onto a single shared 3D geometry, the method eliminates the geometric distortions and registration errors inherent in traditional 2D comparison methods, making it applicable to complex 3D objects.
2. Quantitative Diagnostics: The approach enables the automatic calculation of the physical surface area of alterations using the model's Ground Sample Distance, providing a metric-based tool to track degradation spread.
3. Operational Efficiency: The workflow does not aim to replace the conservator but to streamline the detection phase. It provides an initial analytical tool to rapidly identify macro-areas of alteration, reducing manual workload and labor costs in restoration estimates.
4. Limitations and Scope: The authors modestly note that the method is exclusively suited for monitoring superficial material conservation status. It does not account for macroscopic structural deformations or volumetric losses, for which mesh-to-mesh comparison remains the reference approach. Future work is identified as necessary to refine clustering parameters for geometrically complex artifacts and to better handle edge-effect noise on lateral facets.