Technical Summary: Morphological Priors and Photogrammetric Conditioning for Auditable Generative 3D Miao and Dong Timber Heritage Scenes

1. Problem Statement

Generative AI systems can produce plausible heritage scenes, but their evidential status remains weak when cultural prompts, structural references, 3D generation, and visualization are mixed without clear separation. A "plausible" image is not automatically a documented heritage claim. Specifically, generating Miao-Dong timber heritage scenes is more complex than reconstructing a single facade; it requires preserving scene-level morphology (relationships between drum towers, wind-rain bridges, stilted dwellings, and spatial contexts) while avoiding drift into generic old-town or tourist-street imagery.

The core challenge is establishing an auditable workflow that separates cultural semantics, structural conditioning, asset transfer, and provenance. Current methods often lack the ability to distinguish between measured records, interpretive visualizations, and experimental generated assets, making it difficult for reviewers to assess what specific evidence type proves.

2. Methodology: The SSAT Framework

The study proposes a Semantic-Structure-Asset Transfer (SSAT) framework. This is not a single neural network model but a structured workflow that decomposes the generation process into interpretable, controllable, and scoreable units. The framework operates through four distinct evidence layers:

2.1 Semantic Prior (MASG and LoRA)

• Semantic Structure Asset (MASG): A working semantic graph that links building types, timber components, roof/material cues, spatial contexts, and drift categories. It converts heritage knowledge into a constrained prompt grammar (e.g., encoding a "wind-rain bridge" not just as a bridge, but as a covered timber corridor with specific railings and roof forms).
• LoRA05 (Regional Visual Prior): A Low-Rank Adaptation (LoRA) model trained on a Miao-Dong image inventory. It serves as a regional visual prior to inject domain-specific texture and style without full retraining of the base model.
• Ablation: The study compares generic prompting, taxonomy prompting, MASG prompting, and combinations with LoRA to determine the optimal semantic control.

2.2 Structural Conditioning (Photogrammetry)

• Source: Photogrammetry-derived structural maps from Qingyan, a mixed Guizhou historic/vernacular timber settlement.
• Role: Qingyan is explicitly not treated as Miao-Dong cultural ground truth. Instead, it provides structural and morphological cues (roof rhythm, lane boundaries, node-space organization) that overlap with Miao-Dong architecture.
• Mechanism: Using ControlNet-style logic, structural maps (Canny, Depth, Lineart) are used as image conditions to anchor the generation. Negative controls (shuffled maps, wrong-region maps) test whether the improvement comes from relevant morphology or merely the presence of a condition map.

2.3 Asset Transfer and Provenance (Marble/Broker)

• Pipeline: Generated 2D references are transferred to an external 3D asset workflow (Marble) to produce SPZ, GLB, and PLY files.
• Broker Layer: A provenance logging system that records prompts, references, job IDs, file paths, hashes, validation outcomes, and failure states. It treats the 3D backend as a "black box" but ensures the observable input-output chain is auditable.
• Failure Handling: The system is designed to detect and log failure states (e.g., missing files, corrupted assets) rather than silently passing incomplete data downstream.

2.4 Evaluation Protocol

• Metrics:
  • 2D Stage: FID (distributional proxy), CLIPScore (prompt-image alignment), Edge F1, Depth Rank, and Luminance MSE.
  • 3D Stage: Asset manifests, fixed-view renderings, and blinded mixed-effects ratings.
• Human Evaluation: 86 raters (16 experts, 70 non-experts) evaluated fixed-view renders under blinded conditions. A mixed-effects model was used to account for repeated measures from respondents and scene blocks.

3. Key Contributions

1. Operationalization of Morphology: The paper operationalizes Miao-Dong scene morphology through a semantic graph (MASG) that links building types, components, and spatial relations to prompt tokens and evaluation dimensions.
2. Separation of Evidence Types: It tests LoRA (semantic prior) and photogrammetry-derived structural conditioning as separate mechanisms, demonstrating that they serve different roles (style/semantics vs. spatial structure).
3. Auditable 3D Transfer: It evaluates the transfer of paired scene blocks into 3D assets (C1–C4 strategies) using asset manifests and blinded ratings, moving beyond 2D image generation to inspectable 3D research objects.
4. Provenance and Fault-State Logging: It introduces a Broker layer that records success, warning, and failure states, allowing generated assets to be reviewed as research evidence rather than isolated renderings.

4. Results

• Semantic Control (Experiment A): The combination of LoRA05 + MASG (A4) yielded the strongest semantic prior, achieving the highest scores for building type, timber detail, and drift control. MASG provided the necessary grammar, while LoRA provided the regional visual style.
• Structural Conditioning (Experiment B): Lineart (B4) provided the best balance between structure and style preservation. Shuffled (B5) and wrong-region (B6) controls performed poorly, confirming that structural conditioning must be morphologically relevant to be effective.
• 3D Asset Transfer (Experiment C):
  • C4 (LoRA05 + MASG + Structure-guided) significantly outperformed all other strategies (C1: text-only, C2: generic reference, C3: LoRA+MASG).
  • Statistical Significance: C4 achieved a mean rating of 4.164 compared to C3's 3.725 (p < 0.001). This gain was consistent across all eight tested scene blocks.
  • Blind Evaluation: Both expert and non-expert raters preferred the structure-guided approach, indicating improved visual usability and spatial coherence.
• Provenance (Experiment E): The Broker successfully localized diagnostic stages for failures (e.g., unreachable APIs, corrupted assets), reducing manual diagnostic steps by 3–6 steps per case and preventing invalid assets from entering downstream visualization.

5. Significance and Claim Boundaries

The paper positions itself as an evidence-chain study rather than a demonstration of a generative tool. Its primary significance lies in making generated heritage scenes reviewable by separating evidence into semantic, structural, asset, and provenance layers.

Explicit Claim Boundaries:

• Not Survey-Grade: The study explicitly does not claim to reconstruct a measured Miao-Dong settlement or produce survey-grade 3D models.
• Not Cultural Ground Truth: The Qingyan photogrammetry data is used strictly as a structural/morphological reference, not as a ground truth for Miao-Dong cultural identity.
• Not Deterministic: The workflow does not claim deterministic generation or full backend reproducibility (due to the "black box" nature of external 3D tools like Marble).
• Purpose: The generated assets are intended for early-stage heritage exploration, design communication, and interpretive visualization, not as a substitute for field surveys, conservation diagnosis, or expert heritage approval.

The study concludes that structure-guided cultural-reference transfer improves fixed-view visual interpretability and structural readability. By documenting what each evidence type supports and what it does not, the framework offers a responsible approach to using generative AI in heritage science.