Auditable AI Assisted Semantic Completion Supports Metadata Enhancement in Digital Cultural Heritage Collections

Technical Summary: Auditable AI Assisted Semantic Completion for Digital Cultural Heritage Metadata

Problem Statement
Digital cultural heritage collections are increasingly available as computational data, yet their discoverability is hindered by metadata that is often incomplete, unevenly structured, and lacks semantic richness. While Artificial Intelligence (AI) is frequently proposed as a solution for metadata generation, its adoption in cultural heritage contexts raises significant concerns regarding interpretive authority, cultural bias, and the "black-box" nature of generative models. The core challenge is not merely whether AI can generate labels, but how to create a metadata enhancement workflow that is transparent, reproducible, and auditable by professional cataloguers. Existing records often contain dense information in some facets (e.g., object type, material) but are sparse in others (e.g., subject, classification), creating a need for a system that can recommend missing controlled-vocabulary terms without inventing new ones or obscuring provenance.

Methodology
The study reframes metadata enhancement as a constrained semantic completion task rather than a generative one. Instead of asking a model to produce fluent descriptions, the system uses observed controlled-vocabulary labels within a record to recommend candidate labels for missing facets.

• Dataset: The research utilizes the EUFCC-340K benchmark, a faceted hierarchical dataset derived from Europeana containing over 340,000 records. The study employs a fixed training subset of 25,000 records to ensure reproducibility on ordinary hardware.
• Experimental Design: A masked-label evaluation protocol is used. For each eligible record, one existing label from a target facet (object type, material, subject, or classification) is hidden. The model must rank candidate labels for that facet using the remaining observed labels as context. Success is measured by whether the hidden label appears in the top k candidates (Recall@k).
• Models Evaluated:
  1. Frequency Baseline: Ranks candidates based on global training frequency within the target facet.
  2. Direct Co-occurrence: Builds a label-label co-occurrence matrix from training records. Scores are derived from the mean probability of a candidate appearing given the observed context labels.
  3. Two-Hop Graph Propagation: Extends direct evidence by propagating scores through the label graph (e.g., Object $\to$ Material $\to$ Classification) to capture indirect relations.
  4. Hybrid Ranking: A weighted combination of the direct score, two-hop score, and frequency prior.
• Constraints: The vocabulary is capped at 600 leaf labels (those appearing $\ge$10 times in training) to ensure auditability and efficiency. The workflow prioritizes deterministic, reproducible ranking over generative novelty.

Key Results

• Metadata Asymmetry: Analysis of the EUFCC-340K dataset reveals significant asymmetry in metadata coverage. Object-type and material fields are dense (approx. 77–93% coverage), while subject and classification fields are sparse (6.7% and 12.9% coverage in the training split, respectively).
• Performance of Co-occurrence: On the inner-test split, the direct co-occurrence model substantially outperformed the frequency baseline across all facets.
  • Object Types: Recall@5 improved from 0.190 (frequency) to 0.732 (co-occurrence).
  • Materials: Recall@5 improved from 0.379 to 0.790.
  • Classifications: Recall@5 improved from 0.450 to 0.986.
  • Subjects: While the test set for subjects was small (228 cases), the frequency baseline achieved a Recall@5 of 1.000, which was matched by the co-occurrence model (also 1.000). However, co-occurrence significantly outperformed the frequency baseline in Recall@1 (0.934 vs. 0.272) and Recall@3 (1.000 vs. 0.781), demonstrating superior precision in ranking the correct label at the top of the list.
• Model Complexity: The hybrid model did not consistently outperform the simpler direct co-occurrence model. The study found that the simplest transparent method was often the strongest, suggesting that complex graph propagation or hybrid priors may not be necessary when direct label relations are strong.
• Distribution Shift: The outer-test split (representing different collection compositions) showed that frequency baselines can appear deceptively strong in low-entropy distributions, whereas direct co-occurrence remains sensitive to relational shifts. This highlights the need for split-specific evaluation rather than aggregate scores.

Significance and Claims
The paper claims its primary contribution is a reproducible information-organization workflow that treats AI-assisted metadata enhancement as an auditable ranking problem.

• Auditability and Governance: The study argues that high-value AI in cultural heritage should be judged by reproducibility, interpretability, and provenance preservation, not just predictive accuracy. By using observed co-occurrence, every recommendation can be traced back to specific training data relations, allowing professional cataloguers to inspect, validate, or reject suggestions.
• Human-in-the-Loop Design: The workflow positions AI not as a replacement for cataloguers but as a decision-support layer. It provides ranked candidates for human validation, reducing the cost of review while preserving professional judgment.
• Practical Implementation: The approach requires modest technical resources (no large neural networks, image processing, or proprietary APIs), making it accessible for smaller institutions. It supports a staged implementation pathway: metadata profiling, vocabulary normalization, graph construction, candidate suggestion, and provenance recording.
• Limitations: The author explicitly acknowledges that the method inherits dataset biases (amplifying existing cataloguing imbalances) and does not assess cultural appropriateness or the necessity of adding missing labels in real-world scenarios. The study focuses on the recovery of existing labels within a controlled vocabulary, not the generation of new cultural interpretations.

In conclusion, the paper demonstrates that transparent, co-occurrence-based ranking can effectively support the enrichment of digital cultural heritage metadata, offering a conservative but scalable alternative to opaque generative AI models.