💻 computer science

Explainable Multimodal AI for Adaptive Calibration of Archaeological Sensing Workflows

This paper presents an explainable multimodal AI framework that monitors, assesses, and adaptively calibrates diverse archaeological sensing workflows by integrating deterministic indicators and machine learning to detect acquisition degradation and guide corrective actions.

Original authors: Nevio Dubbini, Daniel P. van Helden, Claudia Sciuto, Martina Naso, Arthur Leck, Clement Joubert, Heeli C. Schechter, Remy Chapoulie, Gabriele Gattiglia

Published 2026-08-04

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Nevio Dubbini, Daniel P. van Helden, Claudia Sciuto, Martina Naso, Arthur Leck, Clement Joubert, Heeli C. Schechter, Remy Chapoulie, Gabriele Gattiglia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a time traveler with a magical camera, but instead of taking photos of people, you are snapping pictures of ancient pottery, rocks, and tools left behind by civilizations long gone. To really understand these treasures, you don't just want a pretty picture; you need to know what they are made of, how they were shaped, and even what chemicals are hiding inside them. Scientists use special "super-senses" to do this: one takes 3D pictures, another sees invisible colors, one checks for hidden metals, and another listens to the tiny vibrations of molecules. But here's the catch: these super-senses are finicky. If the light is wrong, the camera is shaky, or the machine is tired, the data can be garbage. It's like trying to bake a perfect cake but your oven thermometer is broken, your flour is wet, and you can't tell if the cake is actually done or just a sad, flat pancake.

This is where the problem gets tricky. In the world of archaeology, checking if a scan is "good" usually requires a human expert to stare at the data and say, "Hmm, that looks weird." But there are so many artifacts to scan that humans can't check every single one. The big question is: Can we teach a computer to be the quality inspector? Can we build a system that doesn't just take the data, but actually understands why a scan might be bad and suggests how a robot should try again? This paper dives into that exact challenge, proposing a smart, "explainable" AI system that acts like a super-observant guardian for these ancient scanning missions.

The Digital Detective for Ancient Treasures

The authors of this paper, a team of tech-savvy archaeologists and computer scientists, have built a new kind of "digital detective." Their goal is to create a safety net for the automated scanning of archaeological artifacts. Instead of just letting a robot take a picture or a spectrum and hoping for the best, this system acts as a strict gatekeeper. It checks the data the moment it's collected to see if it's trustworthy enough to be used for real science.

Think of the scanning process like a band playing a concert. You have a 3D camera (the drummer), a hyperspectral camera (the guitarist), an X-ray machine (the bassist), and a Raman spectrometer (the singer). Each instrument needs to be perfectly tuned. If the drummer is out of time, the whole song sounds off. In the past, if the drummer messed up, you might not realize it until the concert was over and the recording was ruined. This new framework is like a conductor who listens to every instrument while they are playing. If the guitar is slightly out of tune, the conductor doesn't just say "stop"; they provide a detailed note saying, "Guitarist, your tuning peg needs adjustment," so the musician can fix it before the next song.

How the System Works: The Six-Step Dance

The paper describes a six-step pipeline that every piece of data has to go through. It's a bit like a high-tech security checkpoint at an airport, but instead of checking for knives, it's checking for "bad data."

The Reference Check: Before the robot starts scanning the ancient pots, it takes a few "test shots" of known, perfect samples. This sets the baseline, like tuning a guitar before a show. The system learns what "perfect" looks like for that specific session.
The Feature Translation: Every scan is then translated into a long list of numbers (a "feature vector"). These numbers describe everything: how bumpy the 3D shape is, how bright the colors are, or if the chemical signals are noisy. It's turning a complex image or sound into a simple report card.
The Detective Work: The system looks at these numbers to see if anything looks suspicious. Is the noise level too high? Are the shapes weirdly distorted? It uses statistics to spot patterns that don't fit the "perfect" baseline.
The Judgment Call: This is where the AI steps in. It uses machine learning models (smart algorithms trained on past examples) to decide: Is this a "Good" scan or a "Bad" scan? They tested different types of AI brains for different sensors. For the 3D models, a simple "Logistic Regression" brain worked best. For the complex chemical data (like X-rays and hyperspectral images), they needed smarter "tree-based" brains like CatBoost and XGBoost to handle the messy, non-linear patterns.
The "Why" Explanation (XAI): This is the most exciting part. Usually, AI is a "black box"—it says "Bad," but you don't know why. This system uses "Explainable AI" (XAI) to open the box. It tells the operator exactly what went wrong. Did the light flicker? Was the camera too far away? Was the signal too weak? It's like a teacher not just giving you an "F," but circling the exact math problem you got wrong and explaining how to fix it.
The Feedback Loop: Finally, the system gives instructions. It can say: "This is fine, keep going," "This is okay, but let's tweak the settings and try again," or "This is terrible, stop and get a human to look at it." Currently, this acts as a decision-support tool for the operator; the system identifies the issue and suggests the fix, but the actual automatic adjustment of the robot is a goal for future development.

The Results: A Mixed Bag of Success

The team tested this system on real-world data from the AUTOMATA project, which involves scanning ancient ceramics and stones. They didn't just use perfect lab conditions; they used data that included real-world messiness, like operator mistakes and changing light.

The results were promising, though not perfect.

For 3D Models: The system correctly identified good vs. bad 3D scans 80% of the time. The AI noticed that if the triangles in the 3D mesh were too skinny or the edges were broken, the scan was likely a failure.
For Hyperspectral Imaging (HSI): This was a big win. The system achieved an accuracy of 86% and a very strong score of 0.92 (called AUC), meaning it was excellent at spotting bad spectral data. It could tell if the light was inconsistent or if the sensor was acting up.
For X-Ray Fluorescence (XRF): The system was very good at spotting bad X-ray data, with an accuracy of 94%. It learned to look for things like low signal counts or weird background noise that would make the chemical analysis useless.
For Raman Spectroscopy: This was the trickiest one. The system got an accuracy of 80%, but it struggled a bit more to catch every single bad scan (lower recall). The main culprit? Fluorescence. Sometimes the ancient materials glow in a way that drowns out the signal, and the AI found this hard to distinguish from a simple bad scan.

What the Paper Says (and Doesn't Say)

It is important to understand what this paper doesn't claim. The authors are very careful to say that this system is not replacing the need for human experts or the need for the machines to be physically calibrated. The AI doesn't fix the machine's hardware; it just tells you if the data coming out is garbage and suggests how to improve it.

The paper also clarifies that right now, the system is mostly a "decision support" tool. It suggests what to do, but it hasn't been fully tested in a closed loop where the robot automatically fixes itself without a human pressing a button. The authors suggest that the current results are "preliminary" for the 3D models because they didn't have a huge amount of data yet. They are hopeful, but they know they need more data, especially from robots doing the scanning automatically, to make the system truly robust and fully automated.

The Big Picture

In simple terms, this paper proposes a way to stop archaeologists from wasting time analyzing garbage data. By using a smart, explainable AI system, they can catch mistakes early, identify the correct settings to fix them, and ensure that every scan of an ancient artifact is as clear and accurate as possible. It's a step toward a future where robots can explore history with the same care and attention to detail that a human expert would, but without getting tired or distracted. The system doesn't just say "this is bad"; it tells you why it's bad and how to make it good, turning a potential disaster into a learning opportunity for the machine.

Technical Summary: Explainable Multimodal AI for Adaptive Calibration of Archaeological Sensing Workflows

Problem Statement

The large-scale digitisation of archaeological artefacts increasingly relies on integrating diverse sensing technologies—specifically photogrammetric 3D reconstruction, hyperspectral imaging (HSI), X-ray fluorescence spectroscopy (XRF), and Raman spectroscopy—to capture geometric, spectral, elemental, and molecular data. While these modalities provide internal calibration procedures, they do not guarantee that individual acquisitions are analytically reliable or suitable for downstream multimodal integration.

Current challenges include:

Modality-Specific Variability: Acquisition "quality" is application-specific and dependent on factors such as operator choice, geometry, illumination, and environmental conditions.
Lack of Unified Standards: No standard exists for assessing quality and error propagation across heterogeneous modalities. Existing research addresses quality assessment and machine learning separately within individual modalities.
Manual Intervention: Degradation patterns (e.g., spectral distortions, reconstruction artefacts, detector instability) are often identified manually, which is not scalable for large automated workflows.
Interpretability Gap: While machine learning (ML) shows potential for spectral classification, its integration into archaeological workflows is limited by a lack of interpretability regarding why an acquisition is deemed poor.

Methodology

The paper proposes a unified, modality-independent machine learning framework designed for automatic calibration monitoring, quality assessment, and adaptive acquisition support. The framework treats calibration as a statistical consistency problem, evaluating acquisitions against empirical reference distributions rather than relying solely on instrumental procedures.

The pipeline consists of six stages:

Reference Stage: Modality-specific reference measurements are acquired to define empirical distributions, tolerance intervals, and confidence bounds for nominal operating conditions.
Feature-Space Stage: Each acquisition is transformed into a structured feature vector combining deterministic quality indicators and statistical descriptors. These capture geometric, spectral, spatial, and statistical properties. Outlier descriptors are derived via Mahalanobis distance and density-based anomaly detection.
Exploratory Stage: Univariate/multivariate statistics, correlation analysis, and dimensionality reduction are used to investigate the feature space structure before classification.
Classification Stage: A supervised learning layer (binary "good" vs. "bad" labels) is applied. Different classifiers are selected based on modality complexity:
- 3D Modelling: Logistic Regression.
- HSI: CatBoost (gradient-boosting trees).
- XRF: XGBoost (identified in text as achieving best performance; Table 3 reports metrics for CatBoost).
- Raman: XGBoost.
  Unsupervised methods (Gaussian Mixture Models, DBSCAN, Isolation Forest, One-Class SVM) are used to detect anomalous or unseen degradation patterns.
Explainable AI (XAI) Stage: Interpretability is achieved using SHAP and LIME to identify which features drive classification decisions, linking statistical deviations to physical causes (e.g., illumination inconsistencies, baseline fluctuations).
Feedback Stage: Acquisitions are assigned to one of four classes: acceptable, automatically correctable, reacquire, or human intervention required. This decision is based on a global severity score $S(x) = \alpha D(x) + \beta A(x) + \gamma U(x)$ $S (x) = α D (x) + β A (x) + γ U (x)$ , combining:
- $D(x)$ : Statistical deviation score (Mahalanobis distance).
- $A(x)$ : Anomaly score (Isolation Forest).
- $U(x)$ : Prediction uncertainty ( $1 - \max(p)$ ).

Key Contributions

Unified Multimodal Framework: The first proposed framework to integrate quality assessment and calibration monitoring across photogrammetry, HSI, XRF, and Raman spectroscopy within a single pipeline.
Statistical Consistency Approach: Reinterprets calibration as a statistical consistency problem, evaluating acquisitions against reference distributions rather than just instrumental checks.
Explainable Decision Support: Integrates XAI to provide interpretable diagnostic information, linking specific feature-space deviations (e.g., triangle aspect ratios, fluorescence background) to actionable corrective measures.
Adaptive Feedback Mechanism: Establishes a mapping between feature-space deviations and acquisition-level corrective actions (e.g., adjusting illumination, exposure time, or geometry), laying the groundwork for closed-loop robotic calibration.

Experimental Results

The framework was validated on multimodal datasets from the AUTOMATA project, comprising manually acquired data with intentional variability to simulate real-world conditions. The datasets included:

3D Models: 51 reconstructions (ceramics and lithics).
HSI: 513 artefacts (421 ceramics, 92 lithics) using a Specim IQ camera.
XRF: 941 measurements from 159 artefacts using Olympus Vanta C-series devices.
Raman: 201 measurements from 56 artefacts using a Metrohm i-Raman Plus 785S.

Performance Metrics:

3D Modelling (Logistic Regression): Accuracy 0.80, AUC 0.83. SHAP analysis identified triangle aspect ratios and boundary-edge statistics as key drivers.
HSI (CatBoost): Accuracy 0.86, AUC 0.92. The model effectively separated decision classes based on spectral and spatial consistency.
XRF: The text states that XGBoost achieved the best performance; however, Table 3 reports metrics for a CatBoost model (Accuracy 0.94, AUC 0.93).
Raman (XGBoost): Accuracy 0.80, AUC 0.83. Classification was dominated by fluorescence-related descriptors and peak sharpness metrics.

Note: The authors note that results should be regarded as preliminary due to the limited size of the current datasets and the binary nature of the labels. Class imbalance was not yet addressed via resampling or weighting.

Significance and Claims

The paper claims that the proposed methodology successfully captures meaningful acquisition variability and enables robust quality assessment across heterogeneous sensing modalities. By combining physical quality indicators with machine learning and XAI, the framework supports:

Automatic Discrimination: Distinguishing between acceptable and problematic acquisitions.
Interpretability: Understanding the underlying causes of degradation (e.g., distinguishing between detector instability and illumination issues).
Resource Awareness: Enabling adaptive strategies where marginal cases can be corrected or reacquired, while severe cases are flagged for human review.

The authors position this work as a foundational step toward a fully adaptive robotic acquisition workflow. They explicitly state that while the current implementation serves as a decision-support mechanism, future work will focus on validating the closed-loop feedback within a robotic platform and refining failure-mode classification beyond binary labels. The framework is designed to be inherently adaptive, allowing statistical descriptors and models to evolve as new data is collected under varying operational conditions.