Sampling variability ranks reconstruction error in lacquerware image inpainting across models and collections
This study demonstrates that sampling variability across multiple generative inpainting reconstructions serves as a strong predictor of reconstruction error in lacquerware images, enabling the prioritization of outputs for human review despite its inability to guarantee historical authenticity or eliminate errors entirely.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Museum photographs serve as the primary record for delicate objects like lacquerware, capturing the sheen of a surface or the intricacy of a painted motif. Yet, a photograph is only a flat image; it cannot reveal the layers of material beneath the paint or the true condition of the object if parts of it are missing. When conservators or historians need to imagine what a damaged piece might have looked like, they sometimes turn to artificial intelligence. These computer programs can fill in missing areas of a photo, creating a complete picture that looks convincing to the human eye. However, a convincing image is not necessarily a correct one. The computer might invent a pattern that never existed or smooth over a crack that should remain visible. The central challenge for digital heritage is not just making a pretty picture, but knowing when that picture is likely to be wrong. If a computer generates a restoration, how can a human expert tell if they should trust it or look closer?
A team of researchers set out to answer this question by testing a specific idea: if a computer generates several different versions of the same missing piece, the amount of disagreement between those versions might signal how accurate the result is. They focused on lacquerware, a type of decorated wood or metal coated in a hard, shiny finish, using a collection of photographs from major museums. The researchers took intact photos of these objects and digitally covered small, medium, and large sections of them, simulating damage. They then asked two different artificial intelligence models to "heal" the damage by generating four different possible completions for each missing spot. The key to their method was a simple comparison. They looked at the first three generated images to see how much they differed from one another in the missing area. If the three images looked very similar, the computer was confident. If they looked very different, the computer was uncertain. They then checked this "uncertainty" score against the fourth image, which they had kept hidden, to see if the score actually predicted how far off the computer was from the original, intact photograph.
The results showed a clear pattern. When the three generated images disagreed significantly, the fourth image was usually quite different from the original photograph, meaning the reconstruction contained a large error. Conversely, when the three images agreed closely with each other, the fourth image was much closer to the truth. In the group of 64 original objects, the researchers found that this disagreement was a strong predictor of error. By using this method, they could sort the objects and choose to inspect only the ones where the computer seemed most confident. If they looked at only the 40 percent of objects with the lowest disagreement, the average error in those selected images dropped dramatically compared to looking at all the images at random. This did not mean the computer made better pictures on its own; the images were exactly the same. Instead, the method acted as a filter, helping humans decide which images were worth their time to examine and which were likely to be misleading.
This approach held up even when the researchers tested it on a completely new set of 40 objects from different museums, using a different version of the artificial intelligence model. The link between disagreement and error remained strong, suggesting the method is not just a fluke of one specific dataset. However, the study also revealed important limitations. The computer was much better at reconstructing simple, plain surfaces than it was at handling complex, decorated areas. Because the method naturally favored the simpler objects, a strict application of the filter would have excluded most of the decorated pieces, which are often the most culturally significant. Furthermore, while the method could predict pixel-level errors, it could not determine if a reconstruction was historically accurate or suitable for actual conservation work. The computer might agree on a wrong pattern, or it might generate a plausible but historically impossible design.
The researchers also compared their method to simpler ways of judging image difficulty, such as measuring the number of edges or the amount of texture in the missing area. In the initial group of objects, their disagreement method performed slightly better, but the advantage was small and not always consistent. In the external group of museum objects, the disagreement method showed a clearer benefit over these simpler checks. The study concluded that while this technique does not solve the problem of historical authenticity, it offers a practical tool for museums. It provides a reproducible signal that can help prioritize which digital restorations need human review. By flagging the images where the computer is unsure, it allows experts to focus their attention where it is needed most, ensuring that the effort of verifying digital heritage is spent on the most uncertain and potentially problematic cases.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.