A Heritage-Aware Evaluation Framework for Synthetic Training Images of Kizil Grotto Murals
This paper presents a heritage-aware evaluation framework that utilizes ten weighted indicators across semantics, composition, color, and line to assess and screen synthetic Kizil Grotto mural images generated by various AI models, ultimately identifying Midjourney as the top performer for creating high-quality training datasets.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Deep within the arid landscape of western China lies the Kizil Grottoes, a vast network of caves that once served as a vital stop on the ancient Silk Road. Inside these caves, artists painted thousands of years ago, creating murals that tell stories of Buddhist life using a distinct visual language. These paintings are not just beautiful; they are historical records of religious belief, artistic exchange, and the specific techniques used by ancient craftsmen. However, time and the environment have taken a heavy toll. The pigments have faded, the plaster has cracked, and many sections of the walls have crumbled away, leaving gaps in the visual narrative. For researchers who want to study these artworks or use computers to help restore them, this damage creates a serious problem: there simply are not enough clear, complete images to teach a computer how to recognize the patterns, colors, and stories of the Kizil style.
To solve this shortage, scientists have turned to artificial intelligence, specifically a type of technology known as generative models. These are computer programs capable of creating new images from scratch based on text descriptions or examples. In theory, these tools could generate thousands of new images that look like Kizil murals, providing the massive amount of data needed to train restoration software. But there is a catch. Just because a computer can make an image that looks like a painting does not mean it is a good painting. An artificial image might get the colors right but distort the story, or it might capture the general shape of a figure but miss the specific cultural symbols that give the art its meaning. If researchers feed these flawed images into their training programs, the computer might learn the wrong lessons, potentially ruining future restoration efforts. The challenge, then, is not just making more images, but figuring out which ones are true to the heritage and which ones are just digital noise.
A team of researchers from Beijing University of Posts and Telecommunications and the Kizil Caves Research Institute set out to build a system to solve this exact problem. They developed a new way to evaluate synthetic images specifically for the Kizil Grotto murals, creating a framework that acts like a rigorous quality control check. Instead of asking a computer to simply guess if an image looks "good," they broke down the unique characteristics of Kizil art into four specific areas: the story being told, the arrangement of the picture, the colors used, and the lines that define the shapes. They then assigned a specific importance to each of these areas, deciding that getting the story and the cultural meaning right was the most critical factor, followed by how the picture was composed, the accuracy of the colors, and finally, the smoothness of the lines.
To test their system, the researchers started with a carefully prepared set of real images. They took 1,479 photographs of Cave 38, a specific chamber in the grottoes known for its rich and relatively complete murals. From these photos, they manually restored 94 images to remove the visual noise of cracks and fading, and then selected 24 of the best ones to serve as the "gold standard" for the experiment. These 24 images featured non-human subjects like animals, plants, and decorative objects, chosen to avoid the complex difficulties of generating human faces. Using these 24 reference images, they asked four different popular artificial intelligence systems—Stable Diffusion, Midjourney, DALL·E 3, and Kandinsky—to generate new images. They gave each system the same instructions, asking them to create images in the style of Kizil murals with specific motifs and colors. In total, the systems produced over 10,000 candidate images for the team to evaluate.
The results of this massive screening process revealed that not all artificial intelligence is created equal when it comes to cultural heritage. The researchers found that one system, Midjourney, produced the highest overall quality, scoring the best on the combined scale of story, composition, color, and line. However, the study showed that no single system was perfect at everything. While Midjourney excelled at capturing the right stories and keeping the main subjects centered, another system, DALL·E 3, was better at maintaining the texture and surface details of the original paintings. A third system, Stable Diffusion, was particularly good at matching the specific shades of red and blue found in the ancient murals, while the fourth, Kandinsky, struggled the most with the overall cultural accuracy. The team discovered that a high score in one area could sometimes hide a failure in another; for instance, an image might look visually striking but fail to tell the correct story, making it useless for training a restoration model.
The researchers concluded that their new framework successfully separates the wheat from the chaff, identifying which synthetic images are safe to use for training computers and which should be discarded. They found that the most important factor for these ancient murals was semantic consistency—ensuring that the generated image actually depicted the correct cultural symbols and story elements. Without this, even a beautiful image is misleading. The study does not claim to have solved the problem of digital heritage forever, nor does it declare one artificial intelligence tool as the ultimate winner for all future tasks. Instead, it provides a practical, structured method for curators and scientists to screen thousands of generated images, ensuring that the digital tools used to protect these ancient treasures are trained on data that respects the integrity of the original art. By filtering out the distortions and drifts that artificial intelligence often introduces, this framework helps ensure that the digital future of the Kizil Grottoes remains faithful to their past.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.