Visual Semantics and Multimodal Perception of Digitally Reconstructed Garden Scenes of Jiucheng Palace
This study proposes a multimodal perception framework integrating deep learning, eye-tracking, EEG, and subjective ratings to quantitatively evaluate the spatial expression and experiential effectiveness of digitally reconstructed Jiucheng Palace garden scenes, moving beyond traditional morphological accuracy to optimize historical visualization.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine standing in a garden that no longer exists. For over a thousand years, the Jiucheng Palace in Shaanxi, China, served as a grand imperial retreat where emperors sought leisure and political refuge. Today, only the stone foundations of buildings and the faint outlines of ancient hills remain. To understand how this lost place once felt, researchers use digital tools to rebuild it in three dimensions, creating virtual scenes of what the gardens might have looked like. But a digital model is only as good as the experience it provides. If the reconstruction looks accurate on a computer screen but fails to capture the feeling of the original space, it misses the point. This is where the science of perception comes in. It asks a simple but difficult question: when we look at a digital recreation of a historic garden, what do our eyes, our brains, and our emotions actually tell us about the scene?
A team of researchers from Northwest University in Xi'an set out to answer this by studying the digitally reconstructed Jiucheng Palace. They wanted to move beyond simply checking if the buildings were the right shape. Instead, they built a new way to measure how people experience these virtual gardens. They focused on three distinct types of scenes found in the reconstruction: areas dominated by buildings and paved paths, areas where mountains and water meet, and areas filled with trees and vegetation. To understand the viewer's reaction, they did not just ask people what they thought. They used a combination of modern tools to measure the body's immediate, involuntary responses. They tracked where people looked and for how long, measured brain waves to see how relaxed or engaged the mind was, and asked detailed questions about the feelings the scenes evoked.
The researchers created fifteen different images from their digital model, ensuring that each scene was lit and viewed from the same angle so that the only difference was the landscape itself. They then invited forty-eight volunteers to view these images while wearing sensors. Some of the volunteers were experts in garden design or architecture, while others had no special training in these fields. As the participants looked at the pictures, an eye-tracker recorded every time their gaze stopped on a specific spot, and a brain-sensing device measured electrical activity in their brains. Afterward, the participants filled out a survey describing their feelings about the space, using pairs of opposite words like "open" versus "closed" or "calm" versus "tense."
The results revealed a clear pattern in how people experienced the different types of scenes. The images featuring mountains and water consistently received the highest praise. Participants described these scenes as more immersive and emotionally positive than the others. When the researchers looked at the brain data, they found a matching signal: the mountain-water scenes caused a specific type of brain wave associated with relaxation and calm attention to become stronger. In contrast, the scenes dominated by buildings and stone paths, while clear and orderly, did not generate the same sense of ease or emotional connection. The scenes filled mostly with trees and vegetation fell somewhere in the middle, offering a natural feel but lacking the specific spark of the mountain-water combination.
Interestingly, the way people looked at the images did not change as much as their feelings did. The eye-tracking data showed that people spent roughly the same amount of time looking at all three types of scenes, and their eyes moved in similar patterns. This suggests that while the eyes might scan a garden in a standard way, the brain and the heart react differently depending on what is actually in the view. The study found that the presence of water was a key factor. Scenes with a higher proportion of visible water were rated as more beautiful and more like a true garden. Conversely, scenes with too many buildings or paved walkways were rated lower in terms of their ability to evoke a garden atmosphere. The researchers also discovered that the openness of the sky in the image was linked to how the brain processed the space, suggesting that the feeling of being under an open sky is a crucial part of the experience.
To make these findings useful for future projects, the team developed a simple scoring system. They created two numbers that could be calculated for any digital garden scene. The first number measures the overall feeling a viewer gets from the scene, combining all the survey answers and brain data into a single score for "garden experience." The second number looks at the ingredients of the image itself—how much water, how many buildings, and how much open space are visible—to predict how well the scene will work as a garden. Using this system, they confirmed that the mountain-water scenes were the most successful, scoring highest on both the experience and the ingredient check. The building-heavy scenes scored the lowest.
This work suggests that when we try to bring lost historic gardens back to life on a screen, getting the buildings right is not enough. The magic of a Chinese garden lies in the relationship between the natural world and the built environment, specifically in how water, mountains, and vegetation are arranged. If a digital reconstruction focuses too much on the architecture and ignores the flow of water and the openness of the landscape, it will look accurate but feel empty. By using these new methods to measure human reaction, researchers can now fine-tune their digital models. They can adjust the viewpoint or the amount of water visible to ensure that the final image captures not just the history of the place, but the feeling of being there. This approach offers a way to validate digital heritage projects, ensuring that they convey the true spirit of the past to the people who view them today.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.