Building a cross-platform WebXR heritage application from a drone survey using a language model

Preserving the past has always been a race against time and decay. For centuries, historians and architects have relied on drawings, photographs, and written records to capture what buildings looked like before they crumbled or changed. Today, technology has added a powerful new tool to this effort: high-resolution scanning. By taking thousands of overlapping photographs from drones or lasers from the ground, experts can now build incredibly detailed three-dimensional models of ancient sites. These digital twins are accurate enough to show the texture of a mud brick wall or the curve of a stone arch. However, a significant gap remains between creating these static models and turning them into living, interactive experiences that the public can explore. Traditionally, bridging that gap required a team of software engineers to write complex code, a skill set that most heritage organizations do not possess. This leaves many captured sites sitting as data files, unseen by the people they were meant to protect.

A new approach suggests that the barrier to entry might be lower than previously thought. Instead of writing code line by line, a specialist can now describe what they want an application to do in plain language, asking an artificial intelligence to build the software for them. This method, often called "vibe coding," shifts the focus from technical syntax to the desired outcome. The question is whether a person with no programming background can use this tool to create a reliable, cross-platform application for a real-world heritage site, and what happens when the software is tested on actual devices.

In a recent study, researchers documented how a heritage specialist with no game-programming experience created a fully interactive application for the Sadous heritage village in Saudi Arabia. The village, located about 70 kilometers northwest of Riyadh, is a walled settlement built from sun-dried mud brick, a traditional style known as Najdi architecture. The site exists in two states: a western section where roofs and walls have been restored, and an eastern section of roofless ruins. The researcher began by flying a drone over the village, capturing 10,213 images to create a precise 3D map of the area. This digital model covered 0.160 square kilometers with a level of detail where each pixel represented less than one centimeter on the ground.

With the 3D model ready, the researcher turned to a language model, a type of artificial intelligence capable of generating computer code. The researcher did not write a single line of code. Instead, they acted as a director, describing the desired behavior of the application in natural language. They told the AI what the user should see, how they should move through the village, and what interactions should occur. For example, the researcher described a gameplay mechanic where a player scans a ruined building to trigger a holographic reconstruction, showing the building as it looked when restored. This design choice was not just a visual trick; it allowed players to toggle between the current ruined state and the hypothesized restored state, making the difference between fact and interpretation visible. The AI generated the entire application, including the logic for moving characters, the physics for climbing stairs, and the dialogue for a traditional coffee ceremony, all based on these spoken instructions.

The final result was a single file, roughly 34 megabytes in size, that could run on a desktop computer, an Android phone, an iPhone, and a virtual reality headset without needing any installation or server setup. The application guided players through a series of tasks: arriving at the village gate, walking across rooftops, documenting structures, and participating in a cultural ritual involving the roasting and serving of coffee. The coffee sequence was particularly notable because it required the player to follow specific social rules, such as pouring with the right hand and filling the cup only a quarter full, teaching the custom through action rather than text.

However, the process was not without significant challenges. While the AI could generate the code, it could not always predict how that code would behave on different physical devices. The researchers recorded 27 distinct problems that appeared during the development and testing phases. Ten of these issues only became visible when the application was run on a physical device, such as a mobile phone or a virtual reality headset, and could not be detected by the computer environment where the AI was working. For instance, on a virtual reality headset, the application would sometimes fail to respond to controller inputs because the AI had assumed the device worked like a standard computer browser. Similarly, five other errors were purely cultural; the AI had programmed behaviors that did not match the actual traditions of the region, such as incorrect ways to serve coffee or inappropriate character movements. These cultural mistakes were only caught when a human expert with deep knowledge of the local heritage reviewed the application.

The study highlights a crucial distinction between creating software and verifying it. The AI successfully wrote the code, but it could not fully test its own work. The automated tests the AI wrote passed all checks, yet the application still contained bugs that broke the experience on specific devices. This suggests that while artificial intelligence can handle the heavy lifting of writing code, the responsibility for validation remains firmly with the human. A heritage specialist must still possess the skill to observe the application closely, run it on every type of device it is meant for, and judge whether the cultural and historical details are accurate.

The researchers found that this new method of development is possible, but it changes the nature of the work rather than removing the need for expertise. The specialist did not need to learn to program, but they did need to become a rigorous observer, capable of describing exactly what went wrong so the AI could fix it. The study also raised concerns about the long-term preservation of such applications. Because the code was generated by an AI and is not understood by the human creators, future updates or repairs would depend on having access to a similar artificial intelligence system. Furthermore, the application makes interpretive claims about the past, such as how a building looked when restored, and the study emphasizes that these claims must be clearly distinguished from the physical evidence to maintain historical integrity.

Ultimately, this project demonstrates that a heritage professional can now build a sophisticated, interactive experience for the public without a team of software engineers. The application successfully ran on multiple platforms, allowing users to explore the Sadous village and engage with its history in a new way. Yet, the path to this result was paved with careful human oversight. The AI provided the tools, but the human provided the judgment, the cultural context, and the critical eye needed to ensure the final product was both functional and true to the history it represented. This approach offers a promising path for preserving and sharing heritage, provided that the human role shifts from writing code to rigorously testing and validating the stories the software tells.