Spectrally Enriched 3D Models for Representation and Interpretation of Elemental Data in Augmented Reality

Imagine holding a fragile, centuries-old artifact in your hands. It might be a painted ceramic shard, a delicate piece of jewelry, or a botanical model made of paper and glue. To understand what it is made of, scientists often use a technique called Laser-Induced Breakdown Spectroscopy. This method works by firing tiny, high-energy pulses of light at the surface of an object. Each pulse vaporizes a microscopic amount of material, creating a brief flash of plasma that reveals the chemical elements present at that exact spot. It is a powerful tool for uncovering the history of materials, from identifying ancient pigments to detecting trace minerals. However, there is a catch: the process leaves a tiny crater where the laser hit. If a scientist wants to map the entire surface of an object to see how its composition changes from one area to another, they must fire thousands of these pulses. On a flat, sturdy surface, this might be acceptable. But on a fragile, curved, or three-dimensional object, the cumulative effect of thousands of tiny craters can cause visible damage, altering the very thing the researchers are trying to study.

The challenge, then, is to learn everything possible about an object's chemical makeup without destroying it. For flat objects, researchers have developed a clever workaround: they take a photograph, use a computer to group similar-looking colors together, and then fire the laser only at a few representative spots within each group. This reduces the number of shots needed significantly. But this approach breaks down when the object is not flat. A photograph flattens the world, hiding the curves, bumps, and hidden corners of a real object. If you try to map a curved surface using only a flat image, you might group together two areas that look similar in the photo but are actually far apart on the real object, or you might miss a crucial detail hidden in a shadow or a concave curve. To solve this, a team of researchers at INESC TEC and the University of Porto in Portugal has developed a new way to analyze these complex shapes. They have moved beyond flat images and created a system that understands the true three-dimensional shape of an object, allowing them to map its chemistry with far fewer laser shots and significantly less risk of damage.

The researchers tested their new method on a specific, delicate object: a nineteenth-century papier-mâché model of a mushroom, specifically the Amanita muscaria, which is housed in the Natural History and Science Museum of the University of Porto. The model is made of two separate pieces, a stem and a cap, and was originally created for educational display. Because the material is fragile and the object has a complex, rounded shape, it was an ideal candidate for this experiment. The team began by creating a precise digital twin of the mushroom. Using a 3D scanner that projects blue laser light, they captured the object's exact geometry and surface texture. This process produced a detailed digital model that included not just the shape, but also the color and fine surface details, stored as a high-resolution texture map wrapped around the 3D form.

With the digital model in hand, the team faced the task of deciding where to fire the laser. Instead of looking at a flat photograph, they looked directly at the 3D model. They used a computer algorithm to break the surface of the mushroom into thousands of tiny, irregular patches called superpixels. Crucially, this algorithm did not just look at color; it also considered the physical location of each patch in three-dimensional space. It ensured that a patch of red on the cap of the mushroom was not grouped with a patch of red on the stem if they were far apart in reality, even if they looked similar in a flat image. This step created a map of the object that respected its true geometry, preserving the boundaries between different parts of the mushroom while keeping the surface divided into manageable, coherent regions.

Once the surface was divided into these geometrically aware patches, the team simplified the data. They calculated the average color for each patch and then grouped the patches into larger clusters based on how similar their colors were. This process reduced the complexity of the mushroom's surface from millions of individual pixels to a much smaller number of distinct regions. The result was a segmentation that identified the major visual areas of the object, such as the red cap with white spots and the white stem, while respecting the curves and contours of the physical model. From each of these final clusters, the researchers selected just one or a few representative spots to analyze with the laser.

This strategy led to a dramatic reduction in the number of laser shots required. Instead of scanning the entire surface with over 221,000 potential points, which would have left the fragile papier-mâché riddled with craters, the team selected only 40 specific shots. These 40 shots were enough to capture the essential chemical information for the entire object. By analyzing the light emitted from these few points, they could infer the elemental composition of the surrounding areas within each cluster. The laser pulses were set to a low energy level, creating ablation spots only about 300 micrometers in diameter, ensuring that the damage was microscopic and localized.

The final step was to bring this data back to life in a way that is easy to understand. The team mapped the results of their 40 laser shots back onto the 3D digital model. They then used augmented reality technology to visualize the findings. By wearing a specialized headset, a user could look at the digital mushroom floating in space and see the different chemical clusters highlighted directly on its surface. The interface allowed them to rotate the model, zoom in, and select specific elements to see where they were found. This provided a spatially contextualized view, letting experts and non-experts alike see exactly how the chemical composition related to the physical shape and texture of the artifact.

The study demonstrates that it is possible to extend the benefits of smart sampling from flat, two-dimensional images to complex, three-dimensional objects. By incorporating the actual geometry of an object into the analysis, researchers can avoid the distortions and errors that come from flattening a curved surface. This approach does not just save time; it fundamentally changes the relationship between analysis and preservation. It allows scientists to gather rich, spatially detailed chemical data from fragile historical objects without subjecting them to the cumulative damage of a full scan. The work suggests a new path forward for cultural heritage science, where the integrity of the object is prioritized, and where the tools of analysis are adapted to the true, three-dimensional nature of the world we are trying to understand.