Material-Segmented Per-Pixel Emissivity Correction for Thermographic Anomaly Detection in Cultural Heritage Digital Twins
This paper presents a training-free pipeline that corrects thermographic temperature artifacts in cultural heritage digital twins by deriving per-pixel emissivity from open-vocabulary RGB segmentation, revealing that while the method significantly improves accuracy for low-emissivity anomalies, its impact on typical weathered heritage surfaces is limited due to their naturally high and uniform emissivity.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a detective trying to solve a mystery using a special pair of "heat glasses." These glasses don't show you colors like a normal camera; instead, they show you how hot or cold everything is. This is called thermal imaging, and it's used to find hidden cracks in old statues, spot moisture inside walls, or even see what's happening under the surface of a painting without touching it. But here's the tricky part: these glasses don't actually measure temperature directly. They measure radiance—how much infrared light bounces off or comes out of an object.
To turn that light into a temperature number, the detective has to make a guess about something called "emissivity." Think of emissivity as a "glow factor." A shiny, polished metal spoon has a low glow factor; it reflects the heat of the room around it like a mirror. A piece of rough, old brick has a high glow factor; it glows with its own heat and doesn't reflect much. If your heat glasses assume everything is a "high-glow" brick (which is the standard setting), but you are actually looking at a shiny metal statue, the glasses will get confused. They will think the statue is freezing cold because it's reflecting the cold sky, when in reality, it might be warm. This confusion creates fake "cold spots" that look like damage but aren't, or hides real damage that is there. The big question for scientists is: how do we fix these glasses so they know exactly what material they are looking at, so they can tell the real temperature from the fake reflection?
This paper introduces a clever new way to fix those heat glasses, specifically for looking at old, precious cultural heritage sites like museums and ancient monuments. The researchers, led by Jonathan Klingspon and his team, realized that while the heat camera is confused, a regular color camera (the one on your phone) is not. A regular camera can easily tell the difference between a marble statue, a glass window, and a bronze door. The team built a "magic pipeline" that uses a smart computer program (called SAM 3.1) to look at the regular color photo, identify every single material in the picture, and then assign a specific "glow factor" to each one based on a giant reference book of material properties.
Instead of guessing that the whole picture is one type of material, their method creates a custom map where every single pixel knows exactly what it is made of. They then feed this map into the heat camera's math to calculate the real temperature. To test if this works, they couldn't just use real historical sites because no one has a "perfect truth" temperature for those old stones. So, they built a super-accurate computer simulation—a digital twin of a scene where they knew the exact temperature and materials. In these simulations, when the scene had a big mix of materials (like shiny metal next to rough stone) and a big temperature difference (20 K), their new method slashed the error from nearly 2 degrees down to less than 1 degree (specifically from 1.97 K to 0.91 K). It was much better than just using a single "best guess" number for the whole image.
However, the paper also delivers a very important reality check. The team tested this on real-world heritage sites, like the famous Loggia dei Lanzi in Florence with its bronze statues. They found that while the method works great in theory and in simulations with tricky materials, the real world is often less dramatic. Most old, weathered outdoor statues and buildings are actually covered in dirt, patina, and age, which makes them act like "high-glow" bricks anyway. On these typical surfaces, the correction is tiny—often less than 0.1 K—because the standard guess was already pretty close. The method only makes a huge difference when there are genuine "low-glow" exceptions, like a truly polished metal surface or a glass window, and only when there is a strong temperature difference to begin with.
The researchers also discovered some "danger zones" where this method shouldn't be used. If the scene is all the same temperature (like a warm room with no drafts), the math breaks down, and the correction becomes useless or even harmful. Also, if the computer misidentifies a material—thinking a piece of wood is metal because they look similar in the photo—the temperature calculation can go wildly wrong, especially for shiny materials. They showed that for this to work, the color camera and the heat camera must be perfectly aligned, and the list of materials must be very accurate.
In the end, this paper doesn't claim to have solved all temperature mysteries. Instead, it provides a detailed "operating manual" for when this new trick works and when it doesn't. It proves that using a color camera to guide the heat camera is a powerful tool, but it's not a magic wand that fixes everything. It's a specialized scalpel that works wonders on specific, tricky problems (like finding hidden cracks in a mix of bronze and marble) but offers little help on a uniform, weathered stone wall. The authors suggest that for conservators, the best approach is to use this method to create two separate maps: one showing the corrected temperature and another showing the material map, so they can see both the heat and the "glow factor" side-by-side, ensuring they don't miss a real problem or get fooled by a fake one.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.