💻 computer science

Dual Band Thermal Videography: Separating Time-Varying Reflection and Emission Near Ambient Conditions

This paper presents a dual-band thermal videography framework that resolves the under-determined challenge of separating time-varying emission and reflection in near-ambient conditions by jointly leveraging spectral emissivity constraints and temporal signal dynamics to recover accurate temperature fields.

Original authors: Sriram Narayanan, Mani Ramanagopal, Srinivasa G. Narasimhan

Published 2026-03-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Sriram Narayanan, Mani Ramanagopal, Srinivasa G. Narasimhan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are looking at a shiny, silver coffee pot sitting on a kitchen counter. If you look at it with your normal eyes, you see the pot itself and the reflection of the room in its shiny surface. It's easy to tell the difference.

But if you look at it with a thermal camera (which sees heat instead of light), things get messy. The camera sees two things mixed together in a single blurry image:

The Heat: The actual heat coming out of the coffee pot (because the coffee is hot).
The Reflection: The "ghost" heat bouncing off the pot from the people walking by or the warm walls behind it.

Usually, thermal cameras can't tell the difference between the pot's own heat and the heat it's reflecting. It's like trying to hear a single singer in a choir where everyone is singing the exact same note at the exact same volume. You just hear a wall of noise.

This paper introduces a clever trick to separate the singer from the choir. Here is how they did it, using simple analogies:

The Problem: The "Gray Body" Confusion

For a long time, scientists assumed that objects act like "gray bodies"—meaning they reflect and emit heat in a predictable, boring way. They thought if you knew how much heat an object should emit, you could figure out the rest. But in the real world (near room temperature), this doesn't work. The heat coming from the object and the heat bouncing off it are almost equal in strength, making the math impossible to solve with just one camera.

The Solution: Two Eyes, Two Times

The researchers built a system using two thermal cameras looking at the scene at the same time, but each camera is wearing a different pair of "spectral glasses" (filters).

Camera A sees heat at one specific color (wavelength).
Camera B sees heat at a slightly different color.

They use two main clues to untangle the mess:

1. The "Color" Clue (Spectral Cues)

Think of emissivity (how well something emits heat) like a person's favorite color.

A red shirt might look very red to the eye but invisible to a night-vision camera.
Similarly, a coffee pot might emit heat very strongly at "Color A" but weakly at "Color B."
The Trick: The researchers realized that while we don't know the exact "favorite color" of the object beforehand, the ratio between how it looks in Color A vs. Color B stays constant. It's like knowing that if a person is wearing a red shirt, it will always look 2x redder than blue, no matter how the lighting changes. This constant ratio helps them figure out the object's true identity.

2. The "Speed" Clue (Temporal Cues)

This is the most creative part. They looked at how things change over time.

The Object (The Singer): Heat moves slowly. If you pour hot coffee into a pot, the pot warms up gradually. It's like a slow, smooth melody.
The Background (The Choir): Reflections change instantly. If a person walks past the pot, their reflection appears and disappears in a split second. It's like a sudden, sharp drumbeat.

By watching the video, the computer can say: "That slow, smooth change is the pot heating up. That sudden, jerky change is a person walking by."

How It Works in Real Life

The team tested this on everyday scenes:

The Coffee Pot: They separated the slow warming of the pot from the fast-moving reflections of people walking by.
The Glass Plate: They could see a fingerprint left on a cold glass (which slowly fades as the heat dissipates) while ignoring the reflection of a person's hand moving in front of it.
The Mirror: They could see a palm print on a mirror that was slowly disappearing, distinct from the reflection of a person standing nearby.

Why This Matters

Before this, thermal cameras were like a radio with all the stations playing at once. You could hear something, but you couldn't make out the words.
This new method is like a DJ who can isolate the vocals from the music. It allows us to see the true temperature of an object even if it's shiny, even if people are walking around it, and even if we don't know exactly what the object is made of.

In a nutshell: They used two different "heat colors" and the difference in "speed" between slow heat and fast reflections to mathematically peel apart the real temperature of an object from the confusing shadows of its surroundings. This opens the door for thermal cameras to be used in much more complex, real-world situations, from inspecting buildings to helping robots see better in the dark.

1. Problem Statement

Thermal cameras in the Long-Wave Infrared (LWIR, 8–14 µm) spectrum capture radiation that is a mixture of two components:

Emission: Radiation emitted by the object itself, governed by its temperature and spectral emissivity.
Reflection: Radiation from the surrounding environment reflected off the object's surface.

The Core Challenge: Separating these two components is an ill-posed (under-determined) problem. Without prior knowledge of the object's emissivity, it is impossible to uniquely solve for both temperature and emissivity from a single thermal measurement.

Existing Limitations: Most prior methods assume the background is static, the object is significantly hotter than the background (allowing reflection to be ignored), or the emissivity is known.
The Gap: In near-ambient conditions (typical computer vision scenarios), emitted and reflected signals are of comparable magnitude, and both the object and background can be time-varying. This makes standard assumptions unreliable.

2. Methodology

The authors propose a Dual-Band Thermal Videography framework that resolves this ambiguity by combining spectral cues and temporal cues within a unified optimization framework.

A. Image Formation Model

The paper derives a linear model for the pixel intensity $I_m(t)$ in a specific spectral band $m$ at time $t$ :
$I_m(t) = \epsilon_m U(T_o(t)) + (1 - \epsilon_m) U(T_b(t))$
Where:

$\epsilon_m$ : Spectral emissivity of the object in band $m$ .
$T_o(t)$ : Time-varying object temperature.
$T_b(t)$ : Effective background temperature (reflected radiation).
$U(T)$ : A function converting blackbody temperature to pixel intensity (approximated as linear near ambient conditions).

The system captures video in two spectral bands ( $m=1, 2$ ). The key insight is that while emissivity $\epsilon$ is unknown, it is fixed for a material across time, but different between the two bands (violating the "gray-body" assumption).

B. Two Approaches

The paper presents solutions for both calibrated and uncalibrated scenarios:

1. Calibrated Case (Known Emissivity):

Procedure: Emissivity is measured a priori using a thermocouple and a known blackbody background.
Solution: With $\epsilon_1$ and $\epsilon_2$ known, the system of equations becomes solvable. The authors derive a closed-form analytical solution to estimate the time-varying object temperature $T_o(t)$ and background temperature $T_b(t)$ directly from the two band measurements.

2. Uncalibrated Case (Unknown Emissivity):

Challenge: Without known emissivity, the system is rank-deficient (2 equations, 3 unknowns per pixel per time step).
Key Insights for Constraints:
- Spectral Cues: The ratio of emissivities ( $\epsilon_2/\epsilon_1$ ) is constant for a material.
- Temporal Cues (Object): Heat transport within an object is a smooth, continuous process (governed by heat diffusion equations).
- Temporal Cues (Background): Background reflections often change abruptly (e.g., a person walking by) and are uncorrelated with the object's thermal evolution.
Optimization Framework:
The authors formulate a joint optimization problem to estimate $\epsilon_m$ $ϵ_{m}$ , the smooth emission signal $\tilde{I}(t)$ $\tilde{I} (t)$ , and the noise. The objective function minimizes:
- Smoothness Loss ( $L_{smooth}$ ): Enforces that the estimated emission signal varies smoothly over time (second-order prior).
- Huber Loss ( $L_{Huber}$ ): Ensures the estimated signal stays close to the measured data while being robust to abrupt background changes (outliers).
- Reconstruction Loss ( $L_{MSE}$ ): Uses the ratio constraint $k_2$ (derived from the difference between measured and smooth signals) to reconstruct the original signal.
- Noise Regularization: Models and minimizes per-pixel noise.

3. Key Contributions

Dual-Band Framework: A novel method to separate reflection and emission in near-ambient, dynamic scenes using two LWIR spectral bands, overcoming the limitations of single-band or static-background assumptions.
Physical Modeling: Derivation of a comprehensive image formation model that accounts for gain/offset corrections (narcissus effect) and explicitly models the interplay between spectral emissivity and temporal dynamics.
Uncalibrated Optimization: A robust optimization strategy that recovers emissivity and temperature without prior calibration by leveraging the distinct temporal behaviors of heat transport (smooth) vs. light transport/reflection (abrupt).
Calibration Protocol: A practical procedure to calibrate spectral emissivity using thermocouples and blackbody references, validated against standard spectral libraries.

4. Experimental Results

The authors validated their method through simulations and real-world experiments:

Simulations:
- Compared against state-of-the-art baselines: Blackbody Channel Prior (BCP), naive multi-wavelength pyrometry, and dual-wavelength pyrometry.
- Result: The proposed method significantly outperformed baselines in recovering object temperature, especially under moderate-to-high noise conditions. It successfully recovered emissivity values across a wide range of materials.
Real-World Experiments:
- Calibration: Validated calibrated emissivity values against the ECOSTRESS spectral library and FLIR's reflector method. Their method showed higher accuracy for low-emissivity materials (e.g., chrome, aluminum) where the reflector method failed.
- Separation Scenarios:
  - Coffee Pot: Separated the slow thermal emission of hot liquid from the reflection of moving people.
  - Glass Plate: Separated the emission of a heated glass plate from the reflection of text written with cold/wet fingers.
  - Palm Print: Separated the dissipating heat of a palm print on a cold bulb from the reflection of a person.
- Accuracy: The uncalibrated approach achieved temperature estimation errors of 1.72% (wineglass) and 5.34% (coffee pot) compared to ground-truth thermocouple measurements.

5. Significance

Advancing Thermal Vision: This work moves thermal imaging from simple temperature mapping to physically-based scene understanding. It allows cameras to "see" through reflections, a capability previously limited to visible-light polarization techniques.
Dynamic Environments: Unlike previous methods that required static backgrounds or high-temperature targets, this approach works in dynamic, near-ambient environments, making it applicable to robotics, surveillance, and human-computer interaction.
Material Property Recovery: The ability to recover spectral emissivity and temperature simultaneously without prior calibration opens new avenues for material classification and defect detection in real-time video.
Accessibility: By utilizing standard LWIR cameras with spectral filters, the method offers a pathway to high-fidelity thermal analysis without requiring expensive, specialized multi-spectral hardware.

In conclusion, this paper establishes a foundational framework for disentangling light and heat transport in thermal videos, enabling robust temperature estimation and reflection removal in complex, real-world scenarios.