💻 computer science

Gaussians on Fire: High-Frequency Reconstruction of Flames

This paper presents a novel method for reconstructing dynamic 3D fire from limited camera views by combining static background separation with monocular depth priors, initializing fire as a 3D flow field, and encoding high-frequency features via 3D Gaussians with lifetime and velocity attributes to achieve robust results using affordable, synchronized hardware.

Original authors: Jakob Nazarenus, Dominik Michels, Wojtek Palubicki, Simin Kou, Fang-Lue Zhang, Sören Pirk, Reinhard Koch

Published 2026-07-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jakob Nazarenus, Dominik Michels, Wojtek Palubicki, Simin Kou, Fang-Lue Zhang, Sören Pirk, Reinhard Koch

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Catching a Ghost in a Camera

Imagine trying to take a 3D photo of a flame. This is incredibly hard because fire is tricky:

It's transparent: You can see through it, so cameras struggle to figure out where it actually "is."
It moves fast: It flickers and changes shape in the blink of an eye.
It has no solid shape: It's not like a ball or a chair; it's a swirling cloud of hot gas.

Most modern 3D reconstruction tools (which usually work great for static objects like buildings or people) fail miserably with fire. They try to treat the flame like a solid object that just moves around. But because fire isn't solid, these tools get confused. They either blur the fire into a mess, make it look like it's part of the background wall, or make it disappear entirely.

The Solution: "Transient" Fire Particles

The authors propose a new way to build 3D fire using 3D Gaussian Splatting.

The Analogy:
Think of standard 3D reconstruction like building a statue out of clay. You mold the clay, and it stays there.
The authors' method is more like releasing a swarm of fireflies into a room.

Standard methods try to glue the fireflies to the walls or make them move in a smooth, predictable line.
This new method treats every tiny speck of light in the flame as a "firefly" (a Gaussian) that is born, flies for a split second, and then dies.

These "fireflies" are special because they carry two extra pieces of information:

A Lifespan: They know exactly how long they should exist before vanishing.
A Velocity: They know exactly how fast and in what direction they are flying.

By using millions of these short-lived, flying fireflies, the computer can build a 3D model of the flame that looks real and moves naturally, even if you only have a few cameras watching it.

How They Did It (The Three-Step Recipe)

1. Separating the "Stage" from the "Actors"
Since fire is hard to see, the team first separates the background (the table, the wall, the room) from the fire itself.

They take a video of the fire.
They use a trick to find the "darkest" parts of the video over time to figure out what the room looks like without the fire.
They build a perfect 3D model of the empty room first. This gives them a solid foundation so the fire doesn't get confused with the furniture.

2. Guessing the Wind (The Flow Field)
To make the fire move correctly, they need to know which way the air is pushing it.

They use a special algorithm to track how pixels move between video frames (like tracking a leaf blowing in the wind).
They combine the views from three different cameras to create a 3D "wind map."
They use this map to "seed" the fire. They place their "firefly" Gaussians into the 3D space and tell them, "Start here, and fly in this direction at this speed."

3. The "Sub-Frame" Sync Trick
Fire moves so fast that standard cameras miss details. If you take three photos with three different cameras, they might be off by a tiny fraction of a second, causing the 3D model to look jittery.

The Problem: Cheap cameras (like GoPros) don't have a "sync button" to take photos at the exact same microsecond.
The Fix: The authors built a simple, low-cost device using a microcontroller and a strip of LEDs.
The Analogy: Imagine three photographers trying to snap a photo of a hummingbird. If they aren't perfectly synced, they get different parts of the wing. The authors flashed a specific pattern of lights (a "code") that the cameras could see. By looking at how the light streaked across the camera sensor (a phenomenon called "rolling shutter"), they could mathematically calculate exactly when each camera took the photo, down to the microsecond. This allowed them to stitch the images together perfectly.

The Results

The team tested this on real fires (burning wood, propane, paper) and simulated fires.

Visuals: Their method created sharp, realistic 3D fire that didn't blur or collapse.
Depth: It figured out how far away the fire was much better than other methods.
Motion: The fire moved in a way that felt physically correct, rather than jittery or stuck.

Why This Matters (According to the Paper)

The paper claims this is the first time fire has been successfully reconstructed from just three cameras (a sparse setup) using this type of technology. Previous methods needed dozens of cameras or complex physics simulations to get a decent result.

They also created a new way to measure success. Instead of just asking "Does the picture look pretty?", they asked "Does the fire move in the right direction?" and "Is the 3D depth accurate?" Their method passed these tests where others failed.

In short: They figured out how to catch a ghost (fire) by treating it not as a solid object, but as a cloud of short-lived, fast-moving particles, and they built a cheap, clever trick to make sure all their cameras were watching the ghost at the exact same moment.

Technical Summary: Gaussians on Fire: High-Frequency Reconstruction of Flames

Problem Statement

Reconstructing dynamic fire in 3D from a limited set of camera views presents a unique challenge that current state-of-the-art methods fail to address. Fire is characterized by its volatile nature, transparency, high-frequency appearance, and rapid temporal variation. Existing Gaussian Splatting (GS) approaches, which rely on the assumption that scene content consists of persistent primitives undergoing smooth, trackable motion, fundamentally fail when applied to fire.

The authors identify three recurring failure modes in baseline dynamic GS methods (e.g., 4D Gaussian Splatting, FreeTimeGS) when applied to flames:

Geometric Absorption: The algorithm attempts to explain flame motion by deforming the static background geometry.
Depth Collapse: The system generates plausible 2D image appearances without establishing a meaningful 3D structure.
Temporal Drift: The rendered flame fails to match the observed dynamic evolution over time.

Furthermore, capturing fire in real-world environments is difficult due to the lack of dense multi-view coverage and the need for high temporal resolution. Most existing volumetric reconstruction methods require either dense camera arrays or strong physics-based simulation priors, which are often unavailable or computationally prohibitive.

Methodology

The proposed pipeline reconstructs high-frequency spatiotemporal flame dynamics using 3D Gaussian Splats from as few as three calibrated camera views. The core innovation is a transient Gaussian reconstruction formulation that decouples the static background from the dynamic, short-lived emissive flame structures.

1. Static Scene Reconstruction

To handle the static background under sparse-view constraints, the authors employ a two-stage strategy:

Initialization: They generate a dense point cloud by fusing stereo depth maps (computed via PatchMatchStereo on static frames) with monocular depth predictions (DepthAnythingV2).
Optimization: The static scene is optimized using vanilla 3DGS. To prevent overfitting and degenerate depth solutions caused by limited views, they incorporate a continuous monocular depth regularization term ( $L_{depth}$ ) into the loss function, weighted heavily during training.

2. Dynamic Scene (Fire) Reconstruction

The fire is modeled not as deforming persistent geometry, but as a population of transient flame Gaussians. These are short-lived volumetric carriers of emission with finite temporal support.

Flow-Based Initialization: Instead of relying on sparse feature tracking, the method estimates a 3D volumetric flow field. This is achieved by computing dense 2D optical flow (using MEMFOF) for each view, projecting these flows onto a 3D voxel grid, and solving an optimization problem with Tikhonov regularization to satisfy multi-view projection constraints.
Transient Gaussian Representation: Each Gaussian is initialized from this 3D flow field and assigned specific temporal parameters:
- Lifetime ( $t_\sigma$ ): A finite temporal support (initialized to four times the frame duration) to model the transient nature of fire.
- Linear Velocity ( $v$ ): A constant velocity vector derived from the flow field, determining the position $x(t) = x_0 + (t - t_\mu)v$ .
- Temporal Modulation: Opacity is modulated by a Gaussian function over time, ensuring the primitive appears and disappears smoothly.
Alpha Loss: To prevent dynamic Gaussians from incorrectly populating static background regions, an additional alpha loss is applied, penalizing opacity in areas where the motion mask indicates no fire.

3. Hardware Synchronization

To capture high-frequency fire dynamics (up to 400 Hz) with consumer-grade hardware (GoPro Hero 13), the authors designed a custom, low-cost synchronization system.

Mechanism: An ESP32 microcontroller controls 21 pins. Five pins toggle a series of COB LED strips to create a sub-frame synchronization pattern visible via the rolling shutter effect. Sixteen other pins drive a 15-LED Gray code counter for frame identification.
Precision: By detecting the brightness change of the LED strips on the rolling shutter, the system achieves sub-frame temporal alignment with approximately 15 $\mu$ s precision, essential for reconstructing rapid flame motion.

4. Evaluation Protocol (DVE)

Standard image metrics (PSNR, SSIM) are insufficient for evaluating dynamic 3D reconstruction. The authors propose the Density-Weighted Volumetric Evaluation (DVE) protocol for synthetic scenes with ground truth velocity fields.

Metrics: They compute a density-weighted relative $L_2$ error and a weighted volumetric cosine similarity between the reconstructed Gaussian velocities and the ground truth velocity field. This evaluates both the magnitude and directional alignment of the motion.

Key Contributions

Failure Analysis: The paper identifies sparse-view fire reconstruction as a specific failure case for existing dynamic GS methods, attributing this to the mismatch between persistent-primitive assumptions and the transient nature of fire.
Transient Gaussian Formulation: A novel representation that decouples static geometry from short-lived emissive structures, initialized via a fused 3D flow field rather than persistent tracking.
Practical Capture Setup: A low-cost, high-speed synchronization system enabling multi-view fire capture with commodity hardware, addressing the lack of affordable high-frame-rate synchronization solutions.
Evaluation Protocol: A new metric suite (DVE) that assesses the physical plausibility of reconstructed motion fields, demonstrating that prior GS methods fail to recover correct dynamics even when visual metrics appear competitive.

Results

The method was validated on a real-world dataset of 17 fire sequences (propane, wood, gasoline, etc.) captured at 400 Hz with three cameras, as well as synthetic Mantaflow simulations.

Visual Quality: The proposed method outperforms baselines (4DGS, FreeTimeGS, Grid4D) in all visual metrics (PSNR, SSIM, LPIPS) for both the flame regions and the full image.
Depth Consistency: The method achieves significantly lower depth error (RMSE) compared to baselines, which often suffer from depth collapse or spatial instability.
Motion Accuracy: In synthetic evaluations, the proposed method demonstrates high similarity to ground truth velocity fields (CosSim $\approx$ 0.74), whereas baselines show weak directional alignment (CosSim $< 0.23$ ).
Ablation Studies: Removing the flow-based initialization (RandFlow) or the synchronization (NoSync) degrades performance. Jointly optimizing static and dynamic components (JointOpt) improves simulation metrics but reduces visual fidelity, highlighting a trade-off managed by the proposed decoupled approach.

Significance and Claims

The authors claim that their work bridges a critical gap in dynamic scene representation by demonstrating that sparse-view reconstruction of highly dynamic, emissive, and volumetric phenomena is feasible without heavy physics simulators or dense camera arrays.

By rethinking the unit of representation from "persistent primitives" to "transient volumetric carriers," the method achieves:

Robustness: Stable reconstruction across diverse flame types and lighting conditions.
Efficiency: Real-time rendering capabilities inherent to Gaussian Splatting, combined with a training pipeline that avoids the computational overhead of neural radiance fields (NeRFs).
Physical Plausibility: The ability to recover temporally coherent motion trajectories and depth fields that align with the physical behavior of fire, rather than just matching 2D pixel appearance.

The paper concludes that while the current linear velocity model and reliance on optical flow have limitations (e.g., in turbulent or low-texture regions), the proposed formulation provides a foundational step toward adaptive, learning-based dynamic reconstruction for complex volumetric phenomena.