💻 computer science

A Nationwide Benchmark for Wildfire Initial Attack Failure Prediction with Public Environmental Data

This paper introduces WILDFIREIA, the first nationwide U.S. benchmark for predicting wildfire initial attack failures using only public environmental and contextual data available at fire discovery time, establishing standardized protocols and evaluating 16 models to demonstrate that while such data offers useful signals, it provides an incomplete picture of fire escalation risks.

Original authors: Runyang Xu, Xueqi Cheng, Yushun Dong

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Runyang Xu, Xueqi Cheng, Yushun Dong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a wildfire as a sudden, unexpected guest arriving at a party. The "Initial Attack" is the moment the hosts (firefighting agencies) first see this guest and decide: "Can we handle this with a quick chat and a glass of water, or is this guest going to wreck the whole house?"

The paper you provided, WILDFIREIA, is like a massive, standardized "training simulator" built to help computers learn how to make that split-second decision using only the information available the moment the guest arrives.

Here is a breakdown of what the researchers did, using simple analogies:

1. The Problem: Guessing the Future with a Blindfold

In the past, studies trying to predict if a fire would get out of control relied on secret "playbooks" (like how many firefighters were sent, how fast they arrived, or specific agency logs). These are like the hosts' private notes. Because these notes aren't public, it's hard for scientists to compare their predictions fairly.

Also, many studies only looked at small towns (regional data). The researchers wanted to know: Can we predict a fire's danger using only public, open-source data available the second the fire is spotted?

2. The Solution: The "WILDFIREIA" Simulator

The authors built a giant, national-scale benchmark (a test bed) called WILDFIREIA. Think of this as a giant video game level where:

The Players: 38,128 real wildfires that happened naturally in the US between 2016 and 2020.
The Rules: The computer is strictly forbidden from peeking at the future. It cannot know the final size of the fire, how long it took to put out, or what happened after the first report. It only gets the "snapshot" of the world at the exact moment the fire was discovered.
The Ingredients: The computer is fed a "soup" of public data:
- The Fire Signal: Satellite heat detections (like a thermal camera seeing the fire's glow).
- The Weather: Temperature, wind, and humidity (like checking if the wind is blowing the fire toward the house).
- The Landscape: What kind of trees and dry grass are there? (The fuel).
- The Terrain: Is it a steep mountain or a flat field?
- The Access: How close are the roads and fire stations?
- The People: How many people live nearby?

3. The Experiment: Who Wins the Prediction Game?

The researchers tested 16 different types of AI models (from simple math formulas to complex neural networks) to see which one could best guess if a fire would escape control.

The Results:

The Winner: A model called XGBoost (a type of advanced decision tree) performed the best. It's like a seasoned veteran who looks at the facts and makes a solid guess without overcomplicating things.
The Most Important Clue: The satellite heat signal (FIRMS/VIIRS) was the most unique piece of information. If you take away the satellite data, the AI gets much worse at guessing. It's the "smoke in the air" that tells you the fire is real and active right now.
The Best Static Clue: If you can't see the fire or the weather (no satellite, no current weather data), the type of fuel (dry grass vs. wet forest) is the best thing to look at. It's like knowing the house is made of paper vs. stone; even without seeing the fire, you know paper burns faster.
The Limit: The AI is good at saying "This fire might get big," but it's not great at predicting exactly how long it will take to put out. Why? Because once the fire starts, human decisions (how many trucks to send, where to send them) and changing weather play a huge role that the AI couldn't see at the start.

4. Why This Matters (According to the Paper)

The paper argues that we now have a fair, public, and reproducible way to test wildfire prediction tools.

No Cheating: They made sure no one could "cheat" by using future information (like the final fire size) to train the AI.
Standardized: Everyone can now run their own models on this same dataset and compare scores fairly, just like athletes competing in the same Olympics.
Realistic: It shows that while public data is helpful, it's not magic. It gives a "useful but incomplete" signal. It helps agencies prioritize which fires to watch closely, but it can't predict the future perfectly.

Summary Analogy

Imagine you are a doctor trying to diagnose a patient based only on a photo taken the moment they walked into the ER. You can't see their blood work results yet (future data), and you don't know how they will respond to medicine (human intervention).

WILDFIREIA is the dataset of 38,000 patients where the authors tried to teach computers to say, "This patient looks like they might get sicker," using only the photo, the weather outside, and the patient's age. They found that while the computer can make a decent guess, the most important thing is seeing the actual symptom (the fire's heat) right now, and the patient's underlying condition (the dry fuel) is the next best clue.

Technical Summary: A Nationwide Benchmark for Wildfire Initial Attack Failure Prediction with Public Environmental Data

Problem Statement

The initial attack (IA) phase is the critical window where firefighting resources attempt to halt a wildfire immediately after discovery. Agencies face a triage problem: determining whether a reported fire can be contained by initial response or will escape early control. Underestimating this risk leads to delayed escalation, resulting in larger, costlier, and more dangerous suppression efforts.

Existing research on IA failure prediction faces two primary barriers:

Data Availability: Many studies rely on non-public operational records (e.g., dispatch times, crew sizes, resource logs) that are unavailable at the moment of fire discovery or in public national databases.
Lack of Standardization: Evaluations are often regional or agency-specific, using inconsistent labels, response standards, and evaluation protocols, making cross-study comparison impossible.

Consequently, it remains unclear how well public environmental data available at fire discovery time can support IA failure prediction at a national scale, which data sources provide non-redundant signals, and which models are reliable under a unified protocol.

Methodology: The WILDFIREIA Benchmark

The authors introduce WILDFIREIA, the first U.S. national-scale benchmark for predicting IA failure using only public data available at the time of fire discovery.

1. Data Construction and Alignment

The benchmark aligns 38,128 naturally caused wildfire events from the FPA-FOD (Fire Program Analysis - Fire Occurrence Database) with five categories of public environmental data:

Event Backbone: FPA-FOD records (2016–2020) providing discovery time, location, and final size.
Discovery-Day Fire Signal: FIRMS/VIIRS thermal detections (375m resolution) matched to the discovery day only.
Weather & Fire Danger: gridMET data (approx. 4km resolution) covering temperature, humidity, wind, and fire-danger indices up to the discovery day.
Static Landscape Context: LANDFIRE data (30m resolution) for vegetation, fuel, and topography.
Access & Population: OpenStreetMap (road density, fire station proximity) and WorldPop (population density).

2. Leakage Control and Task Definition

To ensure a fair and reproducible evaluation, WILDFIREIA enforces strict leakage-control rules:

Event Unit: The sample unit is the reported wildfire event, not the satellite detection.
Label Construction: IA failure is defined as a binary label ( $y_i=1$ ) for fires $\ge$ 50 ha and success ( $y_i=0$ ) for fires $\le$ 10 ha. Intermediate sizes (10–50 ha) are excluded to reduce ambiguity.
Forbidden Features: Final fire size, containment timestamps, post-discovery satellite detections, and any operational variables (e.g., crew size) are strictly excluded from model inputs.
Temporal Split: A chronological split is used: 2016–2018 for training, 2019 for validation, and 2020 for testing.

3. Evaluation Protocol

The benchmark evaluates 16 representative models across four families:

Tabular: Logistic Regression, XGBoost, MLP.
Temporal: GRU, TCN, Transformer (processing weather history).
Spatial: ResNet-UNet, Swin-UNet, SegFormer (processing landscape patches).
Spatiotemporal: ConvLSTM, ConvGRU, ResNet3D, PredRNN-V2, UTAE, SwinLSTM.

Models are tested on five specific research questions (RQs):

Full-source predictability: Can public data predict IA failure?
Source necessity: Which sources are non-redundant?
Static-source value: What is the value of static data without dynamic observations?
Weather-history sufficiency: How many days of weather history are needed?
Containment-duration signal: Can discovery-time data predict containment duration (an auxiliary task)?

Key Results

1. Predictability and Model Performance

Public data provides useful but incomplete signal: All model families outperform trivial baselines, confirming that IA failure is predictable from public discovery-time data.
Best Performer: XGBoost achieved the highest performance with an AUPRC of 53.3%, outperforming complex spatiotemporal deep learning models. This suggests the task behaves more as an event-level risk prediction problem than a dense spatial forecasting task.
Recall Limitations: Even the best models achieved a Recall@5% of only ~38%, indicating that public discovery-time inputs alone cannot fully determine suppression outcomes.

2. Source Necessity (Ablation Studies)

FIRMS/VIIRS is the least redundant source: Removing discovery-day thermal evidence caused the largest performance degradation across all models.
Fuel is the strongest static fallback: In the absence of dynamic observations (weather and satellite), fuel data provided the strongest predictive signal among static sources.
Redundancy: Vegetation, topography, access, and population data showed marginal or slightly negative contributions when full-source inputs (including FIRMS and weather) were available, suggesting their signals are partially covered by other sources.

3. Weather History and Static Value

Weather History: Longer weather histories (3–5 days) did not consistently improve prediction over shorter windows (1–2 days). The most useful signal is concentrated near the discovery time.
Static Sources: While static sources (especially fuel) improve prediction over metadata-only baselines, they are insufficient on their own to replace current fire signals or near-discovery weather.

4. Containment Duration

The auxiliary task of predicting containment duration yielded low $R^2$ values (~0.18), indicating that post-discovery factors (tactics, resource availability, weather evolution) dominate the duration outcome and are not captured by discovery-time public data.

Significance and Contributions

The paper claims the following contributions:

First U.S. National Benchmark: WILDFIREIA standardizes IA failure prediction as an event-level task with fixed rules, enabling reproducible comparison across model families and data sources.
Leakage-Controlled Infrastructure: It provides a multi-source data pipeline that strictly excludes outcome-derived information, ensuring that models are evaluated on what is actually available at fire discovery.
Comprehensive Baseline Suite: By evaluating 16 models across tabular, temporal, spatial, and spatiotemporal paradigms, it establishes a performance reference for future research.
Empirical Insights: The study clarifies the specific value of public data sources (highlighting FIRMS/VIIRS and fuel) and demonstrates the limits of predicting suppression outcomes using only pre-discovery environmental variables.

The authors emphasize that while public data offers meaningful early-risk signals, it does not fully determine suppression outcomes, and the benchmark is best suited for early event-level triage rather than precise containment duration forecasting. The code and data caches are released to support reproducible research in early wildfire risk assessment.