🤖 AI

Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation

This paper proposes and evaluates two novel methods, Input Sensor Dropout (ISensD) and Ensemble Sensor Invariant (ESensI), demonstrating through experiments on three Earth Observation datasets that ensemble-based multi-sensor models and sensor dropout techniques significantly enhance prediction robustness against missing sensor data.

Original authors: Francisco Mena, Diego Arenas, Andreas Dengel

Published 2026-05-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Francisco Mena, Diego Arenas, Andreas Dengel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather, the type of crop growing in a field, or the air quality in a city. To do this accurately, you don't rely on just one tool; you use a whole team of "sensors." Some are like cameras taking pictures (optical), some are like radar guns seeing through clouds (radar), and others measure things like wind or soil moisture.

In the world of Earth Observation, scientists build computer models (AI) that act like a super-smart detective, combining all this data to make predictions. The problem? Sometimes, a sensor breaks, gets blocked by clouds, or simply forgets to send data. When a piece of the puzzle is missing, the detective gets confused, and the prediction becomes unreliable.

This paper introduces two new training techniques to teach these AI detectives how to stay calm and accurate even when their team members go missing.

The Problem: The "Missing Team Member"

Think of your AI model as a sports team. If you train the team only when everyone is present, they might struggle if a key player gets injured during the actual game. In Earth Observation, "missing players" happen often. For example, a camera sensor might be blinded by a storm cloud, leaving the model with only radar data. Traditional models often crash or give bad answers in these situations.

The Solution: Two New Training Drills

The authors propose two specific ways to train these models so they become "robust" (tough and adaptable) to missing data.

1. Input Sensor Dropout (ISensD): The "Blindfold Drill"

The Analogy: Imagine a coach training a basketball team. To prepare them for a game where a player might get sick, the coach practices by randomly telling one player to sit out for a few minutes during every practice session. The remaining players have to figure out how to win without that specific person.

How it works:

During the training phase, the computer randomly "turns off" (masks) entire sensors. Sometimes it hides the camera data; other times it hides the radar data.
The model is forced to learn how to make good predictions using whatever data is left.
The Result: Just like the basketball team, the model learns to adapt. If the camera is missing during the real game, the model doesn't panic; it knows how to use the radar data effectively. The paper found this method makes the model much more stable when sensors drop out.

2. Ensemble Sensor Invariant (ESensI): The "Specialized Squad with a Shared Captain"

The Analogy: Imagine you have three different experts: a Weatherman, a Soil Scientist, and a Satellite Analyst.

Old Way: You ask each expert to give their own prediction, and then you average them. If the Satellite Analyst is missing, you just ignore their part. This works okay, but the experts never really talk to each other.
New Way (ESensI): You give each expert their own private notebook (encoder) to analyze their specific data. However, they all write their final report to the same shared Captain (a shared prediction head).
The Captain also has a little "ID card" for each expert that says, "This data is from the Satellite Analyst." This helps the Captain understand the context of the data, even if it's different from the others.

How it works:

The model trains separate "brains" for each sensor type but forces them to all agree on a final answer using the same "brain" at the end.
The Result: This creates a team that is very good at handling missing members. If the Satellite Analyst is missing, the Captain can still make a solid decision based on the other experts because they are all trained to work together under the same rules.

What the Experiments Showed

The researchers tested these methods on three real-world scenarios:

Crop Harvesting: Identifying what crops are growing.
Fuel Moisture: Measuring how dry vegetation is (important for fire risk).
Air Quality: Measuring pollution levels.

They simulated missing sensors by randomly removing data from 10% up to 100% of the test cases.

The Winner: The Ensemble method (ESensI) and the standard Ensemble approach (without the new training tricks) were the most robust. They kept their performance high even when sensors were missing.
The Runner-Up: The Blindfold Drill (ISensD) was also very good, significantly better than standard models that just try to fill in the missing numbers with guesses.
The Catch: There is a trade-off. When all sensors are present and working perfectly, the new methods sometimes performed slightly worse than the standard models. It's like the basketball team practiced so hard for "missing players" that they were slightly less efficient when everyone was actually there. However, the authors argue that being reliable when things go wrong is more important than being perfect when everything is easy.

The Bottom Line

This paper teaches us that if you want an AI to be reliable in the messy real world (where sensors fail), you have to train it to handle failure. By randomly hiding data during training (ISensD) or forcing different sensors to share a common decision-maker (ESensI), we can build Earth Observation models that don't break when the data gets incomplete.

Technical Summary: Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation

Problem Statement

Multi-sensor Machine Learning (ML) models for Earth Observation (EO) integrate data from diverse sources (e.g., optical, radar, weather) to enhance prediction accuracy. However, the persistence of sensor data cannot be guaranteed in real-world deployments due to technical failures or environmental factors (e.g., cloud cover hindering optical imagery). Existing literature has addressed missing data through strategies like temporal dropout (focused on single-sensor inputs) or sensor-invariant models (focused on static images). A significant gap remains in handling missing sensors within multi-sensor time-series scenarios, where standard ML models lack inherent robustness, leading to biased or failed predictions when specific sensor inputs are unavailable.

Methodology

The authors propose two novel methods tailored specifically for multi-sensor time-series scenarios to enhance model robustness against missing sensors:

1. Input Sensor Dropout (ISensD)

Inspired by data augmentation techniques, ISensD introduces a Sensor Dropout (SensD) mechanism at the input level during training.

Mechanism: For a given sample, the input features of entire sensors are randomly masked out.
Implementation:
- Standard SensD: Uses a hyperparameter $r$ (dropout ratio) where sensor features are masked based on a Bernoulli distribution ( $d_s \sim \text{Bernoulli}(r)$ ).
- No Ratio (NR): A parameter-free variant that randomly selects one combination from all possible missing sensor combinations ( $2^{|S|} - 1$ ) for each batch.
Training: The model is trained on these masked inputs using a standard multi-sensor fusion architecture (input-level fusion), optimizing parameters to minimize loss against ground truth.

2. Ensemble Sensor Invariant (ESensI)

Inspired by sensor-invariant models, ESensI proposes an ensemble architecture with a shared prediction head.

Mechanism: The model employs sensor-dedicated encoders ( $E_{\theta_s}$ ) for each sensor to extract features, but utilizes a single, shared prediction head ( $P_\phi$ ) for all sensors.
Sensor Encoding: To ensure the model is aware of which sensor features are being processed, a learnable sensor encoding vector ( $\rho_s$ ) is added to the encoded features before the shared head.
Optimization: Unlike standard ensembles where sensor-specific heads are optimized independently, ESensI jointly optimizes the sensor-dedicated parameters and the shared head parameters, forcing the model to learn invariant representations across sensors.

Experimental Setup

The methods were validated on three multi-sensor temporal EO datasets:

CropHarvest (CH): Multi-crop recognition using Sentinel-2 (optical), Sentinel-1 (radar), ERA5 (weather), and SRTM (topography).
Live Fuel Moisture Content (LFMC): Vegetation moisture estimation using Landsat-8 (optical), Sentinel-1 (radar), and various static environmental sensors.
Particulate Matter 2.5 (PM25): Air quality regression using atmospheric conditions, dynamics, and precipitation data.

Baselines: The proposed methods were compared against four literature baselines:

Input: Standard input-level fusion with mean imputation.
ITempD: Input fusion with Temporal Dropout (adapted for sensors).
Feature: Feature-level fusion using exemplar-based (similarity search) imputation.
Ensemble: Aggregation of sensor-dedicated models, ignoring missing sensor predictions.

Evaluation Metric: The primary metric for robustness was the Performance Robustness Score (PRS), which measures the relative predictive error of models with missing sensors compared to models with full sensor availability.

Key Results

Robustness to Missing Sensors: Both ISensD and ESensI demonstrated significantly higher robustness (higher PRS values) as the percentage of missing sensors increased, outperforming standard baselines (Input, ITempD, Feature).
Ensemble Superiority: The standard Ensemble method (without the invariant head) proved to be the most robust architecture overall, often outperforming the proposed ESensI in raw robustness metrics. However, ESensI improved upon the Ensemble method in specific cases (e.g., PM25 dataset).
ISensD Performance: The ISensD method showed a marked improvement in robustness over its direct baselines (Input and ITempD). The parameter-free "No Ratio" (NR) variant performed comparably to the tuned ratio versions, suggesting it is a viable alternative.
Trade-off in Full-Sensor Performance: A critical observation was the trade-off between robustness and full-sensor accuracy. While the Ensemble and ESensI methods excelled in missing-sensor scenarios, they often exhibited lower predictive performance (F1/R2) when all sensors were available compared to the standard Input method. The authors note that ISensD performed worse than the Input baseline in full-sensor scenarios, suggesting the model may over-prioritize missing-data scenarios during training.

Significance and Claims

The paper claims to address a critical gap in EO literature by providing methods specifically designed for multi-sensor time-series scenarios where sensors may be missing.

Contribution: The introduction of ISensD and ESensI offers practical strategies to increase model robustness without requiring retraining when sensors fail.
Modest Claims: The authors acknowledge that while their methods improve robustness, they do not necessarily maximize predictive accuracy when all data is present. They explicitly state that the source of this robustness is not fully understood—it may stem from model adaptability, a disregard for specific sensor information, or the inherent irrelevance of certain sensor data for the specific prediction task.
Future Work: The authors identify the need to investigate the mechanisms behind this robustness and to balance the trade-off between missing-sensor resilience and full-sensor accuracy.