🤖 machine learning

Leveraging Multi-Temporal Sentinel 1 and 2 Satellite Data for Leaf Area Index Estimation With Deep Learning

This paper proposes a novel deep learning framework that integrates multi-temporal Sentinel-1 and Sentinel-2 satellite data using pre-trained U-Net modules and a seasonality-aware decoder to achieve highly accurate pixel-wise Leaf Area Index estimation with an RMSE of 0.06 and R² of 0.93.

Original authors: Clement Wang, Antoine Debouchage, Valentin Goldité, Aurélien Wery, Jules Salzinger

Published 2026-05-22

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Clement Wang, Antoine Debouchage, Valentin Goldité, Aurélien Wery, Jules Salzinger

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out how many leaves are on a tree in a massive forest, but you can't walk there. Instead, you are looking at the forest from space. This is the challenge of measuring the Leaf Area Index (LAI)—a fancy way of saying "how much leafy green stuff is covering the ground." This number is crucial for understanding how healthy our planet is, how much food crops will produce, and how the climate is changing.

Usually, scientists try to get this number by sending people into the field to count leaves (which is slow and hard) or by using satellites that take pictures. But here's the problem: clouds. Clouds are like a giant, fluffy blanket that blocks the satellite's view, making the optical pictures useless.

This paper presents a clever new way to solve this problem using Deep Learning (a type of AI that learns like a brain) and two different types of satellite "eyes": Sentinel-2 (which takes colorful photos) and Sentinel-1 (which uses radar).

Here is how their method works, explained with some everyday analogies:

1. The Two Eyes: Photos vs. Radar

Think of Sentinel-2 as a photographer. It takes beautiful, detailed pictures of the land, but if it's cloudy, the photo is just a white blur.
Think of Sentinel-1 as a bat using echolocation. It sends out radar waves that can see right through clouds and fog. It doesn't give you a pretty picture, but it tells you the shape and texture of the ground even when the sky is overcast.

The researchers realized that if you combine the "pretty photos" (when the sky is clear) with the "radar echoes" (when the sky is cloudy), you get a much clearer picture of the forest than using either one alone.

2. The Time Machine: Looking at History

The AI doesn't just look at one day; it looks at three days in a row.
Imagine trying to guess how much a plant has grown. If you only look at today, you might be confused. But if you look at what it looked like yesterday and the day before, you can see the pattern. The AI uses this "time machine" approach to understand how the leaves are changing over a short period, which helps it make smarter guesses even if one of the days is cloudy.

3. The Brain: A Specialized Team

The AI they built is like a team of specialists working together, rather than one generalist trying to do everything.

Two Encoders (The Specialists): One specialist focuses only on the radar data and the calendar (seasons). The other specialist focuses on the past photos and the calendar. They work in parallel, each getting really good at understanding their specific type of data.
One Decoder (The Manager): Once the specialists have done their analysis, they hand their notes to a "Manager" (the decoder). This Manager combines the radar info, the photo info, and the seasonal context to draw the final map of leaf density.

4. Training the Brain

Teaching this AI is tricky because the data is complex. The researchers used a two-step training method:

Step 1: They taught the two specialists separately first, so they became experts in their own fields.
Step 2: They put the whole team together and fine-tuned them to work as a unit. They also made sure the AI paid attention to the "season" (like whether it's spring or winter), because leaves look very different depending on the time of year.

The Results

When they tested this system:

On clear days: It was incredibly accurate, almost perfect.
On cloudy days: It did well, but not as perfectly as on clear days. The researchers found that the AI struggled a bit here because there weren't enough examples of "cloudy days" in the training data to teach it how to rely heavily on the radar.
Compared to old methods: Traditional methods that just look at one pixel at a time failed miserably (like trying to guess the weather by looking at a single raindrop). This new method, which looks at the whole picture and context, was a massive success.

The Bottom Line

The paper shows that by combining radar and optical satellite data, and by using a smart AI that understands time and seasons, we can estimate how healthy our vegetation is even when the weather is bad. The authors have shared their code so others can build on this work, hoping to make these "leaf counters" even better in the future.

Technical Summary: Leveraging Multi-Temporal Sentinel-1 and Sentinel-2 Data for Leaf Area Index Estimation

Problem Statement
The Leaf Area Index (LAI) is a fundamental metric for assessing ecosystem health, plant productivity, and carbon cycles. While traditional ground-based methods (destructive sampling, allometric equations) provide accurate local data, they lack spatial coverage. Satellite remote sensing offers a solution for regional to global monitoring; however, optical data (e.g., Sentinel-2) is frequently compromised by cloud cover, which obstructs direct measurements and reduces data availability. Existing deep learning approaches for LAI prediction often rely on single timestamps, single sensor types (optical or radar), or treat pixels independently, thereby ignoring valuable contextual information from neighboring pixels and the complementary nature of multi-sensor data.

Methodology
The authors propose a novel deep learning framework for pixel-wise LAI prediction that integrates multi-temporal data from Sentinel-1 (radar) and Sentinel-2 (multi-spectral) satellites. The core of the methodology involves a custom deep neural network architecture designed to handle heterogeneous input modalities and temporal dependencies.

Dataset Construction: The study utilizes publicly available Sentinel-1 and Sentinel-2 data from various European locations. Samples consist of three consecutive timestamps ( $T \in \{t-2, t-1, t\}$ ) separated by no more than a week. Inputs include:
- Sentinel-1 radar data (VH and VV polarizations).
- Sentinel-2 derived LAI maps (for previous timestamps) and semantic masks (identifying clouds, water, land, etc., generated via the Sen2Cor processor).
- Seasonality information encoded as sine and cosine values representing the day of the year.
- The ground truth is the LAI map at timestamp $t$ .
- The dataset is split into "non-cloudy," "cloudy," and "unique areas" (Czech Republic and Italy) to test generalization.
Architecture: The model features two parallel encoders and a single decoder (Figure 1):
1. Encoder 1: Processes Sentinel-1 radar data, semantic masks, and seasonality.
2. Encoder 2: Processes historical Sentinel-2 LAI values ( $t-2, t-1$ ), masks, and seasonality.
3. Decoder: A U-Net structure that concatenates the feature maps from both encoders to produce the final pixel-wise LAI prediction. The design prioritizes the early shrinkage of multi-temporal data to reduce redundancy.
Training Strategy: Due to the architecture's complexity, a two-step training approach is employed:
1. Pre-training: Each encoder is trained separately with a pixel-wise convolution layer to predict LAI, optimizing Mean Squared Error (MSE).
2. End-to-End Fine-tuning: The pre-trained weights are integrated into the full network. The model is optimized using intermediate supervision, where the loss function combines the MSE of the final decoder output with the outputs of the intermediate encoder stages. The loss function is defined as:
  $Loss = MSE(LAI_{gt}, LAI_{dec}) + \alpha \cdot MSE(LAI_{gt}, LAI_{enc1}) + \beta \cdot MSE(LAI_{gt}, LAI_{enc2})$
  where $\alpha$ and $\beta$ are hyper-parameters (set to 0.1 and 0.15, respectively) to weigh the importance of intermediate stages. The loss calculation explicitly excludes pixels covered by clouds.

Key Results
The proposed method was evaluated on a test set of 256x256 image sections.

Performance: The full model achieved an RMSE of 0.058 and an $R^2$ score of 0.930 on non-cloudy data.
Ablation Studies:
- Using only Sentinel-1 data yielded poor results ( $R^2 = 0.434$ ).
- Using only Sentinel-2 data (with masks and seasonality) performed well ( $R^2 = 0.903$ ) but showed significant degradation on cloudy data ( $R^2 = -0.294$ ).
- The combined approach (All inputs) maintained robust performance across datasets, though performance on "unique areas" (unseen locations) showed higher error rates, suggesting some location-specific bias.
Comparison: The deep learning approach significantly outperformed traditional pixel-wise baselines like Multi-Linear Regression ( $R^2 = -0.0445$ ) and Random Forest ( $R^2 = -0.0166$ ) when tested on single-pixel data without spatial context.

Significance and Claims
The paper claims to present a pioneering investigation into pixel-wise LAI prediction that effectively harnesses the combined, multi-temporal information of Sentinel-1 and Sentinel-2 data via a deep neural network. Key contributions include:

Novel Architecture: The use of dual parallel encoders tailored to specific sensor modalities, unified by a decoder that accounts for seasonality and spatial context.
Handling Complexity: The implementation of intermediate supervision and pre-training to manage the complexity of multi-modal inputs and ensure effective feature representation.
Contextual Awareness: Moving beyond independent pixel prediction to leverage global context and neighboring pixel information, addressing a frequent limitation in prior works.
Open Contribution: The authors have made their code and contributions available to facilitate future improvements in the field.

The authors modestly note that while the method performs well, challenges remain regarding data imbalance (specifically the scarcity of cloudy training samples) and generalization to entirely new geographic locations. They suggest that future work could benefit from assigning higher weights to loss functions for cloudy inputs and incorporating additional meta-information such as terrain or meteorological data.