Predicting Viticulture Potential through an Ensemble of U-Net and a Geospatial Foundation Model

Technical Summary: Predicting Viticulture Potential through an Ensemble of U-Net and a Geospatial Foundation Model

Problem Definition
The paper addresses the challenge of predicting agricultural suitability, specifically for viticulture, using remote sensing data. Traditional methods for determining land suitability involve costly, time-consuming physical surveys that require frequent reassessment to account for changing environmental conditions and climate change. The authors focus on ImageCLEF AI4Agri 2026: Subtask 1, which utilizes the AgriPotential dataset to classify Sentinel-2 satellite multispectral imagery. The goal is to assign a pixel-level agricultural suitability score from 1 to 5 for viticulture in Southern France. The dataset presents specific challenges, including a high proportion of unlabeled pixels (approximately 50% across splits), a large volume of data (~200GB), and the need to model temporal dynamics across 34 time steps (2017–2019).

Methodology
The DS@GT ARC team proposed a weighted ensemble of two distinct deep learning models, both trained independently and combined at inference time.

1. U-Net Architecture (Temporal Stacking):

   • Input Strategy: Instead of explicit temporal modeling, the authors treated the 34 time steps and 15 channels (10 raw Sentinel-2 bands + 5 spectral indices including NDVI, NDMI, NDWI, NDRE, and NBR) as a single input tensor of $34 \times 15 = 510$ channels.
   • Architecture: A standard U-Net extended with residual connections, a base channel dimension of 128, and a depth of 3.
   • Loss Function: The model employs a binary cross-entropy ordinal loss, which accounts for the ordered nature of the suitability classes (1–5). Unlabeled pixels are excluded from the loss calculation.
   • Output: A single-channel output passed through an ordinal layer with 4 learnable thresholds (cumulative sum of softplus-transformed values) to generate binary decision boundaries.

2. Prithvi-EO-2.0 Foundation Model (Seasonal Aggregation):

   • Input Strategy: To align with the pretraining configuration of the Prithvi-EO-2.0 model, the 34 time steps were aggregated into four seasonal means. Only the six spectral bands used during Prithvi's pretraining (Blue, Green, Red, NIR, SWIR-1, SWIR-2) were utilized.
   • Architecture: The model fine-tunes the prithvi_eo_v2_100_tl backbone (100M parameters) with a Feature Pyramid Neck and a UperNet decoder. It incorporates temporal and location coordinate encodings.
   • Training Strategy: The encoder was frozen for the first 7 epochs. The model utilizes a teacher-student pseudo-labeling approach: the trained U-Net acts as a fixed teacher, generating pseudo-labels for unlabeled pixels when confidence exceeds 0.7. These labels are incorporated into the student's loss with a weight of $\lambda=0.3$.
   • Initialization: Ordinal thresholds were initialized from the trained U-Net values to ensure stable convergence.

3. Ensemble Strategy:

   • The final prediction is a weighted logit ensemble: $z = 0.65 \cdot z_{\text{U-Net}} + 0.35 \cdot z_{\text{Prithvi}}$.
   • Data Augmentation: Rotations and flips were applied to bridge the generalization gap between validation and test sets.
   • Normalization: U-Net utilized division by 10,000, while Prithvi used Z-score normalization per dataset split to maintain consistency with pretraining distributions.

Key Results

• Performance: The ensemble achieved a test $\pm 1$ accuracy of 68.32%, ranking 2nd among 7 teams on the public leaderboard.
• Model Comparison: The ensemble outperformed the individual components:
  • U-Net: 66.25%
  • Prithvi: 65.51%
  • Other transformer architectures (TSViT, U-TAE, Swin-V2, Presto) yielded lower test accuracies (ranging from 49% to 64%).
• Observations:
  • The U-Net produced noisier predictions but captured fine-grained spatial patterns.
  • Prithvi generated smoother, more spatially coherent outputs but struggled with complex boundaries.
  • A consistent generalization gap was observed between validation and test sets across all models, suggesting a potential train-test distribution shift or overfitting.

Key Contributions and Claims

• Ensemble Efficacy: The primary contribution is demonstrating that combining a convolutional approach (U-Net with stacked temporal channels) and a geospatial foundation model (Prithvi with seasonal aggregation) yields superior performance compared to either model alone or other complex temporal architectures.
• Temporal Modeling Insight: Contrary to the initial hypothesis that explicit temporal modeling (e.g., via Transformers) would yield the best results, the paper claims that stacking time frames as input channels (treating them as distinct observations for the same pixel) was more effective for this specific task. The authors suggest that semantic segmentation is performed across time frames rather than for each time frame in this context.
• Foundation Model Adaptation: The work demonstrates the viability of adapting a large-scale geospatial foundation model (Prithvi-EO-2.0) to a specific agricultural task using seasonal aggregation and semi-supervised pseudo-labeling to handle unlabeled data.
• Semi-Supervised Learning: The paper highlights the utility of using a U-Net as a teacher to generate pseudo-labels for unlabeled pixels in the Prithvi student model, providing a regularizing effect.

Significance
The paper presents a practical, high-performing approach for predicting viticulture potential from multitemporal satellite imagery. It challenges the assumption that complex temporal architectures are always necessary for time-series remote sensing tasks, suggesting that for fixed-time-frame segmentation, channel-stacking can be a robust baseline. The work also underscores the importance of addressing the generalization gap in remote sensing applications, where distribution shifts between validation and test sets remain a significant challenge. The implementation is made publicly available to support further research in agricultural potential mapping.