Space2Ground 2.0: A Multi-Source Dataset and Framework for Agricultural Monitoring through Fusion of Street-Level and Satellite Imagery

Technical Summary: Space2Ground 2.0

Problem Statement
Accurate, scalable, parcel-level agricultural monitoring faces significant limitations when relying solely on satellite Earth Observation (EO). Optical sensors (e.g., Sentinel-2) are hindered by cloud cover and atmospheric conditions, while Synthetic Aperture Radar (SAR) data (e.g., Sentinel-1), though all-weather, is complex to interpret and less directly linked to specific crop characteristics. Furthermore, the spatial resolution of standard satellite systems often proves insufficient for small or heterogeneous fields. While Very High Resolution (VHR) commercial satellites and aerial imagery offer finer detail, their high costs and limited revisit frequencies restrict large-scale operational use. Conversely, traditional in-situ field surveys are too costly and labor-intensive to provide the continuous, scalable ground-truth data required for modern, data-driven agricultural policies like the Common Agricultural Policy (CAP). Although crowdsourced street-level imagery offers a promising, cost-effective alternative, its integration into operational workflows is hindered by heterogeneous acquisition conditions, sparse coverage, noisy content, and the lack of direct parcel-level annotations.

Methodology
The paper presents Space2Ground 2.0, an automated, end-to-end framework designed to transform crowdsourced street-level imagery into analysis-ready, parcel-linked data by fusing it with Sentinel-1 and Sentinel-2 time series. The methodology was applied to the 2022 growing season in Cyprus, utilizing a reference dataset of 325,673 agricultural parcels from the national Geospatial Aid Application (GSAA).

The processing pipeline consists of four primary stages:

1. Data Acquisition and Retrieval: The framework ingests over 900,000 geo-tagged images collected via vehicle-mounted cameras (GoPro and smartphone) and uploaded to the Mapillary platform. Metadata (timestamps, coordinates, orientation) and computer vision products (object detections, semantic segmentation) are retrieved via the Mapillary API.
2. Semantic Filtering: To remove non-agricultural content (roads, buildings), the pipeline uses Mapillary's semantic segmentation outputs. Images where vegetation and soil occupy less than 20% of the visible scene are discarded, reducing the dataset to ~506,000 images.
3. Image Quality Assessment (NR-IQA): A no-reference image quality assessment employs four deep learning models (MANIQA, HyperIQA, CLIP-IQA, TReS). Images falling in the lowest 5% of quality scores by at least three models are removed, further refining the collection to ~472,000 images.
4. Parcel Association and Refinement:
   • Viewpoint Projection: Using camera location and orientation (or reconstructed vehicle heading), a viewpoint is projected 10 meters ahead to identify the observed agricultural parcel. This simplifies the association process compared to previous front-facing camera approaches.
   • Label Assignment: Images are assigned the crop type label of the intersected parcel from the GSAA database.
   • Clustering and Curation: Deep visual features (VGG-16) are reduced via PCA and clustered (k-means). Clusters dominated by noise or incorrect associations are manually inspected and discarded.
   • Final Output: The pipeline yields a curated dataset of 46,050 annotated street-level images linked to 8,581 agricultural parcels across 14 crop classes.

Key Contributions
The paper outlines three main contributions:

1. Automated Methodology: A scalable pipeline for converting raw, opportunistic crowdsourced imagery into structured, parcel-linked datasets through semantic filtering, quality control, and spatial association.
2. Multi-Source Framework: A reproducible workflow for integrating satellite EO time series with street-level imagery to support agricultural monitoring, visual verification, and dispute resolution (e.g., Area Monitoring System checks).
3. Open Benchmark Dataset: The release of the Space2Ground 2.0 dataset, accompanied by baseline machine learning/deep learning evaluations for multimodal crop classification, facilitating reproducible research.

Experimental Results
The utility of the dataset was evaluated through parcel-level crop classification experiments comparing satellite-only, street-level-only, and multimodal fusion approaches.

• Satellite-Only: Sentinel-1 and Sentinel-2 time series proved the strongest single modality. The XGBoost model achieved the highest overall accuracy (78.90%) and F1-score (69.09%) among single-modality baselines.
• Street-Level-Only: Standalone performance was lower (ViT-B/16 achieved 70.17% accuracy), attributed to the opportunistic nature of the data (limited temporal coverage per parcel) and acquisition uncertainties (occlusions, GPS errors).
• Multimodal Fusion: Integrating both modalities yielded significant improvements.
  • Early Fusion (Feature-level): Concatenating VGG-16 embeddings with satellite features improved accuracy to 82.65%.
  • Late Fusion (Decision-level): Combining predictions from an XGBoost satellite model and a ViT-B/16 street-level model achieved the best performance: 84.12% overall accuracy and 82.78% F1-score.
  • Conclusion: The results demonstrate that street-level imagery provides complementary fine-scale visual information (crop structure, canopy appearance) that enhances classification when fused with satellite time series, particularly improving class-balanced performance (F1-score).

Significance and Claims
The authors position Space2Ground 2.0 as a practical solution to bridge the gap between the increasing availability of heterogeneous remote sensing data and their application in scalable agricultural monitoring. The framework offers a pathway to:

• Enhance the accuracy of parcel-level monitoring and support "desk on-the-spot-checks" (OTSC) under evolving CAP regulations.
• Reduce reliance on costly and labor-intensive physical field inspections.
• Provide a reproducible methodology for generating parcel-linked datasets that can be transferred to other regions with similar data availability.

The paper modestly acknowledges limitations, including road-accessibility constraints, potential occlusions, GPS inaccuracies, and the inherent noise in farmer-declared reference labels (GSAA). However, it asserts that the proposed pipeline, which includes lightweight human-in-the-loop refinement, effectively transforms weakly structured crowdsourced data into a reliable resource for multimodal agricultural AI applications.