Weakly Supervised Spatio-Temporal Candidate Discovery of Dairy Farm Sites from Seasonal Satellite Imagery

Technical Summary: Weakly Supervised Spatio-Temporal Candidate Discovery of Dairy Farm Sites

Problem Statement
Discovering dairy farms from satellite imagery is a challenging weakly supervised spatiotemporal reasoning problem. Farms are rarely isolated visual objects; instead, they manifest as complex spatial configurations of pasture, field boundaries, roads, buildings, and seasonal vegetation patterns. These cues can span hundreds of meters and often resemble villages, industrial sites, or other agricultural land uses. Fully supervised detection is difficult because direct farm labels are often incomplete, boundaries are unavailable, and type-specific labels are sparse or noisy. While deep learning benchmarks exist for land use and scene understanding, they typically assume explicit labels or dense annotations, which are lacking for farm candidate discovery.

Methodology
The paper proposes a weakly supervised pipeline designed to rank spatial clusters likely to correspond to farm-associated sites, rather than producing a definitive farm inventory. The framework integrates seasonal remote sensing cues, weak geographic priors, self-supervised representation learning, and graph-based spatial reasoning.

1. Data and Input: The method utilizes aligned spring, summer, and autumn multispectral imagery from County Cork, Ireland. Each tile (128×128 pixels) contains 36 channels: six Sentinel spectral bands (B2, B3, B4, B8, B11, B12) and six derived indices (NDVI, NDWI, NDBI, NDRE, EVI, and a pasture channel) across three seasons.
2. Self-Supervised Embeddings: A ResNet-18 encoder is trained using the Barlow Twins objective on multi-season tiles without farm labels. This learns visual representations that capture temporal and spectral structures. The projection head is discarded, and the encoder output serves as tile embeddings.
3. Weak Supervision Setup: OpenStreetMap (OSM) farm-related features (farms, farmyards, barns) are split into two disjoint sets: a prior set ($O_{prior}$) and a held-out set ($O_{heldout}$). The prior set provides weak proximity cues during scoring, while the held-out set is reserved exclusively for proxy evaluation.
4. Candidate Scoring: Each tile receives a rule-based score ($f_i$) combining three evidence streams:
   • Weak Proximity ($q_i$): Normalized distance to the nearest feature in $O_{prior}$.
   • Pasture Evidence ($p_i$): A summary of pasture evidence across seasons.
   • Summer Greenness ($v_i$): Normalized summer NDVI.
     The primary score is calculated as $f_i = 0.40q_i + 0.35p_i + 0.25v_i$.
5. Graph Smoothing and Clustering: Scores are refined using a spatial-representation graph. Tiles are connected to their 8 nearest geographic neighbors. A smoothed score ($f^G_i$) is computed by propagating scores based on geographic proximity and embedding similarity. The final score is a weighted combination: $\tilde{f}_i = 0.85f_i + 0.15f^G_i$.
6. Cluster Formation: Tiles above the 92nd percentile of the final score are selected and grouped into connected components based on grid adjacency. These form ranked candidate clusters.

Key Contributions

1. Weak Supervision Pipeline: The paper presents a pipeline for dairy farm candidate discovery that operates without direct farm labels, relying instead on seasonal satellite imagery and open map priors.
2. Integrated Framework: It uniquely combines seasonal pasture dynamics, summer greenness, weak farm proximity, self-supervised tile embeddings, graph-based smoothing, and spatial clustering.
3. Proxy Evaluation Protocol: The authors introduce a protocol that splits OSM farm features into prior and held-out sets. This allows for evaluation against map evidence not used as priors, while explicitly acknowledging that the metrics measure candidate ranking against noisy map evidence rather than definitive farm detection accuracy.

Results
The framework was evaluated on 26,722 valid multi-season tiles from County Cork.

• Selection: The main run reduced the dataset to 535 high-confidence tiles, forming 71 candidate clusters.
• Precision: When ranked by the smoothed method score:
  • The top 5 clusters achieved 0.60 precision within 500 m and 0.80 precision within 1000 m of held-out OSM farm features.
  • The top 10 clusters achieved 0.40 precision within 500 m and 0.80 precision within 1000 m.
• Spatial Alignment: Across all 71 clusters, 19 were within 500 m and 46 were within 1000 m of a held-out feature, with a median distance of 744.3 m.
• Ablation Studies: Variants using only NDVI, only pasture, only summer features, or only OSM priors performed significantly worse. The "Rule only" variant (without graph smoothing) achieved similar top-5 precision but had a higher median distance (784.6 m vs. 744.3 m), indicating that graph smoothing improves spatial consistency and reduces median distance while preserving top-ranked precision.

Significance and Claims
The paper claims that seasonal representation learning and weak geographic priors can effectively reduce large satellite image collections into compact, ranked candidate sets suitable for human review. The results demonstrate that the framework captures meaningful farm-associated spatial structure under weak supervision.

The authors emphasize that this is a candidate discovery task, not a complete farm census. Consequently, recall is low (as the method prioritizes high-confidence clusters over exhaustive inventory), making precision-at-rank the more relevant metric. The evaluation relies on proxy metrics; because OSM features are incomplete and spatially noisy, proximity to held-out features is not treated as definitive ground truth. The paper concludes that weak supervision, self-supervised learning, and graph-based reasoning offer a viable path for discovering agricultural candidates from large-scale imagery where direct labels are unavailable.