Learning to Synergize Semantic and Geometric Priors for Limited-Data Wheat Disease Segmentation

1. Problem Statement

Wheat disease segmentation is critical for precision agriculture but faces two major challenges in real-world field environments:

• Significant Intra-class Temporal Variations: The visual appearance of the same disease changes drastically across different growth stages and infection levels.
• Data Scarcity: Collecting a representative dataset covering all disease stages is labor-intensive, leading to limited annotated data.
• Limitations of Existing Models:
  • Foundation Models (e.g., SAM): While the Segment Anything Model (SAM) offers powerful geometric priors for boundary localization, it lacks disease-specific semantic awareness, leading to poor zero-shot performance on complex, varying disease patterns.
  • Traditional Supervised Models: They struggle with generalization when training data is scarce and cannot handle the extreme diversity of temporal appearances.
  • Prompting Issues: Manual dense prompting for SAM is impractical, while sparse prompts fail to capture irregular lesion boundaries.

The core challenge is how to achieve robust segmentation under limited supervision by automatically leveraging the semantic understanding of one foundation model to guide the geometric precision of another.

2. Methodology: SGPer Framework

The authors propose SGPer (Semantic-Geometric Prior Synergization), a framework that treats wheat disease segmentation as a coupled task of disease-specific semantic perception and boundary localization. It synergizes DINOv2 (for semantics) and SAM (for geometry).

The framework consists of three key components:

A. Disease Perception Adapter (DPA)

To bridge the domain gap between generic pretrained models and specific wheat disease imagery, a lightweight DPA is inserted into both DINOv2 and SAM.

• Mechanism: It includes a distribution calibration step (Layer Normalization with learnable scaling) and a bottleneck module with parallel depth-wise convolutions (kernel sizes 1, 3, 5) to capture multi-granularity disease features.
• Role:
  • In DINOv2: It recalibrates features to enhance sensitivity to disease semantics, making disease regions more distinguishable.
  • In SAM: It steers the segmentation process toward disease-relevant geometric structures, improving focus on lesion extents and boundaries.

B. Semantic-aware Prompt Generator (SPG)

This module converts the refined semantic features from DINOv2 into dense, category-specific point prompts for SAM, eliminating the need for manual prompting.

• Process: The adapted DINOv2 encoder generates dense visual features. A lightweight generator transforms these into pattern maps ($T$) where each channel corresponds to a specific disease category.
• Prompt Extraction: These maps are binarized to extract spatial coordinates, creating a set of dense positive prompts ($P$) for each disease category.
• Supervision: An auxiliary loss ($L_p$) is used to ensure the generated pattern maps are semantically consistent with ground-truth masks, guaranteeing that prompts fall within actual disease regions.

C. Geometric Feedback Segmentor (GFS)

To handle the computational cost and noise of dense prompts, GFS iteratively decodes and prunes prompts using a dual-validation strategy.

• Iterative Decoding: Prompts are sampled in batches and fed into the SAM decoder to generate candidate masks and quality scores ($S_{iou}$).
• Dual-Validation: Each candidate mask is evaluated by:
  1. Geometric Score: SAM's internal confidence ($S_{iou}$).
  2. Semantic Score: A lightweight network projects the mask onto DINOv2 features to check semantic consistency with the target disease category ($S_{sem}$).
• Pruning: The final confidence score is the product of geometric and semantic scores ($S_{final} = S_{iou} \times S_{sem}$). High-confidence masks are kept, and prompts falling within these accepted masks are removed from the pool to prevent redundancy.
• Output: The process repeats until all prompts are processed, aggregating the highest-confidence masks for the final prediction.

3. Key Contributions

1. Novel Formulation: First to frame limited-data wheat disease segmentation as a tightly coupled task of semantic perception and geometric localization.
2. SGPer Framework: A synergistic pipeline that uses DINOv2 to provide robust, category-aware semantic priors (resilient to appearance shifts) to automatically generate dense prompts for SAM, which then exploits geometric priors for precise boundary localization.
3. Disease-Sensitive Adapters: Introduction of the DPA, which effectively aligns generic foundation model representations with specific wheat disease characteristics without catastrophic forgetting.
4. Automated Prompting & Pruning: A mechanism to convert semantic features into dense prompts and dynamically filter them based on geometric-semantic consistency, ensuring high accuracy without manual intervention.

4. Experimental Results

The framework was evaluated on three datasets: EFDv2 (Wheat Disease), OSD (Wheat Organ Segmentation), and GWFSS (Global Wheat Full Semantic Organ Segmentation).

• Performance on EFDv2 (Disease Segmentation):
  • SGPer achieved 72.5% mIoU and 82.8% F1-score, outperforming the strongest baseline (Segformer) by 9.8% mIoU.
  • Critical Improvement: On the "Powdery Mildew" class (limited data, high variation), SGPer achieved 47.4% mIoU, nearly doubling the performance of previous SOTA methods (which hovered around 20%).
• Performance on Organ Segmentation (OSD & GWFSS):
  • Achieved state-of-the-art results on organ segmentation (Ears, Stems, Leaves), demonstrating strong generalizability beyond just disease detection.
  • Notably improved segmentation of complex structures like "Stems" (40.3% mIoU on GWFSS vs. 21.8% for SegNeXt).
• Ablation Studies:
  • Removing DPA caused significant drops in performance, confirming the necessity of domain adaptation.
  • Removing the GFS pruning mechanism reduced performance, highlighting the importance of filtering redundant prompts.
  • Using DINOv2 features for mask quality assessment improved F1 by 6.6% over using SAM features alone.

5. Significance

• Data Efficiency: SGPer demonstrates that high-precision segmentation is achievable with limited annotated data by leveraging the complementary strengths of foundation models.
• Robustness to Temporal Variation: By relying on DINOv2's robust semantic priors, the model remains invariant to the drastic appearance changes of wheat diseases across growth stages, a common failure point for traditional models.
• Automation: The framework eliminates the need for labor-intensive manual prompting or dense annotation, making it highly practical for large-scale agricultural monitoring.
• Generalizability: The success on both disease and organ segmentation benchmarks suggests this semantic-geometric synergy paradigm is applicable to other fine-grained visual tasks with limited data.