Technical Summary: Annotation-Consistent Diffusion Augmentation for Data-Scarce Pest Instance Segmentation

1. Problem Statement

In Integrated Pest Management (IPM), sticky traps are a standard tool for monitoring insect populations. Automating this process via deep learning has progressed from image classification to instance segmentation, which provides critical pixel-level data on species composition, individual counts, and morphology. However, training robust instance segmentation models faces a significant bottleneck: the scarcity of polygon-annotated data for rare pest species.

While geometric transformations and copy-paste methods offer some augmentation, they fail to synthesize genuinely novel morphological variations (e.g., desiccation-induced deformation, postural distortion) inherent to sticky trap imagery. Furthermore, existing generative approaches, such as GANs or general-purpose diffusion models, struggle with two specific challenges in this domain:

1. Annotation Consistency: Instance segmentation requires strict spatial alignment between generated objects and their polygon masks. Even slight boundary shifts invalidate existing labels, necessitating costly relabeling.
2. Domain Adaptation: General models lack the visual priors required to generate realistic sticky trap specimens (e.g., insects flattened against adhesive surfaces) without domain-specific fine-tuning.

2. Methodology

The authors propose a domain-specific, diffusion-based data augmentation framework designed to generate synthetic training images that are annotation-consistent without requiring additional manual labeling. The pipeline consists of four integrated components:

2.1. Polygon-Guided Visual Prior Extraction

Unlike prior works using generic edge detectors (e.g., HED, Canny), this method directly utilizes ground-truth segmentation polygons as structural priors.

• Construction: Ground-truth polygon contours are rendered as white outlines on the image to serve as structural guidance.
• Spatial Replication: To increase diversity, these polygon contours are replicated at randomized positions and orientations within the image. A validity check ensures no overlap with existing instances and that they remain within image bounds.
• Differentiation: Crucially, only the structural priors (polygons) are replicated; the visual appearance of the pest is not copied. This distinguishes the method from conventional copy-paste augmentation.

2.2. Inpaint Mask Generation

An inpainting mask is derived from the original and replicated polygon annotations.

• The mask defines regions for regeneration (pest instances and a surrounding margin) and regions for preservation (the sticky trap background).
• Morphological dilation (7×7 kernel, 4 iterations) is applied to the polygon masks to create a protective margin, ensuring the diffusion model regenerates the pest and its immediate context while preserving the adhesive background texture.

2.3. Controllable Diffusion Model

The generation process utilizes a ControlNet-guided diffusion model (based on Qwen-Image-2512) to synthesize new pest appearances.

• Conditioning: The model receives the replicated visual prior (structural guidance) and the inpaint mask (spatial constraints).
• Class-Specific Adaptation: To bridge the domain gap between general diffusion models and sticky trap imagery, class-specific LoRA (Low-Rank Adaptation) adapters are fine-tuned for each target pest species. These adapters are injected into the encoder and middle layers of the network during generation.
• Prompt Engineering: Text prompts include species-specific morphological descriptions, sticky trap context (yellow adhesive surface), and image quality attributes, alongside negative prompts to suppress unrealistic outputs.

2.4. Self-Informed Post-Filtering

To ensure the quality of generated samples without external models, the authors introduce a self-informed filtering strategy based on a baseline segmentation model ($M_{base}$) trained on the original dataset.

• Feature Extraction: The backbone of $M_{base}$ extracts feature vectors from real pest instances to create class-wise reference prototypes.
• Cosine Similarity: For each generated instance, a feature vector is extracted and compared to the corresponding real-class prototype using cosine similarity.
• Filtering Logic:
  • Annotation-Level: Instances with similarity scores below a threshold ($\tau = 0.3$) are discarded, allowing the removal of failed generations within a single image without discarding the whole image.
  • Image-Level: Images are ranked by the average similarity of their remaining valid annotations, and a top fraction ($\gamma$) is retained for training.

3. Key Contributions

The paper outlines three primary contributions:

1. A Domain-Specific Framework: A unified pipeline for sticky trap instance segmentation that generates annotation-consistent synthetic data using polygon-guided ControlNet inpainting, spatial replication, and class-specific LoRA adaptation.
2. Self-Informed Filtering: A novel post-filtering mechanism that utilizes the feature space of a task-trained baseline model to automatically select high-quality synthetic samples, eliminating the need for external filtering models or manual inspection.
3. Cross-Architecture Validation: Comprehensive validation across five distinct segmentation architectures (YOLOv8, YOLOv11, YOLOv26, Mask R-CNN, and Mask2Former), demonstrating consistent performance gains for underrepresented species.

4. Experimental Results

The framework was evaluated on a dataset from AI Hub (South Korea) focusing on two rare species: Ricania sublimata and Metcalfa pruinosa. The data was split by greenhouse location to test generalization to unseen environments.

• Annotation Consistency: A model-based evaluation confirmed that 87.2% of reused polygon annotations were spatially consistent with the generated pest instances (Mean IoU > 0.72), validating the reliability of the generated labels.
• Performance Gains:
  • Across all five architectures, the proposed method increased the mean mAP50 from 77.1% to 87.5% and mAP50-95 from 63.2% to 65.5%.
  • The optimal image-level filtering ratio was found to be $\gamma = 0.7$, balancing data diversity and quality.
  • The method outperformed conventional augmentation strategies (CutMix, MixUp, Copy-Paste), which showed inconsistent or negative effects on rare species, particularly when data scarcity was severe.
• Architecture Sensitivity: While improvements were consistent across models, architectures with more expressive mask prediction modules (e.g., Mask R-CNN, Mask2Former) showed the largest gains in strict localization metrics (mAP50-95).

5. Significance and Claims

The authors claim that this work addresses a critical paradox in agricultural monitoring: rare pest species are often the most important for early warning yet suffer from the lowest model performance due to data scarcity.

• Bridging the Annotation Gap: The study demonstrates that polygon-guided diffusion augmentation can effectively expand training datasets for underrepresented species without the prohibitive cost of additional manual polygon annotation.
• Task-Aligned Quality Control: By using a "self-informed" filter based on the target task's feature space, the method ensures that synthetic data is not just visually plausible but functionally useful for the specific segmentation task.
• Practical Deployment: The location-based train-test split validates the method's potential for field deployment, where environmental variations (lighting, background contamination) are common.

The paper concludes that while the current validation is limited to specific sticky trap datasets and species, the proposed framework provides a methodological basis for broader application to domain-specific segmentation tasks where annotated data is inherently scarce.