Technical Summary: NEEDL-Bench Dataset for Swiss Needle Cast and Stomata Detection

Problem Statement

Swiss Needle Cast (SNC), caused by the fungus Nothophaeocryptopus gaeumannii, is a significant threat to Douglas-fir (Pseudotsuga menziesii), a keystone species with major ecological and economic value. The disease forms sexual reproductive structures (pseudothecia) that emerge through needle stomata, occluding gas exchange pores and leading to chlorosis and premature needle loss. Current assessment methods rely on manual visual inspection to count stomata occlusion or score needle loss, a process that is time-consuming, resource-intensive, error-prone, and lacks scalability. While computer vision offers a potential solution for standardizing and scaling these measurements, no existing dataset supports the automatic detection of these specific microscopic structures. Furthermore, the task presents unique challenges: the objects of interest (stomata and pseudothecia) are small, often suffer from poor contrast and blur due to limited depth of field in microscopy, and are frequently occluded by other structures or foreign objects.

Methodology

The authors present NEEDL-Bench, a new benchmark dataset and evaluation framework designed to address these gaps.

Dataset Construction

• Data Collection: The dataset comprises 3,250 annotated image tiles derived from 1,082 Douglas-fir needles collected from multiple sites on Vancouver Island, British Columbia.
• Imaging Modalities: To reflect real-world variability, images were captured using two distinct workflows:
  1. Benchtop: High-resolution, controlled lighting conditions using a standard microscope.
  2. Handheld: Portable, in-situ compatible setup using a DinoLite microscope with 3D-printed jigs and multi-focus image stacking to ensure sharpness.
• Annotations: The dataset provides dual annotation types for both stomata and pseudothecia:
  • Keypoints: Centroid locations for counting.
  • Bounding Boxes: Minimum area bounding boxes for localization.
  • Annotations were generated manually by a trained annotator and converted semi-automatically to bounding boxes using the SAM2 segmentation model.
• Preprocessing: Raw images were scaled to a consistent spatial resolution of 360 pixels/mm and split into 256×256 tiles to increase the relative size of the small objects of interest.

Evaluation Strategy

The authors propose two distinct evaluation splits to test model robustness against different data distributions:

1. Random Sampling: Approximates the natural distribution of visual features and expected real-world infection rates.
2. Maximum Diversity Sampling: A sequential sampling method (based on farthest-point sampling in feature space) designed to maximize structural diversity, ensuring models are tested on rare edge cases, such as fully occluded needles or unique textures.

Benchmark Models

The study evaluates four baseline architectures to establish performance baselines:

• YOLOv11 (Nano): Evaluated in both standard bounding-box detection (Detect) and pose estimation (Pose) configurations.
• RF-DETR (Base): A transformer-based detector utilizing a Vision Transformer backbone (DINOv2) for global context.
• FCN-ResNet50: A Fully Convolutional Network adapted for keypoint detection via heatmaps.

Key Results

The evaluation of baseline models on the NEEDL-Bench dataset yielded the following findings:

• Performance Ceiling: The best-performing model, RF-DETR, achieved a maximum average F1 score of 0.8479 on the full dataset. This indicates significant room for improvement, as the task remains challenging even for state-of-the-art models.
• Scaling Laws vs. Inductive Bias: A critical finding is that increasing model capacity does not yield commensurate performance gains.
  • The lightweight YOLOv11n (2.6M parameters) achieved an F1 of 0.8470.
  • The larger FCN-ResNet (35.3M parameters) achieved 0.8436.
  • The RF-DETR (29.0M parameters) achieved 0.8479.
  • Comparisons between YOLO variants of different sizes (Nano, Large, X) confirmed that larger models did not outperform the Nano variant. This suggests that performance on this microscopy task is limited by a lack of domain-specific inductive biases rather than representational capacity.
• Task Formulation (Pose vs. Box):
  • On Random splits, bounding box detection models generally outperformed or matched pose estimation models.
  • On Maximum Diversity splits (containing rare, difficult cases), pose estimation models (YOLOv11-Pose and FCN-ResNet) demonstrated superior robustness, outperforming bounding box detectors. The authors attribute this to the difficulty of defining spatial boundaries for blurry or occluded objects, whereas localizing a centroid remains more stable.
• Acquisition Modality: Contrary to expectations, the Benchtop images proved more challenging than the Handheld images (average F1 ~0.82 vs. ~0.84). This was attributed to less controlled lighting, single-image blur, and reduced needle quality from freeze-thaw storage cycles, whereas the Handheld setup utilized multi-focus stacking to ensure sharpness.
• Class Difficulty: Detection of Pseudothecia consistently lagged behind Stomata detection across all models, likely due to partial occlusions and structural displacement.

Significance and Claims

The paper positions NEEDL-Bench as a necessary resource to enable computer vision research for forestry pathology. Its primary contributions and claims are:

1. Dataset Availability: It provides the first large-scale, annotated dataset for SNC severity at the needle level, supporting both keypoint counting and bounding-box localization.
2. Evaluation Rigor: By introducing a "Maximum Diversity" split, the benchmark forces models to generalize beyond average cases to rare, structurally diverse edge cases, which is critical for real-world deployment.
3. Insight on Model Scaling: The authors claim that for this specific microscopy task, "scaling laws" (simply making models larger) are not the path to improvement. Instead, future gains will likely come from domain-specific inductive biases tailored to handle low contrast, blur, and ambiguous occlusions.
4. Efficiency: The results suggest that lightweight CNN architectures (like YOLOv11 Nano) are highly efficient and effective for this task, offering near-optimal performance with significantly lower computational latency compared to Transformer-based models.

The authors conclude that while current baselines show promise, the problem remains open, particularly regarding the detection of challenging pseudothecia, and calls for future work focused on domain-specific preprocessing and architectural design rather than general model scaling.