EcoMorph: Universal morphological trait quantification from natural language prompts for ecological research

Technical Summary: EcoMorph – Universal Morphological Trait Quantification

Problem Statement
Accurate quantification of morphological traits, such as floral area and insect body size, is fundamental to ecological research, influencing studies on pollinator–plant interactions, habitat quality, and biodiversity monitoring. However, existing methods face significant limitations. Traditional manual measurement is time- and resource-intensive. While automated image analysis and deep learning (DL) approaches have emerged, they typically require task-specific training on extensive labeled datasets for each new application domain. This requirement hinders generalization across diverse taxa and complex field conditions. Furthermore, many high-accuracy computer vision workflows rely on specialized, costly imaging equipment (e.g., controlled greenhouse scanners), limiting their applicability in natural field settings where lighting and backgrounds are variable.

Methodology
The authors present EcoMorph, a modular morphological measurement system designed to quantify traits across diverse ecological contexts without taxon-specific retraining. The system leverages the Segment Anything Model 3 (SAM3), a visual foundation model capable of segmenting arbitrary objects based on natural language prompts.

The EcoMorph pipeline consists of three sequential modules:

1. Object Segmentation Module: Utilizes SAM3 to identify and delineate target structures. Users provide text prompts (ranging from single nouns like "flower" or "insect" to descriptive phrases like "pink flowers" or "brown square cardboard") to guide segmentation. This eliminates the need for domain-specific training data.
2. Morphology Extraction Module: Computes pixel-based measurements from the resulting segmentation masks. It calculates total area (summing pixels of target objects), linear dimensions (extracting width/height from bounding boxes or midpoints), and object counts (discrete instances detected per prompt).
3. Measurement Scaling Module: Converts digital pixel measurements into physical units (e.g., cm², mm) using two calibration approaches:
   • Reference-based scaling: Uses a physical object of known dimensions (e.g., a coin or cardboard square) placed within the image plane.
   • Distance-based scaling: Calculates scaling factors based on camera sensor specifications, focal length, and subject distance, useful when physical references are impractical (e.g., drone imagery).

Key Contributions

• Prompt-Based Universality: Unlike previous tools requiring retraining for specific taxa, EcoMorph uses a single underlying model (SAM3) with natural language prompts to segment diverse biological structures (flowers, insects, leaves) across different scales.
• Modular Architecture: The system decouples object detection, feature extraction, and unit conversion, allowing independent optimization and extension for specific research needs.
• Dual-Scale Validation: The framework was validated across two distinct ecological scales: intermediate-scale (centimeters) floral area and fine-scale (millimeters) insect morphometrics.
• Accessibility: The system is designed to work with consumer-grade imaging hardware (e.g., OAK-1 camera) and is accessible via a web interface, lowering the barrier to entry for high-throughput trait quantification.

Results
The authors validated EcoMorph against ground-truth manual measurements in three specific tests:

1. Floral Area (Simple Background): Under controlled, simple-background conditions, EcoMorph-derived floral areas showed strong agreement with manual ImageJ measurements ($R^2 = 0.935$, $n = 74$).
2. Floral Area (Complex Background): In outdoor garden settings with variable lighting and mixed vegetation, the system maintained high accuracy ($R^2 = 0.928$, $n = 58$) for 95% of images. Valid predictions were achieved despite complex backgrounds, though segmentation failures occurred in images with highly filamentous floral structures.
3. Insect Morphometrics: EcoMorph-derived body area was strongly correlated with hand-measured intertegular distance (ITD) across three bee species ranging from small (Bombus impatiens) to large (Xylocopa virginica) ($r = 0.810$, $n = 349$).
4. Object Counting: For insect counts in a box (7–80 specimens per image), EcoMorph matched manual counts with near-perfect accuracy ($R^2 = 0.9997$, $n = 12$), with a root-mean-square error (RMSE) of 0.50.

Significance and Claims
The paper claims that EcoMorph represents a significant advance for ecological applications requiring measurement across diverse taxa, imaging contexts, and accessibility constraints. By combining prompt-based segmentation with modular measurement, the tool enables high-throughput quantification of area, size, and abundance from heterogeneous image sources without the need for taxon-specific training.

The authors position EcoMorph as a solution that bridges the gap between the high accuracy of controlled-environment deep learning models and the flexibility required for field ecology. They assert that the system supports a broad range of applications, including pollinator and plant trait research, biodiversity and abundance monitoring, and allometric biomass estimation. While acknowledging limitations regarding highly filamentous structures and the computational demands of SAM3 inference compared to lighter models, the authors argue that the elimination of training data requirements and the ability to generalize across complex environments justify the approach for most research applications.