StreetTree: A Large-Scale Global Benchmark for Fine-Grained Tree Species Classification

1. Problem Statement

Fine-grained classification of street trees is essential for urban planning, ecosystem service assessment, and climate regulation. However, progress in this field is severely hindered by the lack of large-scale, geographically diverse, and publicly available benchmark datasets. Existing datasets suffer from several critical limitations:

• Limited Scope: Most are restricted to single countries or cities, lacking global diversity.
• Coarse Granularity: Many lack complete hierarchical taxonomic annotations (Order, Family, Genus, Species).
• Data Type: Existing datasets often rely on satellite imagery (top-down, coarse resolution) or lack individual tree-level geolocation.
• Temporal & Seasonal Gaps: Few datasets offer long-term records or seasonal variations for individual trees, which are crucial for capturing phenological changes.
• Real-World Complexity: Current models struggle with high inter-species visual similarity, long-tailed class distributions, and intra-class variations caused by seasons, lighting, and occlusions in street-level imagery.

2. Methodology: The StreetTree Dataset Construction

The authors constructed StreetTree, the world's first large-scale benchmark dedicated to fine-grained street tree classification. The construction process involved two main stages:

A. Data Collection and Integration

• Sources: Integrated data from academic datasets (e.g., TreeML, U.S. 5 Million), online resources, government agencies, and the Global Biodiversity Information Facility (GBIF).
• Taxonomic Alignment: Created a global tree species catalog (59,417 species) based on TreeGOER and GlobalTreeSearch. All raw labels were aligned to standard GBIF entries to resolve synonyms and ensure Darwin Core compliance.
• Filtering:
  • Restricted to observations from 2015–2025.
  • Filtered for "human observations" only to avoid automated/specimen noise.
  • Excluded records with geospatial inconsistencies or missing coordinates.
  • Retained only trees marked as "present" (excluding dead/absent).
• Urban Focus: Used the Global Human Settlement Layer (GHSL) to overlay and filter data, ensuring only trees within urbanized zones were included.

B. Street View Imagery Acquisition

• Querying: Used geographic coordinates to query the Google Street View (GSV) API, retrieving images from the nearest sampling point.
• Orientation: Computed azimuth angles to download images centered on the target tree.
• Two-Stage Filtering for Quality:
  1. Semantic Segmentation: Used Segformer on a random sample to calculate the pixel percentage of the "tree" class. Images with <20% tree pixels were discarded.
  2. Binary Classification: Trained a ResNet18-based MLP classifier on deep features to automate the filtering of the remaining ~12 million images, removing those with severe occlusion, poor lighting, or blur.

C. Dataset Statistics

• Scale: 12,235,152 street view images representing 3,365,485 individual trees.
• Coverage: 133 countries across five continents.
• Taxonomy: Covers 71 Orders, 241 Families, 1,747 Genera, and 8,363 Species.
• Attributes: Each record includes geolocation, observation date/season, and a four-level taxonomic hierarchy.
• Distribution: Exhibits a long-tailed distribution (highly skewed toward common species) and balanced seasonal coverage (Spring: 25.3%, Summer: 31.6%, Autumn: 28.0%, Winter: 16.1%).

3. Key Contributions

1. Global Scale & Diversity: The first dataset to cover 133 countries with over 12 million images, ensuring broad spatial representativeness.
2. Hierarchical Taxonomy: Provides complete four-level labels (Order–Family–Genus–Species), enabling research in hierarchical classification and representation learning.
3. Rich Metadata: Includes seasonal information and long-term temporal continuity (2015–2025), allowing for the study of phenological variations and growth dynamics.
4. Benchmarking Framework: Introduced StreetTree-18kEval, a rigorous evaluation set designed to prevent data leakage (instance-level separation) and test performance across frequent, common, and rare species categories.
5. Baseline Establishment: Established solid baselines using state-of-the-art vision models to reveal current limitations in handling real-world urban complexities.

4. Experimental Results

The authors evaluated five models (ViT, CLIP, SigLIP, BioCLIP) across different data scales (0.1% to 100% of the training set) on the StreetTree-18kEval benchmark.

• Model Performance:
  • Fine-tuned BioCLIP consistently outperformed all other models across all taxonomic levels and data scales. At 100% data scale, it achieved 45.45% Top-1 accuracy (Order), 39.76% (Family), 36.31% (Genus), and 29.45% (Species).
  • Fine-tuned SigLIP was the second-best performer.
  • Zero-shot CLIP and standard Fine-tuned ViT lagged significantly, highlighting the necessity of domain-specific pretraining and large-scale fine-tuning.
• Data Efficiency: Performance scales linearly with data volume. Even with 100% of the data, models were not saturated, suggesting further gains are possible with more data.
• Long-Tailed Challenge: A strict performance hierarchy was observed: Frequent > Common > Rare. For example, BioCLIP's species accuracy dropped from 30.26% (Frequent) to 20.60% (Rare), indicating a significant bottleneck in generalizing to rare species.
• Fine-Grained Difficulty: There is a large gap between Top-1 and Top-5 accuracy (e.g., 30.26% vs. 52.07% for Species), indicating that models can narrow down candidates but struggle with precise differentiation among visually similar species.

5. Significance and Future Impact

• Urban Governance: StreetTree provides a foundational resource for automating urban tree inventories, reducing the labor and time costs of manual surveys.
• Scientific Advancement: It bridges computer vision and urban ecology, enabling large-scale studies on biodiversity, carbon sequestration, and climate resilience.
• Methodological Insights: The results demonstrate that cross-modal pretraining (specifically with biological knowledge, as in BioCLIP) is superior for fine-grained tasks, but current models still face bottlenecks due to the intrinsic complexity of real-world street imagery (occlusion, lighting, seasonal variation).
• Future Directions: The authors suggest integrating Large Language Models (LLMs) to generate richer semantic descriptions for training and developing methods to mitigate long-tailed distribution issues.

In conclusion, StreetTree represents a paradigm shift in urban forestry research, moving from fragmented, small-scale datasets to a comprehensive, global, and temporally rich benchmark that drives the development of more robust, data-driven urban management systems.