Enhancing Tree Species Classification: Insights from YOLOv8 and Explainable AI Applied to TLS Point Cloud Projections

This paper presents a framework using YOLOv8 and Finer-CAM to achieve 96% accuracy in classifying seven European tree species from TLS 3D point clouds while demonstrating that the model's decisions are interpretable and primarily rely on crown features for most species, with stems playing a more significant role for others.

The Gist

Imagine you are a botanist trying to teach a computer to recognize different types of trees just by looking at them. In the past, computers were like students who could get a perfect score on a test but couldn't explain why they got the answer right. They might have just memorized the color of the sky in the photo or the specific angle of the sun, rather than actually learning what makes an Oak different from a Pine.

This paper is about teaching that computer student to show its work.

Here is a simple breakdown of what the researchers did, using some everyday analogies:

1. The Setup: The "3D Sculpture" vs. The "Flat Photo"

The researchers used a special tool called Terrestrial Laser Scanning (TLS). Think of this as a super-precise 3D scanner that creates a digital "cloud" of millions of tiny dots representing a real tree. It's like having a 3D hologram of a tree.

To make this data easier for a computer to understand, they turned these 3D holograms into 2D side-view photos (like taking a silhouette photo of a tree from the side). They used a powerful AI model called YOLOv8 (a type of "smart eye" that is very good at spotting things) to look at these photos and guess the tree species.

The Result: The AI was incredibly smart, getting 96% accuracy. It could tell a Silver Birch from a European Beech almost perfectly.

2. The Problem: The "Black Box"

Even though the AI was right, nobody knew how it was making those decisions. Was it looking at the shape of the leaves? The texture of the bark? Or was it cheating by looking at a weird pattern in the background?

In the world of AI, this is called a "Black Box." You put a tree in, and a species comes out, but the door is locked, and you can't see inside.

3. The Solution: The "Highlighter Pen" (Finer-CAM)

To open the box, the researchers used a tool called Finer-CAM. Imagine you have a magic highlighter pen. When you show the AI a picture of a tree and ask, "Is this a Pine?", the AI doesn't just say "Yes." It uses this magic pen to highlight the specific parts of the image that made it say "Yes."

But here is the clever part: The researchers didn't just ask, "What looks like a Pine?" They asked, "What looks like a Pine but NOT like a Spruce?" (Since Pines and Spruces look very similar). This allowed the AI to highlight the unique features that distinguish one tree from its "twin."

4. What Did They Find? (The "Detective Work")

By looking at thousands of these highlighted images, they discovered some fascinating secrets about how the AI thinks:

• The Crown is King: For most trees (like Birch, Beech, and Oak), the AI focused heavily on the top part of the tree (the crown). It was looking at the shape of the branches and how they spread out. It's like the AI realized, "Ah, the way the branches fan out is the tree's fingerprint!"
• The Trunk Matters Too: For some trees, like Ash, Pine, and Douglas-fir, the AI looked more at the trunk (stem).
  • The Pine/Douglas-fir Clue: These trees often keep their dead branches stuck to the trunk. The AI learned to spot these "dead branches on the trunk" as a dead giveaway.
  • The Ash Clue: The AI noticed that many Ash trees in their data set had bent trunks.
    • The Warning: The researchers realized this might be a "cheat." Bent trunks aren't unique to Ash trees in real life; they just happened to be bent in this specific dataset. The AI might be learning a shortcut: "If the trunk is bent, it's an Ash." If they show this AI a straight Ash tree in the real world, it might get confused. This is called shortcut learning.

5. The "Blurry Photo" Test

To see how much detail the AI actually needed, they ran an experiment where they made the photos blurrier and blurrier (like looking at a tree through fog).

• The Result: The AI could still identify the trees even when the photo was quite blurry, as long as the basic shape of the tree was visible.
• The Catch: When they removed the internal details (the branches and leaves) and only showed the solid outline (the "silhouette"), the AI's performance dropped. This proves that the AI does need to see the fine details of the branches to be truly accurate, not just the general shape.

Why Does This Matter?

This study is a huge step forward because it moves AI from being a "magic oracle" to a transparent partner.

• Trust: We can now trust the AI more because we know it's looking at the right things (branches and bark) and not just random background noise.
• Fixing Mistakes: By seeing that the AI was "cheating" by looking at bent trunks on Ash trees, the researchers know they need to fix their data. They can go out and scan more Ash trees with straight trunks to teach the AI the real lesson.
• Better Forest Management: In the future, this technology could help foresters use drones or lasers to count and identify trees in vast forests automatically, but now they will know why the computer is making those counts.

In a nutshell: The researchers taught a computer to identify trees, then used a "magic highlighter" to peek inside its brain. They found that the computer is mostly looking at branch shapes, but sometimes it gets tricked by weird quirks in the data. Now, they can fix those tricks to make the computer even smarter and more reliable.

Technical Summary

1. Problem Statement

Tree species classification using Terrestrial Laser Scanning (TLS) 3D point clouds has achieved high accuracy using Deep Learning (DL) models, particularly Convolutional Neural Networks (CNNs). However, these models operate as "black boxes," making their decision-making processes opaque.

• The Gap: While standard metrics (accuracy, F1-score) confirm performance, they do not reveal which structural features (e.g., stem, crown, branching patterns) the model uses to differentiate species.
• The Risk: Models may rely on "shortcut learning," learning artifacts (e.g., scanner-specific point patterns or dataset biases) rather than biologically relevant features, leading to poor generalization in real-world scenarios.
• The Challenge: Existing Explainable AI (XAI) methods like standard Class Activation Mapping (CAM) often highlight features common to similar classes rather than those that discriminate between them. Furthermore, manually interpreting thousands of saliency maps is prone to cognitive bias and impractical.

2. Methodology

The authors developed a framework combining YOLOv8 for classification and Finer-CAM (a variant of Grad-CAM) for interpretability, applied to 2D side-view projections of TLS data.

A. Data Preparation

• Dataset: A subset of the FOR-Species20k benchmark dataset.
• Selection: Filtered for 7 European tree species (Silver Birch, European Beech, European Ash, Scots Pine, Norway Spruce, Douglas-fir, English Oak) where TLS data was available from at least three different sources to ensure diversity.
• Volume: 2,445 individual trees (balanced to ~1,000 per class where possible).
• Preprocessing:
  • 2D Projection: TLS point clouds were converted into orthographic 2D side-view images (640x640 px) from four angles (0°, 45°, 90°, 135°).
  • Noise Mitigation: Points were randomly shifted by up to 0.5 cm to prevent models from learning scanner-specific regular point patterns.
  • Pixel Aggregation: Point counts were aggregated into raster cells and log-transformed to create grayscale images.

B. Model Architecture

• Classifier: YOLOv8 (specifically yolov8s-cls), chosen for its superior performance over YOLOv5 and ability to handle larger image resolutions.
• Training Strategy: Five independent models were trained using 5-fold cross-validation to ensure robustness.
• Input: Each tree was represented by four 2D side-view images; final predictions were based on the average logits of these four views.

C. Explainable AI Framework (Finer-CAM)

• Technique: Finer-CAM was used to generate saliency maps. Unlike standard CAM, Finer-CAM highlights regions that contribute to the target class ($c$) but suppresses features common to the most similar contrastive classes ($c'$).
• Contrastive Classes: For each target tree, the three species with the next highest output logits were used as contrastive classes.
• Faithfulness Testing: The authors employed a Least Relevant First (LeRF) pixel removal strategy. Pixels were iteratively flipped to white (background) starting from the lowest saliency. Model confidence was tracked to verify if removing low-saliency pixels had little effect until high-saliency regions were removed.

D. Heuristic Integration & Quantitative Analysis

To move beyond visual inspection, the authors developed a heuristic to link saliency maps to structural features:

1. Image Partitioning: Binary masks were generated to separate Tree, Stem, and Crown.
2. Sub-segmentation: The crown was further divided into Base, Middle, Top, and an Edge Buffer (capturing fine branches).
3. Quantification: The ratio of "salient pixels" (pixels above an Otsu threshold) within each segment was calculated. Statistical tests (Dunn's test with Holm-Bonferroni correction) were used to compare these ratios across species.

E. Perturbation Experiments

To test model reliance on specific features, the authors trained models on:

• Binary Shape: Trees reduced to solid shapes (removing internal structure).
• Reduced Visibility: Images downsampled to various resolutions (10px to 320px) to degrade the visibility of internal branching.

3. Key Contributions

1. Novel Framework: Introduced a systematic method to link Finer-CAM saliency maps to specific structural tree features (stem vs. crown vs. sub-segments), reducing human bias in interpretation.
2. Faithfulness Validation: Demonstrated that Finer-CAM is a faithful explanation method for TLS-based tree classification, as model confidence drops only when the most discriminative pixels are removed.
3. Feature Attribution: Quantified the specific contribution of stems and crowns to species differentiation, revealing species-specific reliance patterns.
4. Dataset Artifact Detection: Identified a potential "shortcut learning" artifact where the model relied on stem bends for European Ash classification, a feature likely specific to the dataset rather than the species biology.

4. Key Results

A. Classification Performance

• The five YOLOv8 models achieved a mean accuracy of 96% (SD = 0.24%) and a macro F1-score of 92.5% on the test set.
• Confusion matrices showed that models primarily confused conifers with other conifers and broadleafs with other broadleafs, indicating biologically plausible confusion patterns.

B. Saliency Map Analysis (Feature Importance)

• Overall: 68.6% of salient pixels were located in the crown segments, while stems contributed less on average.
• Species-Specific Patterns:
  • Crown-Dominant: Silver Birch, European Beech, Norway Spruce, and English Oak relied heavily on crown features (specifically branches and edge details).
  • Stem-Dominant: European Ash, Scots Pine, and Douglas-fir showed significantly higher reliance on stem features.
    • Ash: 64.4% of salient maps highlighted bends in the stem. The authors suspect this is a dataset artifact (shortcut learning) rather than a true biological trait, as stem bends are not unique to Ash.
    • Pine/Douglas-fir: Reliance on stems was linked to retained dead branches (a biological trait of these species).
• Crown Sub-segments:
  • Spruce: Uniquely relied heavily on the crown middle segment.
  • Birch: Relied on the crown base.
  • Beech: Relied on the crown top.

C. Perturbation Experiments

• Internal Structure: Models performed best when internal tree structure (branches) was visible. Performance dropped significantly when images were downsampled to resolutions below 0.5 meters (where individual branches become indistinguishable).
• Shape Only: When trained only on binary tree shapes (habitus), performance dropped to an F1-score of 78.8%, confirming that while shape helps, detailed internal structure is critical for high accuracy.
• Point Density: Models performed similarly on binary images (no density info) and grayscale images, suggesting point density itself is not a primary driver for classification; the structure is.

5. Significance and Implications

• Trust and Interpretability: This study moves beyond "black box" reporting, providing evidence that DL models are learning biologically relevant features (branching patterns, crown shape) rather than just noise.
• Data Quality Control: The identification of the "stem bend" artifact in Ash classification highlights the critical need for XAI to detect dataset biases before operational deployment.
• Efficiency: The finding that models can discriminate species using only ~20% of the most relevant pixels suggests future datasets could be optimized by focusing on high-contribution features, potentially reducing the need for full-tree scans.
• Future Directions: The authors call for standardized TLS data collection protocols to avoid artifacts and the development of XAI methods that can directly assess feature importance without relying on 2D projections.

In summary, the paper validates that YOLOv8 is highly effective for TLS-based tree classification and, through Finer-CAM, reveals that the model successfully learns species-specific structural traits, though it remains susceptible to dataset-specific shortcuts.