Assessing airborne laser scanning and aerial photogrammetry for deep learning-based stand delineation

This study demonstrates that deep learning-based forest stand delineation achieves comparable accuracy using temporally aligned digital photogrammetry-derived canopy height models and digital terrain models as it does with airborne laser scanning data, suggesting that large-scale, consistent datasets can be assembled without relying on the more complex and temporally misaligned ALS data.

The Gist

Imagine you are a forest manager trying to organize a massive, wild library. In this library, the "books" are trees, and the "shelves" are called forest stands. A forest stand is just a group of trees that look and act the same (same age, same species, same size) so they can be managed together.

Traditionally, humans have had to draw the lines between these shelves by hand, looking at aerial photos and guessing where one group of trees ends and another begins. It's slow, it's subjective (two experts might draw the lines differently), and it's hard to scale up.

This paper is about teaching a computer (using Deep Learning) to draw these lines automatically, and testing if it can do it just as well as a human expert.

Here is the breakdown of their experiment, explained with simple analogies:

1. The Problem: The "Mismatched Puzzle"

To teach the computer, you need to show it pictures of the forest and the "correct" answer (the lines drawn by an expert).

• The Old Way: They used two different types of data: Aerial Photos (taken by a plane with a camera) and Laser Scans (ALS, where a plane shoots lasers to measure height).
• The Glitch: The photos and the laser scans were often taken at different times (maybe a year apart). It's like trying to solve a puzzle where the picture on the box is from 2023, but the puzzle pieces are from 2024. Trees might have been cut down or grown, making the "correct" answer look wrong compared to the input data. This limits how much data you can use to train the AI.

2. The Solution: The "All-in-One" Camera

The researchers wanted to know: Can we use a newer technology called Digital Aerial Photogrammetry (DAP) instead of lasers?

• How DAP works: It takes thousands of overlapping photos and uses math to build a 3D model of the trees, just like the lasers do, but using only the photos.
• The Benefit: Since the 3D model and the color photos come from the exact same flight, they are perfectly synchronized in time. No more mismatched puzzles!
• The Fear: Critics worried that DAP might be "smoother" and miss small details (like gaps in the canopy) compared to the super-precise lasers.

3. The Experiment: The "Taste Test"

The researchers set up a "taste test" for their AI (a neural network called U-Net). They trained three different versions of the AI on data from six different towns in Norway:

1. Team Laser: Used the old-school Laser Scans + Photos.
2. Team Photo: Used the new DAP 3D models + Photos.
3. Team Terrain: Used DAP + Photos + a map of the ground elevation (DTM).

They asked: Which team draws the lines most like the human experts?

4. The Results: The "Twin" Effect

The results were surprising and very encouraging:

• All teams performed equally well. The AI using the new "Photo-only" 3D models (Team Photo) did just as good a job as the one using the expensive Laser scans.
• The "Twin" Effect: When they compared the AI's drawings to the human expert's drawings, there was some disagreement (which is normal; humans disagree too). BUT, when they compared the three different AIs to each other, they agreed almost perfectly.
  • Analogy: Imagine three different chefs making a cake. They all taste slightly different from the "perfect" recipe (the human expert), but they all taste almost identical to each other. This means the AI has learned a very consistent way of seeing the forest, even if it's not exactly how a human sees it.
• The Ground Map didn't help: Adding the "Terrain" map (DTM) didn't make the AI any better. In this specific area (which is relatively flat), the AI could already "feel" the hills and valleys just by looking at the trees and shadows.

5. The Catch: "Over-zealous" Cutting

The AI was very good at finding the forest, but it sometimes got a little too excited about drawing lines.

• It would draw tiny, jagged lines around small patches of trees that a human would ignore.
• Analogy: It's like a child coloring inside the lines but making the lines so wiggly and detailed that the picture looks messy. The researchers noted that these tiny errors can be fixed later with a simple "smoothing" tool, just like editing a photo.

6. Why This Matters

This study is a big deal for the future of forestry because:

1. It's Cheaper and Easier: You don't need expensive laser scanners. You just need a plane with a camera.
2. It's Faster: Because the data is perfectly aligned in time, you can train the AI on much larger areas without worrying about "mismatched" data.
3. It's Consistent: The AI is less tired and less subjective than a human. It draws the lines the same way every time.

In a nutshell: The researchers proved that a computer can learn to organize a forest just as well using "smart photos" as it does using expensive lasers. This opens the door to automating forest management on a massive scale, saving time and money while keeping our forests healthy.

Technical Summary

1. Problem Statement

Accurate forest stand delineation is critical for forest inventory, management, and financial analysis but remains a largely manual, subjective, and time-consuming process. While recent studies have demonstrated that deep learning (specifically U-Net architectures) can automate this task using Airborne Laser Scanning (ALS) and aerial imagery, a significant operational bottleneck exists: temporal misalignment.

• The Issue: ALS data and aerial imagery are often acquired at different times (e.g., years apart). This leads to inconsistencies (e.g., harvests or growth occurring in one dataset but not the other), reducing the available training data and limiting scalability.
• The Proposed Solution: Digital Aerial Photogrammetry (DAP) point clouds, derived directly from aerial imagery, offer temporally aligned structural data. However, DAP-derived Canopy Height Models (CHMs) are known to have smoother surfaces and reduced representation of canopy gaps compared to ALS.
• Research Gaps:
  1. Can DAP-derived CHMs effectively replace ALS-derived CHMs for training deep learning models without sacrificing accuracy?
  2. Does the inclusion of a Digital Terrain Model (DTM), which aids manual interpreters in steep terrain, improve automated delineation performance?

2. Methodology

Study Area and Data

• Location: Follo district, southeastern Norway (6 municipalities).
• Data Acquisition (2021): All data was acquired in the same year to ensure temporal consistency.
  • Reference Data: Expert-delineated forest stand maps (5 classes: Non-forested, Bare/Regeneration, Young Thinning, Old Thinning, Mature) created by human interpreters using stereoscopic imagery and ALS.
  • Input Data:
    1. Multispectral Aerial Imagery: RGB + NIR (0.1m resolution).
    2. ALS Point Cloud: 5 pulses/m² density.
    3. DAP Point Cloud: 25 points/m² density, derived from the same imagery.
    4. DTM: Derived from ALS ground points.

Data Pre-processing

• Rasterization: CHMs and DTMs were generated at 1m resolution.
• Normalization: Heights were scaled to [0, 1].
• Input Stacks: Three distinct input configurations were created for the model:
  1. RGBI-ALS: Imagery + ALS-derived CHM (5 channels).
  2. RGBI-DAP: Imagery + DAP-derived CHM (5 channels).
  3. RGBI-DAP-DTM: Imagery + DAP-derived CHM + DTM (6 channels).

Model Architecture and Training

• Architecture: U-Net (Semantic Segmentation).
• Loss Function: Focal Tversky Loss (to handle class imbalance and focus on hard examples).
• Validation Strategy: Municipality-level 6-fold cross-validation. This ensures spatial independence by withholding entire municipalities as test sets, preventing data leakage from spatial autocorrelation.
• Hyperparameter Tuning: Conducted using the Optuna framework with a Tree-structured Parzen Estimator (TPE) sampler. Parameters included learning rate, batch size, filters, kernel size, and loss function weights.
• Metrics: Macro-averaged Matthew's Correlation Coefficient (mMCC), Overall Accuracy (OA), Intersection over Union (mIOU), F1-score (mF1), User's Accuracy (mUA), and Producer's Accuracy (mPA).

3. Key Contributions

1. Validation of DAP as an ALS Substitute: The study provides empirical evidence that DAP-derived CHMs can effectively replace ALS-derived CHMs in deep learning workflows for stand delineation, despite DAP's tendency to smooth canopy surfaces.
2. Temporal Alignment Feasibility: Demonstrates that large-scale, temporally aligned datasets (using DAP) can be assembled to train robust deep learning models, solving a major scalability issue in operational forestry.
3. Assessment of DTM Utility: First published study to test the inclusion of DTM in automated stand delineation workflows, finding it provided no significant performance gain in this specific context.
4. Quantification of Subjectivity: Revealed that inter-model agreement is significantly higher than agreement with human reference data, highlighting that "errors" in automated models often reflect the inherent subjectivity of human stand delineation rather than model failure.

4. Results

Performance Metrics

• Overall Accuracy (OA): All three models achieved high OA values between 0.90 and 0.91.
• Macro-Averaged Metrics:
  • mMCC: Ranged from 0.60 to 0.62 across all models.
  • mIOU: Ranged from 0.52 to 0.57.
  • mF1: Ranged from 0.66 to 0.67.
• Comparison: There were no substantial differences in performance between the RGBI-ALS, RGBI-DAP, and RGBI-DAP-DTM models. The RGBI-DAP model performed slightly better on average, but the differences were negligible.

Inter-Model vs. Model-Reference Agreement

• Crucial Finding: The agreement between the different models (e.g., RGBI-ALS vs. RGBI-DAP) was substantially higher (mMCC ≈ 0.84) than the agreement between any model and the human reference data (mMCC ≈ 0.60).
• Implication: The models learned a consistent representation of forest structure. The discrepancies with the reference map suggest that human delineation is highly subjective, and the models are not necessarily "wrong" but rather consistent in their own logic.

Specific Observations

• DTM Impact: Adding the DTM did not improve performance. In fact, models with DTM tended to overfit and stopped training early (due to early stopping criteria), suggesting the terrain information was either redundant (captured implicitly by imagery/CHM in flat terrain) or added unnecessary complexity.
• Deciduous Forests: DAP CHMs showed reduced reliability in deciduous stands (due to leaf-off conditions affecting photogrammetric matching), yet the deep learning model compensated using spectral and contextual cues, maintaining overall performance.
• Over-segmentation: All models exhibited a tendency to over-segment, creating small patches (<0.2 ha) that fall below the minimum mapping unit for operational forestry.

5. Significance and Conclusion

Operational Scalability:
The study confirms that forest management agencies can rely on DAP-derived CHMs combined with aerial imagery for deep learning-based stand delineation. This removes the dependency on expensive, temporally misaligned ALS data, allowing for the creation of large, consistent training datasets using existing aerial photography projects.

Robustness of Deep Learning:
The U-Net framework proved resilient to variations in input data quality (smooth DAP CHMs vs. detailed ALS CHMs). The model successfully utilized complementary spectral information to compensate for structural limitations in the DAP data.

Limitations and Future Directions:

• Terrain: The lack of DTM benefit may be specific to the relatively flat study area; DTM might be crucial in steep, complex topography.
• Post-processing: The raw output contains fragmented boundaries. Operational deployment will require post-processing (smoothing, merging small polygons).
• Annotation Costs: The current approach still relies on extensive manual annotation. Future work should focus on weakly supervised or semi-supervised learning to reduce the cost of data preparation.

In summary, this paper validates a scalable, cost-effective workflow for automated forest stand delineation that leverages temporally aligned photogrammetric data, offering a viable path toward operationalizing deep learning in forestry.