💻 computer science

Positioning radiata pine branches requiring pruning by drone stereo vision

This paper presents a drone-mounted stereo vision system that combines deep learning-based branch segmentation and disparity estimation with a centroid-based triangulation algorithm to accurately localize radiata pine branches for autonomous pruning, demonstrating the feasibility of low-cost solutions for forestry automation.

Original authors: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a forest full of Radiata Pine trees. These trees are like the "gold" of New Zealand's economy, but to make them grow tall, straight, and knot-free (like a perfect pencil), workers have to trim off the lower branches.

The Problem:
Right now, humans have to climb these trees or use dangerous equipment to do this trimming. It's risky, expensive, and hard to find people willing to do it. It's like asking a human to perform delicate surgery while hanging from a rope in a storm.

The Solution:
The researchers in this paper built a robotic drone that can do the trimming itself. But for a drone to cut a branch, it needs to know two things:

Where is the branch? (Is it a branch or just a leaf?)
How far away is it? (If the drone gets too close, it crashes; too far, it misses.)

To solve this, they gave the drone "super-vision" using a stereo camera (two lenses, just like human eyes) and a brain made of Artificial Intelligence (AI).

Here is how they built the drone's brain, explained simply:

1. The "Eyes": Finding the Branch

First, the drone needs to spot the branch in a sea of green leaves. The researchers tested different AI "eyes" to see which one was best at drawing a mask around the branch.

The Contest: They pitted famous AI models (like YOLO and Mask R-CNN) against each other.
The Result: Think of YOLO (You Only Look Once) as a speedster sprinter. It saw the branches almost instantly and with incredible accuracy, even though the training data was small. It was the clear winner for spotting the target.

2. The "Depth Perception": Measuring the Distance

Once the drone sees the branch, it needs to know exactly how far away it is. This is the hardest part.

The Old Way (SGBM): Imagine trying to guess the distance to a tree by squinting and comparing two photos. The old method (called SGBM) is like a calculator that does math step-by-step. It works okay, but if the lighting is weird or the branch is blurry, it gets confused and the numbers are "noisy" (like static on an old TV).
The New Way (Deep Learning): The researchers tried using advanced AI models (like NeRF and RAFT). Think of these as experienced artists. Instead of just doing math, they "learned" what depth looks like by studying thousands of other images.
- The Analogy: If the old method is a robot trying to measure a room with a ruler, the new method is a human architect who can feel the space. The new AI produced much smoother, clearer depth maps, making the drone's "vision" much more reliable.

3. The "Brain": Putting It All Together

Now the drone has a list of branch locations and a map of distances. But the data is messy—some numbers are wrong outliers.

The Filter: The researchers created a clever algorithm (using something called MAD filtering) to clean up the data.
The Metaphor: Imagine you are trying to guess the average height of a group of people. If one person is a giant and another is a midget, they skew the average. This algorithm is like a smart bouncer who says, "Hey, you two are weird outliers, get out of the line," and then calculates the average of the real people left. This gives the drone a very precise distance to the branch.

The Big Picture

The researchers proved that you don't need expensive, heavy equipment (like giant lasers) to prune trees. You can use a cheap drone with a stereo camera and smart AI software to do the job.

Why does this matter?

Safety: No more humans hanging from trees in dangerous positions.
Efficiency: The drone can work faster and more precisely than a tired human.
Cost: It uses affordable cameras instead of million-dollar sensors.

What's Next?
The drone is currently being tested in a lab with fake branches. The next step is to take it outdoors into the real wind, rain, and sunlight to see if it can handle the chaos of a real forest. If it works, we might soon see fleets of robotic drones buzzing through forests, trimming trees like a swarm of helpful bees, keeping our timber industry safe and efficient.

1. Problem Statement

The forestry industry, particularly in New Zealand, relies heavily on Pinus radiata (radiata pine) for timber production. To produce high-quality, knot-free wood, trees require regular pruning. However, traditional manual pruning is hazardous, with fatality and injury rates significantly higher than the national average, leading to labor shortages.

While drones offer a potential solution for autonomous pruning, existing systems face two major limitations:

High Cost and Complexity: Current solutions often rely on expensive LiDAR sensors or multiple high-precision cameras.
Limited Capability: Many existing drone systems can only cut thick branches or require manual operation, failing to detect and prune thinner branches (as small as 10 mm) autonomously.

The core challenge addressed in this paper is developing a low-cost, autonomous drone system capable of detecting and precisely localizing thin radiata pine branches using only a stereo camera, eliminating the need for expensive LiDAR while maintaining the accuracy required for robotic arm actuation.

2. Methodology

The proposed system is a two-stage pipeline mounted on a drone equipped with a ZED Mini stereo camera: Branch Segmentation and Depth Estimation.

A. Data Collection and Segmentation

Dataset: A custom dataset of 71 stereo image pairs (122 images total) was captured indoors under varying lighting conditions using a ZED Mini camera (1920x1080 resolution).
Segmentation Models: The study compared several state-of-the-art object detection and segmentation architectures:
- Mask R-CNN: Variants with different backbones (ResNet-50/101, X101) and heads (C4, DC5, FPN).
- YOLO Series: YOLOv8 (n, s, m, l, x) and YOLOv9 (c, e) segmentation models.
Evaluation Metric: Performance was measured using mAP50-95 (Mean Average Precision across IoU thresholds from 50% to 95%).

B. Depth Estimation (Disparity to Depth)

The system calculates depth ( $z$ ) using the triangulation formula $z = W/d$ , where $W$ is the baseline $\times$ focal length and $d$ is the disparity. Two approaches were evaluated:

Traditional Method: Semi-Global Block Matching (SGBM) combined with Weighted Least Squares (WLS) filtering. This involves cost calculation (AD, SD, NCC), cost aggregation, and post-processing (left-right consistency, sub-pixel enhancement).
Deep Learning Methods: Several neural network architectures were evaluated and fine-tuned:
- Stereo Matching Networks: PSMNet, ACVNet, GWCNet, MobileStereoNet, and RAFT-Stereo.
- NeRF-Supervised Stereo: A novel approach using Neural Radiance Fields to synthesize training data, allowing the model to learn without ground-truth depth annotations.
- Monocular Baselines: MiDaS and Depth Anything were tested but deemed less accurate than binocular methods for this specific task.

C. Branch Localization Algorithm

To determine the precise distance from the drone to a branch:

Triangulation: The segmentation mask provides pixel coordinates surrounding the branch.
Centroid Calculation: A centroid-based algorithm groups predicted points into triangles to find centroids, which are expanded to include neighboring sample points.
Outlier Rejection: A Median Absolute Deviation (MAD) filter is applied to remove depth outliers.
Final Distance: The mean of the retained depth values within the $median \pm k \cdot MAD$ range is calculated to determine the final branch distance.

3. Key Contributions

Low-Cost Stereo Vision Pipeline: Demonstrated the feasibility of using a single stereo camera (ZED Mini) instead of LiDAR for precise branch positioning in forestry.
Comparative Analysis of Segmentation: Provided empirical evidence that YOLOv8 and YOLOv9 variants significantly outperform Mask R-CNN in both speed and accuracy (achieving >98% mAPbox) on the specific task of branch segmentation, even with a small dataset.
Depth Estimation Evaluation: Systematically compared traditional SGBM against modern deep learning stereo matching networks.
Robust Localization Algorithm: Proposed a centroid-based triangulation method with MAD outlier rejection to compute reliable branch distances from noisy disparity maps.
NeRF Integration: Explored the use of NeRF-supervised training to generate high-quality depth maps without requiring ground-truth depth annotations, addressing a common bottleneck in forestry data collection.

4. Results and Analysis

Segmentation Performance:
- YOLOv8s-seg achieved the highest mAPbox50-95 of 99.5% on the branch dataset.
- YOLOv8n-seg achieved 98.9%.
- Mask R-CNN models showed lower performance (e.g., Mask R-CNN R101-FPN at 84.09%), likely due to the small dataset size and the models' complexity relative to the simple task.
Depth Estimation Performance:
- Qualitative Results: At distances of 1–2 meters, deep learning-based disparity maps (specifically NeRF-Stereo and RAFT-Stereo) produced significantly more coherent and continuous depth estimates compared to SGBM.
- SGBM Limitations: The traditional SGBM method struggled with textureless regions and noise, resulting in fragmented depth maps.
- Generalization: Models pre-trained on urban datasets (KITTI) performed poorly on tree branches due to domain shift. Fine-tuning on specific data or using NeRF-synthesized data was necessary for accurate results.
Localization Accuracy: The centroid-based filtering successfully reduced noise, allowing for accurate distance estimation suitable for guiding a robotic arm.

5. Significance and Future Work

Economic and Safety Impact: This research paves the way for affordable, autonomous pruning drones that can operate safely, reducing the reliance on dangerous manual labor and lowering the barrier to entry for precision forestry.
Scalability: By proving that low-cost stereo vision can replace expensive LiDAR, the technology becomes more accessible to the broader forestry industry.
Future Directions:
- Data Expansion: Collecting data in diverse outdoor environments to improve model robustness.
- Ground Truth: Generating high-quality ground-truth depth maps (potentially using LiDAR as a supplement) to further train and validate deep learning models.
- Optimization: Exploring more efficient network architectures and resource management strategies to enable real-time processing on edge devices (drones).
- Generalization: Extending the method to different tree species and environmental conditions.

In conclusion, the paper successfully validates a computer vision pipeline that combines advanced segmentation (YOLO) and deep learning-based stereo vision to enable autonomous, precise pruning of radiata pine, offering a viable alternative to traditional, hazardous manual methods.