💻 computer science

Real-Time Branch-to-Tool Distance Estimation for Autonomous UAV Pruning: Benchmarking Five DEFOM-Stereo Variants from Simulation to Jetson Deployment

This paper evaluates five variants of the DEFOM-Stereo foundation model for real-time branch-to-tool distance estimation in autonomous UAV pruning, identifying the newly introduced DEFOM-PrunePlus as the optimal solution that balances sufficient accuracy for safe actuation with the inference speed required for onboard deployment on NVIDIA Jetson hardware.

Original authors: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

Published 2026-03-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a drone to act like a master gardener. The drone's job is to fly up to a tree, find a thin, fragile branch, and snip it off with a pair of robotic scissors.

The problem? The drone can't "feel" the branch. It has to rely entirely on its eyes (cameras) to guess exactly how far away the branch is. If it guesses wrong, it might crash into the tree or cut the branch too early, damaging the tree.

This paper is about building the "brain" for that drone. The researchers tested five different versions of a computer vision system to see which one is smart enough to see the branches clearly but fast enough to react in real-time.

Here is the breakdown of their journey, explained simply:

1. The Training Ground: A Digital Sandbox

You can't easily teach a drone by letting it crash into real trees thousands of times. So, the researchers built a virtual world using a video game engine (Unreal Engine 5).

The Analogy: Think of this like a flight simulator for pilots. They created 115 different virtual trees and took 5,520 "photos" of them from different angles.
The Superpower: In the real world, it's hard to know the exact distance to a tiny twig. But in the video game, the computer knows the exact distance to every single pixel. This gave the AI perfect "answer keys" to learn from, something impossible to get in the real world.

2. The Five Contenders: The "Brain" Variants

The researchers took a powerful AI model (called DEFOM-Stereo) and created five different versions of it, ranging from "Super Brain" to "Pocket Brain." They wanted to find the sweet spot between smarts and speed.

The "Super Brain" (ViT-S & ViT-L): These are the most intelligent models. They can see the thinnest branches with incredible precision.
- The Catch: They are like a Formula 1 car with a heavy engine. They are so smart that they take too long to think. On the drone's small computer, they only "blink" (update their view) about 2 times a second. That's too slow for a drone flying fast; by the time it thinks, it might have already crashed.
The "Pocket Brains" (PruneStereo & PruneNano): These are tiny, lightweight models. They are incredibly fast, updating 7–8 times a second.
- The Catch: They are like a toddler with a calculator. They are fast, but they make big mistakes. They might think a branch is 1 meter away when it's actually 2 meters. If the drone acts on this, it will crash.
The "Goldilocks" Model (PrunePlus): This is the new hero of the story. The researchers built a middle-ground version. It's not as perfect as the Super Brain, but it's much faster.
- The Result: It updates about 3.3 times a second. It's not perfect (it might be off by a few inches), but it's fast enough to keep the drone safe and steady while it approaches the branch.

3. The Real-World Test: From Video Game to Reality

After training in the video game, they put the models on a real drone computer (an NVIDIA Jetson).

The Test: They showed the AI photos of real trees it had never seen before.
The Result: The "Goldilocks" model (PrunePlus) was able to look at a real photo and correctly figure out which parts were branches and which were sky, even though it had only ever seen video game trees. This proves the training worked!

4. The Final Verdict: Why Speed Matters More Than Perfection

The paper concludes that for a drone pruning trees, being "good enough" and fast is better than being perfect and slow.

The Analogy: Imagine you are driving a car in fog.
- If you have a GPS that tells you exactly where you are, but it updates only once every 10 seconds, you might crash because the car moved while you were waiting for the next update.
- If you have a GPS that updates every second but is slightly off (maybe off by 5 meters), you can still drive safely because you get constant updates to adjust your steering.

The Winner: The DEFOM-PrunePlus model.
It strikes the perfect balance. It's fast enough to give the drone constant updates as it flies toward the branch, allowing the drone to slow down and cut safely. The "Super Brain" is too slow to be useful right now, and the "Pocket Brains" are too inaccurate to be safe.

Summary

The researchers built a video game to teach a drone how to see tree branches. They tested five different "brains" and found that the middle-ground brain is the best for the job. It's fast enough to keep the drone safe and smart enough to find the branch, proving that sometimes, a slightly imperfect but quick solution is better than a perfect but slow one.

1. Problem Statement

Autonomous tree pruning using Unmanned Aerial Vehicles (UAVs) is a safety-critical task requiring the onboard perception system to estimate the metric distance between a cutting tool and thin tree branches in real-time.

Challenges:
- Geometry: Branches are slender, highly self-occluded, and often occupy only a few pixels, making them difficult for standard stereo matching.
- Environment: Scenes involve repetitive bark textures, transparent leaves, and strong depth discontinuities against sky backgrounds.
- Constraints: The system must operate on embedded hardware (NVIDIA Jetson) with strict latency requirements to support closed-loop control (approach, align, and actuate without collision).
- Data Scarcity: Obtaining dense, ground-truth depth labels for thin branches in real-world forestry is impractical.

2. Methodology

A. Synthetic Data Generation (UE5)

To overcome the lack of real-world depth labels, the authors created a task-specific synthetic dataset using Unreal Engine 5 (UE5):

Setup: A simulated ZED Mini stereo camera rig.
Content: 115 unique tree instances captured from three viewpoints (upward, downward, parallel) at a fixed 2-meter standoff distance.
Scale: 5,520 stereo pairs with dense, exact EXR depth maps, providing perfect supervision for thin-branch geometry.
Split: 80% training, 10% validation, 10% testing.

B. Model Architecture & Variants

The study benchmarks five variants of DEFOM-Stereo (a foundation-model-based stereo matcher combining DINOv2 priors with stereo correlation). The variants represent a spectrum from full-capacity to highly compressed:

DEFOM-Stereo ViT-S & ViT-L (Full Capacity):
- Use full-resolution DINOv2 backbones (ViT-S: ~21M params; ViT-L: ~304M params).
- Feature Dual DPT Decoders with ~66 convolutional layers (RefineNet fusion) and 3-level ConvGRU refinement.
- Goal: Establish the accuracy ceiling.
DEFOM-PrunePlus (Balanced):
- Strategy: "Reduce quantity, preserve quality."
- Retains full-resolution DINOv2 and full-strength GRU.
- Replaces the dual DPT decoders with a single 14-layer EnhancedDPTDecoder.
- Reduces iteration counts but maintains high representational capacity.
DEFOM-PruneStereo (Moderate Compression):
- Strategy: "Reduce, but do not replace."
- Reduces DINOv2 input resolution (0.75x), uses a 6-layer FastDPTDecoder, narrows encoder channels, and reduces GRU levels.
DEFOM-PruneNano (Aggressive Compression):
- Strategy: "Replace with efficient equivalents."
- Uses 0.5x DINOv2 resolution, Ghost modules (for ~50% FLOP reduction), a 3-layer TurboDecoder, SharedGhostEncoders, and Depthwise-Separable GRUs (DS-GRU).

C. Deployment & Evaluation

Hardware: NVIDIA Jetson Orin Super (16 GB) mounted on a UAV.
Inference: Measured in native PyTorch (FP32) and optimized TensorRT (FP16).
Metrics: End-Point Error (EPE), D1-all, $\delta_1$ (accuracy within 1.25x), Depth MAE (cm), and Latency (FPS).
Validation: Zero-shot inference on real tree-branch photographs to test Sim-to-Real transfer.

3. Key Contributions

Task-Specific Synthetic Dataset: A high-fidelity UE5 dataset with 5,520 stereo pairs and exact depth supervision for thin branches, addressing the annotation bottleneck in forestry robotics.
Five-Model Benchmark: A comprehensive comparison of foundation-model stereo variants, revealing a three-tier accuracy hierarchy and a "steep accuracy cliff" when compressing the complex DPT decoder.
DEFOM-PrunePlus: Introduction of a novel balanced variant that preserves critical architectural components (full-resolution DINOv2, strong GRU) while compressing the decoder, achieving the best deployable trade-off.
Real-World Deployment Analysis: Empirical evidence that accuracy-speed trade-offs are non-linear; models with high FPS but poor depth accuracy (PruneStereo/Nano) are unsafe for UAV actuation, while the most accurate model (ViT-S) is too slow for real-time control.
Sim-to-Real Validation: Demonstration that models trained only on synthetic data can preserve branch continuity and depth ordering on real photographs, validating the synthetic-only training pipeline.

4. Results

Synthetic Benchmark (UE5 Test Set)

ViT-S (Full): Best overall accuracy (EPE: 1.74 px, Depth MAE: 23.40 cm, $\delta_1$ : 95.90%).
ViT-L: Slightly lower EPE (1.63 px) but worse metric depth performance and significantly higher compute cost.
PrunePlus: Moderate accuracy (EPE: 5.87 px, Depth MAE: 64.26 cm, $\delta_1$ : 87.59%).
PruneStereo/Nano: Significant degradation. PruneNano has Depth MAE > 1m, making it unusable.

Jetson Orin Deployment (TensorRT FP16)

Model	FPS	Depth MAE (cm)	Usability
ViT-S	~2.2 FPS	23.40	Too Slow (Latency ~450ms limits closed-loop control)
ViT-L	~0.8 FPS	26.94	Impractical
PrunePlus	~3.3 FPS	64.26	Recommended (Balanced for approach guidance)
PruneStereo	~6.9 FPS	57.63	Unsafe (High error despite speed)
PruneNano	~8.5 FPS	112.16	Unsafe (Unreliable depth)

Critical Insight: At a UAV approach speed of 0.3 m/s, ViT-S (2.2 FPS) leaves ~13.5 cm of unmonitored travel per frame, which is risky for final actuation. PrunePlus (3.3 FPS) updates every ~9 cm, providing sufficient reaction margin despite its coarser depth accuracy.
Zero-Shot Transfer: ViT-S and PrunePlus successfully preserved branch continuity and depth ordering on real photos, whereas lightweight variants showed fragmentation.

5. Significance & Conclusion

Practical Recommendation: DEFOM-PrunePlus is identified as the optimal solution for current onboard UAV pruning. It offers the best balance between latency (3.3 FPS) and safety (Depth MAE ~64 cm), enabling responsive approach guidance.
Architectural Insight: The study proves that the multi-scale RefineNet fusion in the DPT decoder is critical for resolving thin-branch geometry. Aggressive compression (removing these layers) leads to catastrophic failure in depth estimation, regardless of frame rate.
Future Path: The authors suggest that future hardware acceleration (INT8 quantization, next-gen Jetson) could make the highly accurate ViT-S model deployable. Until then, PrunePlus serves as the practical standard, potentially refined with real-world fine-tuning to close the remaining domain gap.

This work bridges the gap between high-fidelity foundation models and the strict latency constraints of embedded robotics, providing a concrete roadmap for deploying vision-based control in hazardous forestry operations.