💻 computer science

UE5-Forest: A Photorealistic Synthetic Stereo Dataset for UAV Forestry Depth Estimation

This paper introduces UE5-Forest, a photorealistic synthetic stereo dataset generated in Unreal Engine 5 using 115 scanned trees and a ZED Mini-simulated rig to provide pixel-perfect ground-truth disparity maps for training supervised stereo matching networks in UAV forestry applications where real-world data is unobtainable.

Original authors: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

Published 2026-03-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Invisible Branch" Puzzle

Imagine you are a drone pilot trying to teach a robot arm to prune pine trees. To do this safely, the drone needs to know exactly where every tiny branch is, down to the centimeter.

The problem is that real forests are messy.

The "Foggy Glasses" Effect: Real depth sensors (like LiDAR or stereo cameras) get confused by thin, overlapping branches and dense leaves. It's like trying to count individual threads in a tangled ball of yarn while wearing foggy glasses. The sensors often fail, giving the robot bad data.
The "No Map" Problem: To teach a computer to see depth, you usually need a "ground truth" map (a perfect reference guide). But in a real forest, nobody can make a perfect map of every leaf and branch. Without this map, the computer can't learn how to see properly.

The Solution: Building a "Perfect Digital Forest"

The researchers decided to stop trying to measure the messy real world and instead build a perfect, fake world inside a video game engine.

They created UE5-Forest, a dataset made entirely inside Unreal Engine 5 (the same high-tech engine used to make modern video games like Fortnite or The Matrix).

Here is how they did it, step-by-step:

1. The Actors: 115 "Digital Trees"

Instead of growing real trees, they downloaded 115 ultra-realistic 3D tree models from a library called Quixel Megascans.

The Analogy: Think of these not as simple cartoon trees, but as "digital twins." They were created by scanning real trees with lasers, so they have the exact same bark texture, leaf shape, and weird branch angles as real trees.
They put these 115 trees into a virtual forest.

2. The Camera: The "Drone Twin"

They didn't just take random pictures. They built a virtual drone inside the game.

The Analogy: Imagine you have a real drone with a specific camera (the ZED Mini). The researchers built a clone of that exact camera inside the computer. It has the same lens, the same sensor size, and the same distance between its two eyes (stereo baseline).
This ensures that the "fake" photos look exactly like the "real" photos a real drone would take.

3. The Dance: The "Orbit"

For every single tree, the virtual drone flew in a perfect circle around it.

It took photos from three different heights: looking straight at the tree, looking up at the top, and looking down at the bottom.
It took 48 photos per tree.
Total: 115 trees × 48 photos = 5,520 perfect photo pairs.

The Magic Trick: The "Perfect Answer Key"

This is the most important part. In the real world, we don't know the exact distance to a leaf. But in a video game, the computer knows everything.

The Analogy: Imagine a teacher giving a student a math test. In the real world, the teacher has to guess the answer. In this video game, the teacher is the computer, and it knows the answer down to the decimal point before the test even starts.
For every single pixel in the photo, the game engine generated a perfect "disparity map" (a map showing exactly how far away every pixel is).
This gives the AI a "perfect answer key" to learn from, which is impossible to get in the real world.

Why This Matters: The "Training Gym"

The researchers released this dataset to the public. Think of it as a gym for AI robots.

Before: AI robots trying to learn forestry had to train on blurry, confusing data or fake data that didn't look real. They were like athletes training in the dark.
Now: They can train on this "UE5-Forest" dataset. They learn to recognize branches and depth using these perfect, high-quality images.
The Goal: Once the robot is strong and smart from training in this "perfect gym," it can go out into the messy real world and prune trees much more accurately and safely.

Summary

The paper introduces UE5-Forest, a massive collection of 5,500+ fake-but-photorealistic tree photos taken by a virtual drone. Because it's a video game, every photo comes with a perfect depth map (the answer key). This solves a major bottleneck in robotics, allowing AI to learn how to see and prune trees in a way that was previously impossible due to the messiness of real forests.

1. Problem Statement

The core challenge addressed by this research is the lack of dense, ground-truth disparity maps required to train supervised stereo matching networks for autonomous UAV forestry operations (specifically pruning).

The Bottleneck: In real-world forestry environments, obtaining accurate depth data is nearly impossible. Thin, overlapping branches and complex canopy geometries defeat conventional depth sensors like LiDAR and structured-light systems.
Current Limitations: Existing solutions often rely on "pseudo-ground truth" generated by foundation models. However, errors in these teacher models propagate to student networks, creating an unverified accuracy ceiling.
The Gap: While synthetic datasets exist for urban driving (e.g., Scene Flow, SYNTHIA), none target the specific geometric and textural complexities of forest canopies with pixel-perfect labels.

2. Methodology

The authors developed UE5-Forest, a synthetic dataset generated entirely within Unreal Engine 5 (UE5) to bridge the gap between simulation and reality.

A. Asset Generation

Source Material: 115 photogrammetry-scanned tree models were sourced from the Quixel Megascans library.
Diversity: The assets cover a wide spectrum of species, morphologies, and growth forms (deciduous and coniferous), preserving authentic bark textures, leaf geometry, and branch topology.
Rendering Engine: UE5's Nanite (for high-polygon geometry) and Lumen (for global illumination) were used to render physically based lighting, realistic shadows, and material interactions at interactive rates.

B. Camera Simulation & Geometry

To minimize the "sim-to-real" domain gap, the virtual camera rig was calibrated to match the exact specifications of the ZED Mini camera mounted on the authors' physical UAV:

Baseline: 63 mm
Focal Length: 2.8 mm
Sensor Width: 3.84 mm
Resolution: 1920 × 1080 pixels
Focal Length (pixels): Calculated as 1400 px.

C. Data Acquisition Strategy

For each of the 115 trees, a simulated drone orbited the trunk at a fixed radius (up to 2 meters) to mimic the working distance of pruning operations.

Viewpoints: 3 elevation rings were used:
1. Horizontal (0°)
2. Elevated (+45°)
3. Depressed (−45°)
Azimuth: 16 equally spaced viewpoints (22.5° intervals) per ring.
Total Output: $115 \text{ trees} \times 48 \text{ views} = \mathbf{5,520}$ rectified stereo pairs.
Ground Truth: Pixel-perfect disparity maps were mathematically derived from the engine's depth buffer using the formula $D = (f \cdot B) / Z$ .

3. Key Contributions

First Forestry-Specific Synthetic Stereo Dataset: UE5-Forest is the first publicly available dataset targeting UAV forestry with dense, error-free disparity labels, filling a critical void in the literature.
High-Fidelity Simulation: The dataset leverages UE5's advanced rendering (Lumen/Nanite) and photogrammetry assets to achieve visual authenticity that closely mimics real-world forestry conditions.
Optical Alignment: By strictly matching the ZED Mini camera intrinsics, the dataset ensures that disparity ranges and perspective geometry in the synthetic data align directly with real-world hardware, facilitating easier domain adaptation.
Comprehensive Analysis: The paper provides a statistical characterization of the dataset (disparity distributions, scene diversity) and a qualitative validation against real-world imagery from the "Canterbury Tree Branches" dataset.

4. Results and Analysis

Disparity Distribution: The ground-truth disparity values range from approximately 44 to 700 pixels, corresponding to the 0.1m–2m operating distance. The distribution shows a concentration of values (60–200 px) for trunks and major branches, with a long tail for fine foliage.
Visual Fidelity: Qualitative comparisons with real-world images from Canterbury, New Zealand, confirm that the synthetic renders exhibit naturalistic shadow patterns, ambient occlusion, and texture details comparable to real photography.
Dataset Statistics:
- Total Pairs: 5,520
- Resolution: 1920 × 1080
- Split: 82.6% Training (95 trees), 8.7% Validation (10 trees), 8.7% Test (10 trees).
- Format: RGB pairs, 32-bit float disparity maps (.pfm), and validity masks.

5. Significance and Future Impact

Enabling Supervised Learning: UE5-Forest provides the "gold standard" labels necessary to train deep learning stereo matching networks for forestry, a task previously hindered by the lack of reliable ground truth.
Hybrid Training Pipelines: The dataset is designed to be used in conjunction with real-world data. It can serve as a pre-training source (due to its perfect labels) followed by fine-tuning on real, unlabeled data (using pseudo-labels), effectively combining the benefits of both approaches.
Benchmarking: It offers a rigorous testbed for evaluating stereo algorithms specifically on vegetation-rich scenes, moving beyond urban driving benchmarks.
Future Directions: The authors plan to expand the dataset to include multi-tree compositions (simulating dense stands), varied environmental lighting (time of day, weather), and sensor noise simulation to further reduce the sim-to-real gap.

In conclusion, UE5-Forest represents a significant step forward in agricultural robotics and computer vision, providing the essential data infrastructure needed to advance autonomous UAV pruning systems for the forestry industry.