Iterative Hybrid Discrete-Continuous Viewpoint Planning for UAV Photogrammetry

Technical Summary: Iterative Hybrid Discrete-Continuous Viewpoint Planning for UAV Photogrammetry

Problem Statement
Unmanned aerial vehicle (UAV) photogrammetry relies on camera networks that provide sufficient surface coverage, image overlap, parallax, and resolution to ensure high-quality 3D reconstruction. However, conventional flight patterns, such as nadir "lawnmower" trajectories, are often poorly adapted to complex scene geometries. This misalignment leads to local reconstruction errors, incomplete surfaces, and imprecise models when structures are observed from unsuitable angles, excessive distances, or with insufficient overlapping images. While existing methods address these issues through online Next-Best-View (NBV) approaches or model-based planning, there remains a need for a method that balances local reconstruction quality with global image-network robustness without incurring excessive acquisition costs.

Methodology
The paper proposes an iterative hybrid discrete-continuous viewpoint planning method that generates a targeted flight path based on a proxy reconstruction. The process assumes a two-stage acquisition: an initial exploratory flight (e.g., a lawnmower pattern) creates a coarse proxy mesh, which is then used to plan a second, optimized flight.

The methodology operates through the following stages:

1. Reconstruction Heuristics: The planner evaluates surface points and viewpoint sets using specific photogrammetric scores:

   • Point-Level Scores: These estimate reconstruction quality based on frontality (alignment with surface normals), imaging distance (adherence to target Ground Sampling Distance), parallax (angular diversity), and multi-view observation count.
   • Viewpoint-Set Scores: These evaluate the global network properties, rewarding visibility (number of points observed), pairwise overlap (shared scene content between views), and graph connectivity (ensuring the image network forms a single connected component).
   • A global objective function combines these local and global scores.

2. Hybrid Discrete-Continuous Planning:

   • Candidate Generation: The system identifies weakly observed surface regions and generates candidate viewpoints around them. This involves sampling frontal views and side views at specified angular offsets, constrained by camera and gimbal limits.
   • Viewpoint Addition: New candidates are added iteratively to ensure minimum observation counts and repair weak regions, but only if they improve the global objective.
   • Clustered Continuous Optimization: To refine poses, the method employs the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). Viewpoints are partitioned into spatial clusters to reduce dimensionality. CMA-ES is selected over gradient-based or Nelder-Mead methods because the reconstruction objective is non-convex, non-differentiable, and discontinuous due to discrete visibility changes.
   • Viewpoint Culling: Redundant viewpoints are removed. If a viewpoint observes no points or creates excessive overlap without improving the global score, it is discarded to minimize image acquisition.

3. Two-Level Viewpoint Design: The final set combines two types of viewpoints:

   • Close-range detail viewpoints: Generated around specific weak regions to improve local resolution and reconstruction quality.
   • Model-coverage viewpoints: Generated to frame larger portions of the proxy model, ensuring global context and image-network stability.

4. Trajectory Ordering: The resulting unordered set of viewpoints is converted into an executable flight path by solving a Travelling Salesman Problem (TSP) to determine an efficient traversal order.

Key Contributions
The paper identifies three primary contributions:

• A reconstruction objective function that integrates local photogrammetric quality metrics with global image-network properties (visibility, overlap, and connectivity).
• A hybrid discrete-continuous planner that iteratively adds, optimizes (via clustered CMA-ES), and removes viewpoints.
• A two-level viewpoint design strategy that balances close-range detail views for local accuracy with wider model-coverage views for global robustness.

Results
The method was evaluated on three synthetic scenes derived from real-world models: Tu Duc's Tomb, the Church of St. Sophia, and the Mexico City Metropolitan Cathedral. The proposed approach was compared against Smith et al. [1] and Yan et al. [13].

• Accuracy and Completeness: The proposed method generally improved both reconstruction accuracy and completeness. It achieved the best 90th-percentile error across all scenes and the highest completeness at the 0.075 m threshold. For the Tomb and Church scenes, it outperformed baselines at the stricter 0.050 m threshold, though Yan et al. [13] showed slightly better performance at 0.050 m for the Church.
• Efficiency: The method achieved these quality improvements with significantly fewer images than Yan et al. [13] and fewer or comparable images to Smith et al. [1]. Consequently, alignment times were substantially lower than Yan et al. and comparable to Smith et al., demonstrating a favorable trade-off between reconstruction quality and acquisition cost.
• Visual Evidence: Error visualizations indicated that the proposed method produced fewer severe error regions around complex facades and entrances compared to the baselines.

Significance and Limitations
The paper claims that the proposed method offers a superior trade-off between reconstruction quality and acquisition efficiency by balancing local detail with global network robustness. It effectively addresses the limitations of fixed trajectories by adapting to scene geometry through an iterative, proxy-based approach.

However, the authors modestly acknowledge a key limitation: the method's dependence on the quality of the initial proxy reconstruction. Regions missing or poorly reconstructed in the proxy (such as internal geometries) may not be prioritized during the second flight planning. The paper suggests that future work could incorporate proxy reconstruction images to detect missed details and integrate flight-time and flight-quality terms into the optimization objective to further enhance practical UAV trajectory efficiency.