Motion-Uncertainty-Aware Next-Best-View Planning for Moving Object Reconstruction

Technical Summary: Motion-Uncertainty-Aware Next-Best-View Planning for Moving Object Reconstruction

Problem Statement
Active 3D reconstruction of moving objects faces a critical challenge: the "decision-to-execution" delay. Standard Next-Best-View (NBV) planners for static objects assume that a viewpoint selected during planning remains informative upon execution. However, for moving objects, motion during this interval alters self-occlusion and visible surface regions, rendering a viewpoint evaluated at a single predicted pose potentially redundant or misaligned. Existing approaches typically address only parts of this problem: NBV planners optimize surface coverage but assume static objects, while motion-aware active perception prioritizes tracking or visibility maintenance over reconstruction coverage. This work addresses the gap by formulating NBV planning for an unknown rigid object undergoing planar motion, explicitly accounting for object motion uncertainty during the planning-to-execution interval.

Methodology
The proposed framework reformulates coverage-driven NBV planning as a belief-space viewpoint selection problem. Instead of scoring candidates at a single nominal object pose, the system evaluates viewpoints based on their expected coverage-driven score over a predictive distribution of future object states. The online receding-horizon planning loop consists of four key stages:

1. Object State Estimation and Prediction:

   • The system fuses noisy planar position measurements (from an external source) with depth observations.
   • A fixed-lag Gaussian Process (GP) smoother estimates the current object state (position and latent velocity) and predicts the state one step ahead ($t_{k+1}$).
   • This yields a Gaussian predictive belief $p(x_o^{k+1} | D_k)$ containing both position and velocity uncertainties.

2. Prediction-Conditioned Candidate Generation:

   • Candidate viewpoints are generated on an uncertainty-adaptive ellipse centered at the predicted object location.
   • The ellipse's semi-axes are scaled anisotropically based on the predicted positional covariance (larger along directions of higher uncertainty) while maintaining a nominal stand-off distance.
   • The ellipse orientation aligns with the nominal predicted heading derived from the mean velocity.

3. Reachability Filtering:

   • Candidates are filtered against the robot's kinematic constraints (maximum linear and angular velocity) to ensure they are reachable within a single planning step.

4. Motion-Uncertainty-Aware Evaluation:

   • Since the expected coverage score has no closed-form expression due to nonlinear visibility and discretization, the system uses Monte Carlo sampling.
   • $N$ samples are drawn from the predictive object-state belief. For each sample, the relative camera-object configuration is computed, and a deterministic projection-based NBV score (PB-NBV) is calculated.
   • The final score for a candidate viewpoint is the average of these scores across all samples, representing the expected observation quality under uncertainty.
   • The viewpoint with the highest expected score is selected for execution.

Key Contributions

• Formulation: A motion-uncertainty-aware NBV formulation that integrates GP-based state prediction with coverage-driven viewpoint selection, explicitly reasoning about how execution-time state uncertainty affects viewpoint quality.
• Pipeline: An online planning pipeline featuring uncertainty-adaptive candidate generation, reachability filtering, and Monte Carlo estimation of expected coverage scores.
• Validation: Comprehensive simulation and real-world experiments demonstrating improved reconstruction completeness compared to baselines.

Experimental Results
The framework was evaluated against two baselines: Non-predictive NBV (scoring at current state without propagation) and Prediction-only Tracking (following the predicted target without coverage scoring).

• Simulation: Across varying levels of process noise, the proposed method achieved the highest reconstruction completeness. It outperformed Non-predictive NBV by maintaining informative viewpoints despite object motion and outperformed Tracking-only by actively diversifying views to reveal unseen surfaces rather than just maintaining proximity.
• Ablation Studies:
  • Belief-Space Evaluation: Monte Carlo scoring (using 40 samples) significantly outperformed scoring only at the predicted mean, confirming the necessity of accounting for nonlinear uncertainty effects.
  • Candidate Geometry: Using an uncertainty-adaptive ellipse yielded higher completeness than an isotropic ring, demonstrating the value of aligning candidate search with directional uncertainty.
• Real-World Experiments: Using a mobile robot with a depth camera and injected position noise, the method successfully transferred simulation trends. It achieved higher completeness (e.g., ~79.8% vs. ~66.0% for Non-predictive at high noise) by selecting views that exposed new surfaces as the object moved, rather than simply tracking it.

Significance and Claims
The paper claims that moving-object reconstruction benefits from explicitly coupling predictive state estimation with coverage-driven viewpoint selection. The results demonstrate that:

1. Prediction alone is insufficient: Merely following a predicted target (Tracking-only) fails to maximize surface coverage.
2. Coverage without prediction is insufficient: Standard NBV methods that ignore execution-time uncertainty (Non-predictive) degrade in performance as object motion increases.
3. Uncertainty-aware planning bridges the gap: By evaluating viewpoints over a distribution of plausible future states, the system selects actions that remain informative despite motion uncertainty, leading to more complete 3D reconstructions in dynamic environments.

The work is modest in its scope, focusing on planar motion of rigid objects with known robot poses and external position measurements. It acknowledges limitations regarding localization drift, the separation of tracking from reconstruction, and approximations inherent in voxel-based scoring, suggesting future work could integrate onboard tracking and handle more complex motion models.