Technical Summary: SCREP – Scene Coordinate Regression and Evidential Learning-based Perception-Aware Trajectory Generation

Problem Statement

Autonomous flight in GPS-denied indoor environments requires trajectories that maintain tightly bounded visual-localization errors. Traditional map-based visual localization methods, such as feature matching (FM) or Structure from Motion (SfM), suffer from high computational costs for map reconstruction and poor scalability in large environments. While Visual-Inertial Odometry (VIO) offers an alternative, it is limited by cumulative drift even with perception-aware guidance.

Scene Coordinate Regression (SCR) presents a learning-based alternative that directly predicts 3D scene coordinates for every pixel from a single image, enabling drift-free absolute pose estimation via a Perspective-n-Point (PnP) solver. However, SCR outputs often contain outliers, operate at low frequencies with high latency, and not all predicted pixels are reliable. These characteristics complicate the use of SCR in real-time planning and control, particularly when the planner must actively steer the camera to maximize localization accuracy.

Methodology

The authors propose SCREP, a perception-aware trajectory planner that couples an evidential learning-based SCR pose estimator with a receding-horizon trajectory optimizer. The system architecture consists of two main components:

1. Evidential SCR Pose Estimation (E-SCRNet)

The system utilizes a neural network trained via deep evidential regression to predict scene coordinates and their associated uncertainties.

• Uncertainty Quantification: Unlike standard Bayesian Neural Networks (BNNs) that require multiple forward passes (e.g., Monte-Carlo dropout), evidential learning provides closed-form estimates of both aleatoric (data noise) and epistemic (model uncertainty) uncertainties in a single forward pass.
• Entropy Metric: The network outputs a Normal Inverse-Gamma (NIG) distribution for each pixel. The authors define the total uncertainty as the predictive entropy of this distribution. This entropy map serves as a unified metric to identify reliable scene coordinates.
• State Estimation: A PnP-RANSAC module converts the scene coordinates into camera poses. To address the latency and outlier issues of SCR, a fixed-lag smoother fuses these low-rate SCR estimates with high-rate IMU data, producing a high-quality, high-frequency pose estimate.

2. Perception-Aware Trajectory Optimization

The planner operates in a receding-horizon fashion, optimizing position and yaw trajectories sequentially to track a reference path while maximizing localization accuracy.

• Representation: Trajectories are parameterized as clamped, uniform B-splines of degree 3.
• Optimization Objective: The cost function is a weighted sum of terms including waypoint tracking, smoothness, and dynamic constraints. Crucially, the Field-of-View (FOV) cost is perception-aware:
  • It rewards keeping scene coordinates with low entropy (high certainty) within the camera's frustum.
  • It penalizes high-entropy pixels by down-weighting them or removing them from the cost calculation.
  • The yaw degree of freedom is decoupled from the trajectory tangent, allowing the camera to orient specifically toward informative viewpoints without compromising the vehicle's path.

Key Contributions

1. Real-Time Perception-Aware Planner: The authors present a receding-horizon trajectory planner that explicitly injects E-SCRNet uncertainty into the optimization process, facilitating high-accuracy localization in GPS-denied environments.
2. Entropy as an Uncertainty Metric: The work empirically demonstrates that predictive entropy is a more informative and well-spread uncertainty metric compared to separate aleatoric and epistemic measures, effectively correlating with prediction error.
3. Validation: The approach is validated through both simulation and hardware-in-the-loop (HIL) experiments, showing superior localization performance compared to baselines.

Experimental Results

The system was evaluated in the FlightGoggles simulator and via HIL experiments using a quadrotor.

• Simulation Performance:
  • Compared to baselines (Forward, Vanilla perception-aware, and Fisher Information Matrix-based), the proposed method reduced translation RMSE by at least 4.9% and rotation RMSE by at least 30.8% on average.
  • Specifically, against the strongest baseline (Vanilla), the method achieved a statistically significant improvement in mean translation error (95% confidence interval [0.27, 1.84] cm).
  • The system demonstrated robustness across eight different trajectories in two distinct environments (stata1 and stata2).
• Hardware-in-the-Loop (HIL):
  • In HIL experiments, the fused pose estimation using the proposed planner reduced translational RMSE from 29.7 cm to 22.7 cm and rotational RMSE from 6.0° to 2.3° compared to a Forward-facing baseline.
• Computational Efficiency:
  • The average runtime for the full pipeline (SCR inference, smoothing, and optimization) was approximately 91.3 ms (62.2 ms for SCR, 0.8 ms for smoothing, 28.3 ms for optimization), which is sufficient for real-time operation given the module execution periods.

Significance and Claims

The paper claims to be the first active-perception work utilizing Scene Coordinate Regression (SCR). By leveraging evidential learning, the system overcomes the latency and outlier challenges typically associated with SCR, making it viable for real-time onboard robotics.

The authors emphasize that their approach allows for drift-free absolute pose estimation over long trajectories using only a compact neural network, avoiding the memory and scalability issues of feature-matching maps. While the current work focuses on SCR, the authors note that benchmarking against VIO- and SfM-based planners is left for future work, as are extensions to obstacle avoidance and robustness to appearance changes. The primary contribution is the demonstration that uncertainty-aware trajectory generation significantly enhances localization accuracy when using learning-based scene coordinate regression.