Technical Summary: VGP-Nav

Problem Statement

Reliable robotic navigation requires the seamless integration of accurate global localization and dense, metric-consistent obstacle perception. While multi-sensor systems (combining cameras with LiDAR) are effective, they introduce significant deployment overhead, complex spatial-temporal calibration, and synchronization challenges, particularly on dynamic platforms like humanoid robots where mechanical vibrations complicate cross-sensor alignment.

Vision-only approaches offer a low-cost alternative but face a critical limitation: existing monocular systems typically decouple global localization from metric geometric perception. Furthermore, feed-forward 3D reconstruction models, while capable of predicting poses and dense geometry in a single pass, suffer from inherent scale ambiguity. Their outputs lack a physical metric scale, rendering them unsuitable for the precise distance measurements required by robot controllers for collision-free planning.

Methodology: VGP-Nav

The authors propose VGP-Nav, a unified framework for Metric-Aware Visual Geometric Perception that relies solely on monocular RGB input. The core insight is to anchor localization-grounded visual geometry to physically meaningful scale constraints derived from ground-plane geometry. The system operates through the following pipeline:

1. Geometry-Aware Retrieval Strategy

To address the issue of spatial redundancy in large-scale visual databases (where top-k similar images often share clustered poses), VGP-Nav employs a two-stage retrieval process:

• Spatial Outlier Rejection: A larger pool of candidates is retrieved, and those deviating significantly from the geometric median pose are discarded to eliminate perceptual aliasing.
• Diversity-Driven Selection: A greedy strategy (inspired by Farthest Point Sampling) selects a subset of reference images that maximizes a combined score of visual similarity and spatial distance (translation and viewing direction). This ensures a diverse set of viewpoints with sufficient geometric baselines, which is critical for capturing the ground plane.

2. Weighted Motion Averaging

Once reference images are selected, a feed-forward reconstruction backbone (VGGT) predicts relative poses and dense geometry. To robustly estimate the global 6-DoF pose of the query image, the system uses a two-stage weighted averaging strategy:

• Rotation Averaging: Absolute rotation hypotheses are computed and represented in quaternion space. A dominant consensus is identified based on mutual angular consistency, and hypotheses deviating from this consensus are suppressed.
• Translation Averaging: Using the consensus rotation, the camera center is recovered via a robust multi-view triangulation problem solved through Iteratively Reweighted Least Squares (IRLS). This process integrates three priors: rotation consistency, geometric conditioning (promoting large-baseline intersections), and residual robustness to downweight spatial outliers.

3. Ground-Anchored Scale Recovery

To resolve the scale ambiguity of monocular reconstruction, the system anchors the geometry to the ground plane:

• Initialization: The robot estimates the ground height in a gravity-aligned world frame using a known geometric prior (the fixed distance between the camera and the robot base).
• Online Maintenance: During navigation, the reconstructed point cloud is aligned to this anchored ground height. Points are categorized based on their height relative to the ground: points near the ground are traversable, while points between 0.15m and the camera height are projected onto the ground plane to form a 2D metric occupancy map. This allows for the generation of collision-free trajectories without further optimization.

Key Contributions

1. Unified Framework: VGP-Nav is a single pipeline that jointly supports metric visual localization and geometric obstacle sensing using only a monocular RGB camera, eliminating the need for multi-sensor fusion.
2. Tightly Coupled Paradigm: The method introduces a paradigm that couples localization and perception to enable robust scale recovery. By anchoring feed-forward visual geometry to the ground plane via global pose priors, it effectively resolves monocular scale ambiguity.
3. State-of-the-Art Performance: The framework achieves state-of-the-art results on visual localization benchmarks and demonstrates successful deployment on real mobile robots, validating its practical readiness for autonomous navigation.

Experimental Results

The authors evaluated VGP-Nav across three fronts:

• Visual Localization: Tested on the 7-Scenes and Cambridge Landmarks datasets. VGP-Nav achieved new state-of-the-art performance among Relative Pose Regression (RPR) methods in both indoor and outdoor environments. It matched the accuracy of Absolute Pose Regression (APR) methods in indoor settings and surpassed them in outdoor scenarios, all without requiring scene-specific training (unlike APR and Scene Coordinate Regression methods).
• Metric Perception: Evaluated in simulation (InternScenes) with modified layouts and new obstacles. VGP-Nav outperformed single-view (MoGe-2) and multi-view (MapAnything) metric geometry baselines in Accuracy, Completeness, and Overall scores. Notably, it achieved superior results compared to MapAnything despite using only a coarse estimate of camera height rather than full Ground Truth camera parameters.
• Real-World Deployment: The system was deployed on a Unitree G1 humanoid robot equipped with an Intel RealSense D455 camera in a cluttered office environment. It successfully navigated to goal positions in 13 out of 20 trials, maintaining a steady geometric reference despite torso oscillations and mechanical vibrations.

Significance and Claims

The paper claims that VGP-Nav bridges the gap between relative geometry and metric geometric perception, enabling reliable real-world localization and obstacle sensing without complex hardware. By resolving scale ambiguity online through ground-plane constraints, the system provides a reliable metric reference for downstream planning. The authors highlight the approach's potential for scalable, low-cost, and safe autonomous navigation, particularly for platforms where multi-sensor calibration is prohibitive.

The work acknowledges limitations, noting that failure cases primarily occurred in narrow passages with textureless surfaces where localization was lost. Future work is proposed to integrate temporal consistency and semantic understanding to further enhance stability and real-time efficiency.