Technical Summary: Planar-SfM

Problem Statement

Structure from Motion (SfM) systems traditionally struggle with scenes dominated by planar surfaces, such as indoor sports arenas or architectural environments. In these scenarios, standard epipolar geometry-based methods become degenerate due to a two-parameter ambiguity, leading to an ill-posed estimation problem prone to errors from outlier correspondences. While existing pipelines often treat planar surfaces as a limitation, this paper argues that they present an opportunity for robust geometric constraints.

Methodology

The proposed framework, Planar-SfM, reframes the problem by treating scenes as collections of planar surfaces. It leverages homography decomposition to estimate relative camera poses, even in single-plane scenarios where traditional methods fail. The pipeline consists of four main stages:

1. Homography Pose-Graph Construction:
   The system processes input images (assumed to be calibrated) to detect and match features. For every image pair $(i, j)$, it estimates multiple homographies ($H_{ij}$), where each homography corresponds to a visible planar surface. The scene is modeled as a pose-graph $G=(V, E)$, where nodes represent cameras and edges represent homographies. Each homography is decomposed to yield a relative pose $(R_{ij}, c_{ij})$ and a surface normal $n_{ij}$.

2. Edge Similarity Scoring:
   To distinguish reliable homographies from those arising from incorrect correspondences, the method computes a similarity matrix $S$. This score evaluates the consistency between incident multi-edges based on:

   • Visual Evidence: The number of feature points that are inliers in both homographies.
   • Structural Evidence: The angular similarity between the orientation derived from homography decomposition and an auxiliary orientation estimated via the essential matrix (which is plane-independent).
   • Geometric Consistency: The angular similarity between the surface normals estimated by the homographies.

3. Graph Embedding and Pruning:
   The multi-edges of the pose-graph are embedded onto the real line using a spectral embedding approach. This maps edges to a scalar value $h$ such that consistent edges (high similarity scores) are embedded close together. The algorithm then identifies a minimum-range spanning tree on this weighted graph. This step effectively filters out unreliable edges (outliers) while retaining a maximally consistent set of connections required for pose recovery.

4. Pose Estimation:
   Using the filtered spanning tree, the method aggregates multiple homography-based estimates for each edge into a single relative pose estimate via confidence-weighted averaging.

   • Orientation: Recovered by propagating through the spanning tree.
   • Position: Since a tree lacks the rigidity required for absolute position recovery, the spanning tree is iteratively extended to a bi-connected graph by adding edges that minimize embedding inconsistency. Absolute positions are then recovered using least-unsquared-deviations optimization.

Key Contributions

The paper outlines three primary contributions:

• An Embedding Framework: A novel method to filter unreliable homography-based pose estimates, enabling motion recovery in challenging single-plane scenarios where traditional epipolar methods fail.
• Multi-Plane Aware Strategy: A technique that leverages independent estimates from different planar regions (or segmentations of a single plane) to enhance accuracy in both planar and general 3D scenes.
• Comprehensive Evaluation: Demonstrations of superior performance on highly planar datasets and competitive or state-of-the-art results on general unconstrained scenes.

Experimental Results

The method was evaluated on two datasets:

1. OTE (Overtime Elite) Basketball Dataset: A real-world sports broadcasting scenario with 46 calibrated cameras around a basketball court.
   • Result: Planar-SfM outperformed the GLOMAP baseline at $3^\circ$ and $5^\circ$ error thresholds (86.55% vs 84.72% and 95.55% vs 91.12% AUC, respectively), while remaining competitive at the $1^\circ$ threshold.
2. IMC Phototourism Dataset: Eight outdoor scenes captured under unconstrained conditions.
   • Result: The method achieved state-of-the-art performance across all thresholds ($3^\circ, 5^\circ, 10^\circ$), surpassing both classical methods (COLMAP, PixSfM) and learning-based approaches (VGGSfM, DFSfM).

Ablation Study:
Experiments on the OTE dataset confirmed that both the confidence scoring mechanism and the graph pruning (spanning tree selection) are crucial. Disabling either component resulted in a dramatic drop in accuracy, validating their complementary roles in outlier rejection.

Significance

The paper claims that by viewing planar surfaces as a source of geometric constraints rather than a limitation, it is possible to develop a unified SfM approach effective for both highly planar and general 3D environments. The significance lies in the ability to recover robust camera poses in scenarios (like indoor sports arenas) where standard epipolar geometry is degenerate, while simultaneously maintaining high performance in general unconstrained settings.