💻 computer science

ProBA: Probabilistic Bundle Adjustment with the Bhattacharyya Coefficient

ProBA is a probabilistic bundle adjustment framework that replaces rigid point tracks with 3D Gaussian landmarks and a kinematic pose graph to enable robust, cold-start Structure-from-Motion optimization from unstructured data by explicitly modeling spatial uncertainty and resolving mirror ambiguities.

Original authors: Jason Chui, Hector Andrade-Loarca, Daniel Cremers

Published 2026-04-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jason Chui, Hector Andrade-Loarca, Daniel Cremers

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a massive, 3D jigsaw puzzle, but there are two major problems:

You don't have the picture on the box. You have no idea what the final scene looks like, where the camera was standing, or even how the camera lens is zoomed. You are starting from absolute zero (a "cold start").
The puzzle pieces are messy. Instead of clean, perfect edges, you have thousands of fuzzy, slightly wrong connections. Some pieces fit perfectly; others are clearly from a different puzzle entirely.

This is the challenge of 3D Reconstruction (building a 3D map from 2D photos). Traditional methods, like the famous COLMAP, are like rigid, old-school puzzle solvers. They try to force pieces together immediately. If the pieces are a little fuzzy or the starting guess is slightly off, the whole puzzle collapses, or the solver gets stuck in a "local minimum" (a dead-end where the pieces look almost right, but the whole picture is twisted or mirrored).

Enter ProBA (Probabilistic Bundle Adjustment). Think of ProBA not as a rigid puzzle solver, but as a flexible, intuitive sculptor.

Here is how ProBA works, broken down into simple concepts:

1. The "Fuzzy Cloud" vs. The "Hard Point"

Old Way (Rigid Points): Traditional methods treat every match between two photos as a single, hard point. If that point is slightly wrong (noisy), it pulls the whole 3D structure off course. It's like trying to balance a house of cards on a single, shaky toothpick.
ProBA Way (3D Gaussians): ProBA treats every match as a fuzzy cloud (a 3D Gaussian). Imagine a cloud of dust instead of a solid marble.
- If the match is good, the cloud is small and tight.
- If the match is noisy or wrong, the cloud simply expands. It gets bigger and fluffier.
- The Magic: By letting the "cloud" get bigger, the system admits, "I'm not sure about this spot." This prevents the bad data from breaking the whole structure. It smooths out the bumps in the road, allowing the solver to keep moving forward even when the data is messy.

2. The "Mirror Maze" Problem

The Issue: In 3D reconstruction, there is a classic trick called the "mirror ambiguity." A scene and its mirror image (where everything is flipped inside-out) often look exactly the same from the camera's perspective. Traditional solvers often pick the wrong mirror image and get stuck there forever.
ProBA's Solution: ProBA is like a detective who refuses to pick a side immediately. It runs two parallel investigations at the same time:
1. The "Real World" hypothesis.
2. The "Mirror World" hypothesis.
- It lets both scenarios play out. As it gathers more evidence, one scenario starts to fall apart (the math stops working), while the other becomes crystal clear. ProBA then simply picks the winner. This ensures it never gets trapped in the wrong mirror world.

3. The "Adaptive Network" (Pruning Bad Links)

The Issue: When you have thousands of photos, you can't check every single photo against every other photo (that's too slow). You build a "view graph" (a network of connections). But sometimes, the computer connects two photos that don't actually belong together.
ProBA's Solution: Imagine a social network where you are trying to find the truth. If a friend tells you something that contradicts everyone else, you start to doubt them.
- ProBA does this dynamically. It looks at every connection (edge) in the network. If a connection is causing a lot of confusion (high error), ProBA turns down the volume on that link. It effectively says, "This connection is unreliable; I'll listen to the other 19 friends instead."
- This happens automatically during the process, filtering out the bad data without needing a human to clean it up first.

4. Starting from Scratch (Cold Start)

Old Way: You usually need to know roughly where the camera was standing before you start. If you guess wrong, the whole thing fails.
ProBA Way: It starts with zero knowledge. It assumes the camera is at the center, looking straight ahead, with a standard zoom. It then uses the "fuzzy clouds" and the "dual-hypothesis" strategy to slowly, gently figure out where the camera actually was and what the scene looks like. It's like waking up in a dark room and slowly turning on the lights, rather than trying to jump into the dark and guess where the furniture is.

The Result

Because ProBA is flexible, probabilistic, and smart about handling mistakes:

It works on messy, real-world photos (like random vacation snaps) where old methods fail.
It builds denser, more complete 3D models (you get the whole building, not just the corners).
It is robust. Even if 50% of the data is garbage, ProBA can still find the truth because it knows how to ignore the noise.

In a nutshell: ProBA replaces the rigid, brittle "point-and-shoot" logic of old 3D scanners with a flexible, "fuzzy-logic" approach that knows how to handle uncertainty, fix its own mistakes, and solve the puzzle even when the picture on the box is missing.

1. Problem Statement

Classical Bundle Adjustment (BA) is the standard for refining camera poses and 3D geometry in Structure-from-Motion (SfM) and SLAM. However, it faces three critical limitations in modern, unstructured environments:

Dependency on Initialization: Traditional BA requires precise metric initializations and known camera intrinsics. Without them, the optimization landscape is highly non-convex, leading to convergence in local minima or divergence.
Fragility to Noise: Classical pipelines rely on rigid "point tracks" (chains of consistent pixel observations). When integrating modern dense matchers (which offer high-fidelity but noisy correspondences), these rigid heuristics fail, causing reconstruction collapse.
Mirror Ambiguity: In prior-free SfM, a scene and its depth-inverted (mirrored) reflection are mathematically indistinguishable, often causing solvers to converge to the wrong geometric manifold.

Existing attempts to solve initialization-free BA (e.g., pseudo-object space errors) often remain computationally brittle or fail to resolve scale/mirror ambiguities robustly.

2. Methodology: ProBA Framework

ProBA reformulates Bundle Adjustment as a probabilistic optimization problem over a sparse view graph, replacing discrete point tracks with continuous volumetric representations.

A. Kinematic View Graph Parameterization

Instead of optimizing absolute poses directly, ProBA constructs a Minimum Spanning Tree (MST) of the view graph.

Relative Pose Chain: Camera poses are parameterized as relative transformations ( $\delta_i \in \mathbb{R}^6$ ) along the tree edges. Absolute poses are derived by chaining these relative transformations.
Benefit: This kinematic structure naturally propagates updates and smooths the optimization landscape, preventing the solver from getting stuck in poor local minima during the "cold start" (identity initialization).

B. Probabilistic Landmark Representation (3D Gaussians)

ProBA replaces discrete 3D points with isotropic 3D Gaussians ( $G(p_i, \sigma_i^2 I)$ ).

Soft Spatial Constraints: Instead of forcing matched points to share an exact 3D coordinate, the method optimizes for the probabilistic overlap of corresponding Gaussians.
Uncertainty Handling: If a correspondence is noisy or the initial pose is wrong, the Gaussian's variance ( $\sigma^2$ ) adaptively expands. This "softens" the loss function, allowing the optimizer to tolerate outliers without collapsing the geometry.
Objective: The system minimizes a unified Negative Log-Likelihood (NLL) objective combining:
1. 2D Reprojection Loss: The likelihood of observing a 2D point given the projected 3D Gaussian.
2. 3D Consistency Loss: The likelihood of two corresponding Gaussians overlapping in 3D space.

C. Adaptive Edge Weighting

To handle erroneous topological links in the view graph:

The system computes an instantaneous reliability score for each edge based on its local NLL relative to its neighbors.
An Exponential Moving Average (EMA) smooths these scores over time, dynamically down-weighting (pruning) edges that contribute to high error while up-weighting consistent ones. This prevents a single bad edge from corrupting the entire pose chain.

D. Dual-Hypothesis Mirror Regularization

To resolve the mirror ambiguity (depth inversion):

ProBA concurrently optimizes two independent hypotheses: a primary world and a mirrored world (transformed by a reflection matrix).
A regularization term guides the mirrored hypothesis toward a valid reflection of the primary.
Upon convergence, the hypothesis with the lower NLL is selected. This allows the system to "self-disambiguate" without prior metric knowledge.

3. Key Contributions

Initialization-Free Optimization: ProBA achieves robust convergence from a strict cold start (identity poses, random intrinsics) without requiring prior metric estimates or feature tracks.
Volumetric Uncertainty Modeling: By using 3D Gaussians instead of points, the method explicitly models spatial uncertainty, smoothing the non-convex loss landscape and naturally down-weighting noisy dense matches.
Unified Objective & Mirror Resolution: The framework integrates 2D and 3D constraints into a single NLL objective and employs a dual-hypothesis strategy to mathematically resolve mirror ambiguities.
Scalability: By optimizing over a sparse view graph with adaptive edge weighting and avoiding $O(N^2)$ exhaustive matching, the method scales efficiently to large, unstructured datasets.
Open Source: The authors provide a highly optimized CUDA implementation.

4. Experimental Results

ProBA was evaluated on four standard benchmarks: DTU, Mip-NeRF 360, ETH3D, and IMC2021 (in-the-wild).

Accuracy: ProBA significantly outperforms classical baselines (COLMAP with SIFT, DKM, RoMa) and learning-based foundation models (MASt3R-SfM, VGGT).
- On ETH3D, it achieved 91.4% Relative Translation Accuracy (RTA@5°) and 97.9% Relative Rotation Accuracy (RRA@5°), surpassing COLMAP+DKM (65.5% RTA) and VGGT.
- On Mip-NeRF 360, it achieved near-perfect accuracy (100% RRA, 99.2% RTA).
Robustness: Unlike classical methods that fail or produce fragmented geometry when fed dense matches, ProBA successfully reconstructs coherent 3D topologies in "in-the-wild" scenarios with uncalibrated intrinsics.
Ablation Studies:
- Removing the Soft Spatial Constraint caused convergence failure.
- Removing Mirror Regularization caused the system to converge to inverted (mirror) solutions on textureless datasets (DTU).
- Isotropic Gaussians were crucial; anisotropic Gaussians led to overfitting and local minima.
Efficiency: While slightly slower than feed-forward models, ProBA is computationally stable and avoids the "track-building explosion" that causes classical dense pipelines to crash on large scenes.

5. Significance

ProBA represents a paradigm shift in 3D reconstruction:

Bridging the Gap: It successfully bridges the gap between the mathematical rigor of classical geometric solvers and the robustness of deep learning-based dense matching.
Unstructured Environments: It provides a scalable foundation for SfM and SLAM in scenarios where initialization is impossible (e.g., autonomous exploration, historical site scanning) and where data is noisy or uncalibrated.
Theoretical Insight: The work demonstrates that modeling spatial uncertainty volumetrically (via Gaussians) is a superior strategy for handling the non-convexity of global optimization compared to rigid point-based heuristics.

In summary, ProBA enables robust, dense, and metric-accurate 3D reconstruction from unordered images without any prior information, solving long-standing issues of initialization sensitivity and mirror ambiguity.