PoInit-of-View: Poisoning Initialization of Views Transfers Across Multiple 3D Reconstruction Systems

1. Problem Statement

The paper addresses a critical security vulnerability in modern 3D reconstruction systems, specifically those relying on Structure-from-Motion (SfM) for initialization (e.g., NeRF, 3D Gaussian Splatting, MVS).

• The Gap: Existing adversarial attacks on 3D reconstruction typically backpropagate gradients through the entire reconstruction pipeline as a monolithic block. They often target single-view losses or specific rendering outputs without analyzing the internal geometric modules.
• The Vulnerability: The authors identify that the SfM initialization module is the geometric core of most 3D pipelines. It performs keypoint detection, feature matching, and camera pose estimation to establish a sparse 3D structure. If this foundation is corrupted, all subsequent dense reconstruction and neural rendering steps fail.
• The Goal: To develop a poisoning attack that targets the SfM initialization phase to induce cross-view inconsistencies. The objective is to create perturbations that are imperceptible to humans but cause a collapse in geometric initialization, leading to catastrophic failure in downstream 3D reconstruction tasks across diverse systems (black-box transferability).

2. Methodology: PoInit-of-View

The proposed method, PoInit-of-View, injects imperceptible perturbations into a subset of input multi-view images to disrupt the geometric consistency required for SfM.

A. Core Mechanism: Cross-View Gradient Inconsistency

Instead of optimizing for a specific rendering loss, the attack optimizes for Cross-View Gradient Inconsistency (CVI).

• Concept: SfM relies on the assumption that corresponding regions of a 3D point across different views share similar local gradient structures (edges/textures).
• Attack Strategy: The method intentionally maximizes the gradient difference between a "reference" clean view ($R_i$) and a "poisoned" view ($A_j$) for corresponding 3D points.
• Mathematical Formulation: The attack minimizes a loss function $L_{CVI} = \|G(R_i) - G(A_j)\|_1$, where $G$ represents the Sobel gradient.
• Theoretical Basis: The paper provides a theoretical analysis (Theorem 1) proving that exceeding a specific gradient inconsistency threshold ($L_{th}$) causes the probability of successful feature matching to drop exponentially. This leads to a "correspondence collapse," where the SfM algorithm cannot establish enough valid camera poses or 3D points to proceed.

B. Optimization Framework (Proxy-Guided PGD)

Since standard SfM pipelines (like COLMAP) are non-differentiable, the authors use a differentiable proxy to guide the attack:

1. Proxy Model: A differentiable 3D Gaussian Splatting (3DGS) model is trained on clean views to approximate the victim's cross-view behavior.
2. Gradient Flow: The proxy renders multi-view images, allowing the calculation of gradients for the $L_{CVI}$ loss.
3. Optimization Loop:
   • Inner Loop: Updates the poisoned images using Projected Gradient Descent (PGD) to maximize $L_{CVI}$ while minimizing perceptual distortion (using SSIM and Total Variation regularizers).
   • Outer Loop: Periodically refreshes the proxy model to account for changes in the poisoned images' appearance and visibility.
4. Constraints: The perturbations are bounded by an $\ell_\infty$ norm ($\epsilon$) to ensure they remain visually imperceptible.

3. Key Contributions

1. Identification of a Fundamental Vulnerability: The paper reveals that the geometric initialization (SfM) is intrinsically fragile to subtle cross-view inconsistencies, a vulnerability previously overlooked by attacks focusing on the entire pipeline or single views.
2. PoInit-of-View Algorithm: A novel poisoning method that targets the initialization phase by inducing cross-view gradient inconsistencies, effectively breaking the feature matching and triangulation processes.
3. Theoretical Analysis: The authors derive a theoretical bound connecting cross-view gradient inconsistency to the collapse of feature correspondences, providing a mathematical explanation for why the attack causes global reconstruction failure.
4. High Transferability: The attack is downstream-agnostic. Because it corrupts the input data before the specific reconstruction architecture (NeRF, 3DGS, MVS) begins, the same poisoned images can successfully attack diverse 3D systems without retraining the attack for each specific model.

4. Experimental Results

The method was evaluated on three standard datasets: NeRF-Synthetic, Tanks and Temples (T&T), and Mip-NeRF360, targeting COLMAP (SfM+MVS), Instant NGP (NeRF), and 3D Gaussian Splatting.

• Performance Degradation:

  • Black-Box Transfer: In transfer settings (e.g., training on 3DGS, attacking NeRF), PoInit-of-View outperformed single-view baselines by 25.1% in PSNR reduction and 16.5% in SSIM reduction.
  • SfM Collapse: The attack caused a sharp decline in SfM stability metrics. On average, the number of registered images dropped by ~70%, and triangulated keypoints collapsed by over 80% compared to clean inputs.
  • Visual Quality: Rendered views from poisoned inputs showed severe structural collapse, artifacts, and distorted geometry, whereas clean inputs produced high-fidelity reconstructions.

• Stealthiness:

  • The perturbations were visually imperceptible. Metrics between clean and poisoned images showed high similarity (PSNR $\approx$ 27.8 dB, SSIM $\approx$ 0.86).
  • Despite the subtle pixel changes, the geometric initialization failed completely, demonstrating that the attack targets the structure of the data rather than just pixel intensity.

• Ablation Studies:

  • Removing the cross-view gradient loss (using only photometric loss) failed to collapse SfM, confirming that cross-view inconsistency is the critical factor.
  • The attack effectiveness increased with the perturbation budget ($\epsilon$), with significant failure occurring at $\epsilon > 12/255$.

5. Significance and Implications

• Security Risk: The paper demonstrates that 3D reconstruction systems are highly vulnerable to "poisoning at the source." An attacker does not need access to the specific neural network architecture (NeRF vs. 3DGS) to destroy the reconstruction; they only need to corrupt the input views' geometric consistency.
• Paradigm Shift: It shifts the focus of adversarial research in 3D vision from optimizing end-to-end rendering losses to targeting the specific geometric modules (SfM) that underpin these systems.
• Defense Implications: The findings suggest that future defenses must focus on robust multi-view feature matching and adversarial regularization in pose optimization, rather than just protecting the neural rendering network.
• Broad Impact: Since SfM is the standard initialization for almost all modern 3D reconstruction and neural rendering pipelines, this vulnerability affects a vast ecosystem of applications, including autonomous driving, AR/VR, and robotics.

In summary, PoInit-of-View is a highly effective, transferable, and stealthy attack that exploits the geometric fragility of the SfM initialization stage, causing a total collapse of 3D reconstruction pipelines across different architectures.