💻 computer science

AdvSplat: Adversarial Attacks on Feed-Forward Gaussian Splatting Models

This paper introduces AdvSplat, the first systematic study demonstrating that feed-forward 3D Gaussian Splatting models are vulnerable to adversarial attacks, and proposes efficient black-box algorithms that use frequency-domain perturbations to significantly disrupt 3D reconstruction without accessing model internals.

Original authors: Yiran Qiao, Yiren Lu, Yunlai Zhou, Rui Yang, Linlin Hou, Yu Yin, Jing Ma

Published 2026-03-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yiran Qiao, Yiren Lu, Yunlai Zhou, Rui Yang, Linlin Hou, Yu Yin, Jing Ma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a magical camera that can take just two photos of a room and instantly build a perfect, 3D hologram of it. You can walk around this hologram, look at it from any angle, and it looks exactly like the real thing. This is what Feed-Forward 3D Gaussian Splatting (3DGS) does. It's like a super-fast, instant 3D printer for the digital world, used in everything from video games to self-driving cars.

But, just like any powerful technology, it has a secret weakness. This paper, titled AdvSplat, is the first to expose a "magic trick" that can break this system.

Here is the story of how they broke it, explained simply:

1. The Setup: The Instant 3D Builder

Think of the old way of making 3D models as a sculptor spending weeks chipping away at a block of marble for each specific statue. It's slow and can't be reused.

The new Feed-Forward 3DGS is like a master chef who has memorized millions of recipes. If you hand them two photos of a kitchen, they instantly "cook up" a 3D hologram of that kitchen without needing to learn the recipe again. It's fast, efficient, and ready for the real world.

2. The Villain: The Invisible Glitch

The researchers asked: "What if someone wanted to ruin this magic?"

They discovered that you don't need to smash the camera or delete the photos. You just need to add a tiny, invisible "glitch" to the input photos.

The Analogy: Imagine you are looking at a beautiful painting. Someone adds a few specks of dust to the canvas. To your eye, the painting looks exactly the same. But, if you look at the painting through a specific pair of "magic glasses" (the 3D model), those specks of dust cause the whole painting to melt into a blurry mess.

In the paper, they call these "adversarial perturbations." They are so small that the human eye can't see them, but they confuse the AI completely.

3. The Attack: Two Ways to Break the Magic

The researchers tried two different ways to create these glitches:

Method A: The "White-Box" Attack (The Insider)

First, they pretended to be the engineers who built the model. They knew exactly how the "chef" thought (they had access to the internal code). They used a math trick called PGD to calculate exactly which pixels to tweak to cause the maximum chaos.

Result: It worked perfectly. The 3D hologram collapsed. The quality dropped from "HD movie" to "static on an old TV."

Method B: The "Black-Box" Attack (The Hacker)

In the real world, you usually don't know how the model works. You can only send photos in and get 3D images out (like using an API). This is harder because you are guessing in the dark.

The Problem: Trying to guess the right glitch by randomly changing pixels is like trying to find a specific grain of sand on a beach by picking up one grain at a time. It would take forever and cost a fortune in computer power.

The Solution: The "Low-Frequency" Trick
The researchers realized that images are made of two types of information:

High Frequency: Tiny details (like the texture of a brick or a hair).
Low Frequency: The big shapes and colors (like the outline of a building or the sky).

They realized that the 3D model relies heavily on the big shapes. So, instead of trying to mess with every single pixel (the whole beach), they only messed with the big shapes (the outline of the beach).

The Analogy: Imagine you want to ruin a song. You could try to change every single note (impossible). Or, you could just change the bass line (the low frequencies). If you mess up the bass, the whole song sounds terrible, even if the melody is fine.

They used a mathematical tool called DCT (Discrete Cosine Transform) to isolate these "bass lines" of the image, added their invisible glitches there, and then turned it back into a photo.

Result: They broke the model with way fewer guesses. It was fast, cheap, and effective.

4. The Aftermath: Why Should We Care?

The paper shows that even a tiny, invisible change to a photo can turn a perfect 3D reconstruction into garbage.

The Scenario: Imagine a bad actor uploads a photo of a building to a public 3D map service. They add their invisible glitch. Now, anyone who tries to build a 3D model of that building gets a distorted, unusable mess.
The Risk: This could ruin trust in autonomous driving (if the car sees a fake 3D road), virtual reality, or digital archives.

Summary

AdvSplat is a wake-up call. It tells us that while these new "instant 3D builders" are amazing, they are surprisingly fragile. Just like a house of cards, a tiny, invisible breeze (the adversarial attack) can bring the whole thing down. The researchers hope this discovery will help engineers build stronger, safer models that can't be tricked so easily.

1. Problem Statement

Context: 3D Gaussian Splatting (3DGS) has become a dominant paradigm for real-time, high-fidelity 3D reconstruction. However, traditional 3DGS relies on per-scene optimization, which limits scalability and inference speed. To address this, Feed-Forward 3DGS models have emerged. These models use large-scale pretraining to reconstruct 3D scenes from a few input views in a single forward pass, enabling rapid inference and generalization.

The Gap: While the robustness of traditional 3DGS against adversarial attacks has been studied, the security of feed-forward 3DGS models remains largely unexplored. These models are increasingly targeted for commercial deployment (e.g., in autonomous driving and robotics), yet their vulnerability to malicious manipulation is unknown.

Core Challenge:

Black-Box Constraints: In real-world scenarios, attackers typically do not have access to model weights (white-box), only input/output via APIs.
High Dimensionality & Cost: Feed-forward 3DGS models are large-scale, and their outputs exist in high-dimensional pixel spaces. Traditional black-box attacks (transfer-based or random search) are ineffective due to feature space discrepancies and the lack of decision boundaries (since the task is reconstruction, not classification). Furthermore, the high computational cost of querying these large models makes query-intensive attacks impractical.

2. Methodology: AdvSplat

The authors propose AdvSplat, the first systematic study of adversarial attacks on feed-forward 3DGS. The framework operates in a black-box setting and introduces a novel frequency-domain parameterization to improve query efficiency.

A. White-Box Baseline (Vulnerability Revelation)

To first establish that these models are vulnerable, the authors applied a standard Projected Gradient Descent (PGD) attack (white-box) assuming access to model weights.

Objective: Maximize the reconstruction loss (MSE + LPIPS) between the rendered novel views and the ground truth.
Result: Even imperceptible perturbations caused a catastrophic collapse in reconstruction quality, proving the models lack inherent robustness.

B. Black-Box Attack Strategy

Since white-box access is unrealistic, AdvSplat focuses on black-box attacks using two variants: Gradient-Based and Gradient-Free.

1. Frequency-Domain Parameterization (The Core Innovation)
To address the high computational cost and dimensionality of pixel-space attacks, the authors transform the perturbation search into the frequency domain:

Block-wise DCT: Input images are partitioned into $n \times n$ blocks. A Discrete Cosine Transform (DCT) is applied to each block.
Low-Frequency Focus: Perturbations are applied only to the low-frequency components (top-left $s \times s$ sub-block) of the DCT coefficients.
Rationale: Low-frequency components contain the main structural information. Perturbing them allows for significant semantic disruption with far fewer parameters than the full pixel space, drastically reducing the number of required model queries.
Reconstruction: The perturbed coefficients are transformed back to the image space via Inverse DCT (iDCT) to generate the final adversarial example.

2. Attack Algorithms

Gradient-Based (GB): Uses Natural Evolution Strategies (NES) to estimate gradients. It samples noise in the frequency domain, queries the model, and approximates the gradient direction to update the perturbation.
Gradient-Free (GF): Uses Covariance Matrix Adaptation Evolution Strategy (CMA-ES). It iteratively samples candidate perturbations from a multivariate Gaussian distribution, evaluates their loss, and updates the mean and covariance of the distribution to converge on the optimal perturbation without explicit gradient calculation.

3. Key Contributions

First Security Study: The first work to systematically investigate the robustness of feed-forward 3DGS models against adversarial attacks.
Efficient Black-Box Framework: Proposed two query-efficient attack algorithms (Gradient-Based and Gradient-Free) that operate without access to model internals.
Frequency-Domain Optimization: Introduced a DCT-based parameterization strategy that significantly reduces the dimensionality of the search space, making black-box attacks on large 3D reconstruction models feasible and efficient.
Comprehensive Evaluation: Validated the method across multiple state-of-the-art models (DepthSplat, NoPoSplat, AnySplat) and datasets (RE10K, DL3DV).

4. Experimental Results

The authors evaluated AdvSplat on three victim models using the RE10K and DL3DV datasets.

Quantitative Performance:
- PSNR Degradation: The attack caused massive drops in Peak Signal-to-Noise Ratio. For example, on DepthSplat (RE10K), PSNR dropped from 21.09 to 7.57 (a 64.1% decrease) using the Gradient-Based variant.
- Structural Similarity (SSIM): Dropped significantly (e.g., from 0.710 to 0.289), indicating severe structural loss.
- Perceptual Loss (LPIPS): Increased drastically (e.g., +154.8%), confirming the rendered images are perceptually very different from the ground truth.
- Semantic Consistency: CLIP and DINO similarity scores also dropped significantly, showing the attack disrupts semantic content.
Qualitative Results:
- Visualizations showed that while the input adversarial images looked nearly identical to clean images (imperceptible to the human eye), the rendered novel views were completely corrupted, often appearing as noise, black artifacts, or washed-out white images.
- 3D Gaussian point cloud visualizations revealed that the attack drastically altered the colors and opacities of the Gaussians.
Efficiency of DCT:
- Experiments comparing attacks with and without DCT showed that the DCT-based approach converged faster and achieved higher loss values with the same number of queries, validating the efficiency of the frequency-domain strategy.
Attack Strength:
- The attack remained effective across various perturbation budgets ( $\epsilon$ ). Even at low budgets ( $\epsilon = 8/255$ ), the reconstruction quality collapsed.

5. Significance and Implications

Security Risk: The paper highlights a critical, overlooked vulnerability in the emerging field of generalizable 3D reconstruction. An adversary could upload imperceptibly perturbed images to a public feed-forward 3DGS service, causing the system to generate garbage 3D models for all downstream users.
Commercial Impact: This poses a severe risk to commercial applications in autonomous driving, robotics, and digital content creation, where trust in 3D reconstruction is paramount.
Call to Action: The work serves as a wake-up call for the community to prioritize robustness and defense mechanisms for feed-forward 3DGS models before they become widely deployed. It establishes a baseline for future research into defending against such frequency-domain adversarial attacks.