💻 computer science

Adversarially Guided Diffusion for LiDAR Range Image Synthesis

This paper proposes the first diffusion-based unrestricted adversarial attack for 2D LiDAR range image segmentation, which leverages adversarial guidance during sampling to generate realistic, manifold-preserving examples that induce controllable and transferable segmentation errors.

Original authors: Stavros Bouras, Antonios Makris, Alexandros Gkillas, Aris S. Lalos, Konstantinos Tserpes

Published 2026-07-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Stavros Bouras, Antonios Makris, Alexandros Gkillas, Aris S. Lalos, Konstantinos Tserpes

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're driving a self-driving car. Its eyes are LiDAR sensors, which shoot out laser beams and turn the world into a giant, 3D map made of millions of tiny dots. To understand what's a car, a pedestrian, or a road, the computer squashes this 3D map into a flat, 2D "range image"—like a heat map where every pixel tells the computer how far away something is.

Now, imagine a digital prankster wants to trick this car. They don't want to just poke a few pixels with a tiny, invisible speck of noise (which is how most hackers try to do it). Instead, they want to reimagine the whole scene from scratch, creating a fake but perfectly realistic world that looks exactly like a real street but makes the car's brain go haywire.

That is exactly what this paper introduces: a new way to create these "unrestricted" fake worlds using a Diffusion Model. Think of a Diffusion Model like a master artist who starts with a canvas full of static (like TV snow) and slowly, step-by-step, cleans it up until a beautiful picture emerges. Usually, this artist follows a strict recipe to paint a realistic street.

The Big Idea: Guiding the Artist with a "Trick" Compass
The authors figured out how to sneak a secret compass into the artist's hand. As the artist paints the picture, this compass (called an "adversarial guidance") whispers, "Hey, make that road look like a sidewalk," or "Make that car look like a bush."

But here's the magic: the compass doesn't just scribble messy lines. It guides the artist to paint a scene that is still incredibly realistic. The fake street looks just like a real one, with smooth curves and proper geometry, but the self-driving car sees it completely differently.

What They Found (The Good News for Hackers, The Bad for Safety)
The researchers tested this on a famous dataset called SemanticKITTI. They found that by turning up the "strength" of their secret compass (which they call the guidance scale), they could control how much the car's segmentation accuracy drops.

With a gentle nudge (guidance scale 3), the car's accuracy dropped by about 25.7% (relative to a clean baseline).
With a stronger push (scale 6), the drop was 46.8%.
With a really strong push (scale 10), the accuracy dropped by 64.2%.

Even better, this trick works on different types of car brains, not just the one they used to train the compass. When they tried it on a different car model (RangeNet++), the accuracy still dropped by 40.1% with that same strong push. This suggests the trick is very good at fooling different systems, not just the one it was designed for.

What They Argue Against (The "Old Way" is Messy)
The paper explicitly argues against the old way of hacking, which involves taking a real photo and adding tiny, invisible specks of noise (using methods like FGSM or SegPGD).

The authors show that these old methods are like trying to paint a masterpiece by adding tiny, high-frequency static noise. It looks grainy and unnatural to a human eye.
In their tests, the old methods created images with a "Fréchet Range Image Distance" (FRID) score of 207.5 (a measure of how fake it looks), while their new method kept the score down to 136.1 (very close to real life).
The paper suggests that this new method offers a distinct effectiveness–realism trade-off. Unlike the old methods which are constrained by small perturbations, this approach generates samples that are highly realistic while still causing significant performance degradation, making them a unique and powerful alternative for testing system robustness.

How Sure Are They?
The authors are very confident in their measurements. They didn't just guess; they ran 1,024 different simulations for each test. They measured the "Chamfer Distance" (a way to check if the 3D shapes are twisted) and found their method created larger, more structured changes (going from 0.52 to 1.38) compared to the old methods, which barely moved the geometry at all.

They also tested a "targeted" version, where they specifically tried to turn a road into a sidewalk. In one example, they successfully reclassified 35.8% of the road pixels as a sidewalk, while keeping the rest of the scene looking normal.

The Bottom Line
This paper suggests that we can now generate fake LiDAR scenes that are so realistic they cause self-driving cars to make significant errors in their perception, all while looking perfectly natural to a human observer. It's not a solved problem for safety yet, but it proves that the "unrestricted" way of attacking—rewriting the whole scene rather than just poking it—is a powerful new tool that researchers need to understand.

Technical Summary: Adversarially Guided Diffusion for LiDAR Range Image Synthesis

Problem Statement
LiDAR semantic segmentation is a critical perception task for autonomous driving, where models process 3D point clouds (often projected into 2D range images) to assign semantic labels to scene elements. These models are susceptible to adversarial examples—inputs crafted to mislead the model while remaining imperceptible or structurally inconspicuous. While adversarial attacks have been extensively studied for image classification and 3D point cloud segmentation, unrestricted adversarial examples (UAEs) in the 2D LiDAR range image domain remain largely unexplored. Existing norm-bounded attacks (e.g., FGSM, PGD) apply small perturbations to fixed inputs, which can introduce high-frequency noise and fail to generalize well across different architectures. There is a need for methods that generate realistic, manifold-consistent adversarial samples capable of inducing structured segmentation errors without relying on fixed input perturbations.

Methodology
The paper proposes a diffusion-based unrestricted adversarial attack that steers the reverse sampling process of a latent diffusion model using adversarial guidance derived from a segmentation loss.

Framework: The method utilizes a pre-trained conditional latent diffusion model (specifically sem2lidar) trained on the SemanticKITTI dataset. This model generates LiDAR range images conditioned on dense semantic label maps.
Adversarial Guidance: Instead of perturbing an existing image, the attack modifies the reverse diffusion trajectory. It employs Training-Free Guidance (TFG), where an external, differentiable surrogate segmentation network ( $g_\psi$ ) provides the guidance signal.
Loss Function: The guidance is driven by a dense per-pixel cross-entropy loss ( $L_{adv}$ $L_{a d v}$ ) calculated between the predictions of the surrogate network on the generated latent and the conditioning labels (for untargeted attacks) or a target label map (for targeted attacks).
- Untargeted: Maximizes loss against ground-truth labels to degrade overall performance.
- Targeted: Minimizes loss to force specific regions (source class) into a target class.
Sampling Process: The attack uses a deterministic DDIM sampler. To ensure the generated samples retain global scene structure before introducing adversarial noise, the guidance is applied only after a predefined fraction ( $\tau^\star$ ) of the reverse trajectory has elapsed. The guidance step nudges the latent representation in the direction of the loss gradient:
$z_{t-1} = z^{ddim}_{t-1} + s \cdot \frac{g_t}{\|g_t\|_2}$
where $s$ is the guidance scale and $g_t$ is the normalized gradient of the adversarial loss.
Differentiability: Since the diffusion decoder includes a non-differentiable quantization step, the method employs a differentiable decoding path to propagate gradients from the segmentation network back to the latent space without retraining the diffusion model.

Key Contributions

First Diffusion-Based UAE for 2D LiDAR: The authors present the first unrestricted adversarial attack targeting 2D LiDAR range image semantic segmentation using a diffusion model.
Controllable Degradation: The method demonstrates that attack strength can be precisely tuned via the guidance scale ( $s$ ) and the start fraction ( $\tau^\star$ ), offering a trade-off between effectiveness and realism.
Strong Transferability: The generated adversarial examples exhibit significant transferability to black-box architectures (e.g., attacking RangeNet++ using gradients from CENet and vice versa), outperforming norm-bounded baselines in transfer scenarios.
Realism Preservation: Unlike norm-bounded attacks that often introduce high-frequency noise, the proposed method generates samples that remain statistically close to the real LiDAR data manifold, preserving geometric structure and visual smoothness.

Experimental Results
Experiments were conducted on the SemanticKITTI dataset using RangeNet++ and CENet as target networks.

Effectiveness: The attack achieved a relative mean Intersection over Union (mIoU) drop of up to 64.2% in white-box settings and 58.4% in transfer settings (with guidance scale $s=10$ ).
Comparison with Baselines:
- Compared to FGSM (Fast Gradient Sign Method) and SegPGD, the proposed method achieved comparable or superior white-box degradation while maintaining significantly better distributional realism (measured by Fréchet Range Image Distance, FRID).
- For instance, at $s=10$ , the attack achieved a 64.2% drop with an FRID of 136.1, whereas FGSM at $\epsilon=0.120$ achieved a 63.2% drop but with a much higher FRID of 207.5, indicating a larger distributional shift.
- Crucially, the proposed method showed superior transferability compared to SegPGD. At comparable white-box strength, the diffusion-based attack achieved a 40.1% transfer drop versus 21.7% for SegPGD.
Visual Quality: Visual comparisons (Figure 2) revealed that norm-bounded baselines exhibited high-frequency noise, while the guided diffusion attack produced smooth, scene-consistent images.
Targeted Attacks: The method successfully demonstrated targeted capabilities, such as reclassifying "road" pixels as "sidewalk," with specific control over the reclassification rate.

Significance and Claims
The paper claims to fill a critical gap in the literature by addressing unrestricted adversarial attacks in the 2D LiDAR range image domain, a space previously unaddressed by diffusion-based methods. The authors argue that their approach offers a distinct effectiveness–realism trade-off. By operating in the generative regime rather than the perturbation regime, the method avoids the high-frequency artifacts typical of norm-bounded attacks and produces adversarial examples that are more difficult to defend against using purification techniques (which often rely on small perturbation budgets). The work suggests that manipulating the reverse diffusion trajectory allows for the creation of adversarial samples that generalize better across different segmentation architectures, providing a more robust evaluation of model safety in autonomous driving systems.

Technical Summary: Adversarially Guided Diffusion for LiDAR Range Image Synthesis

More like this