🤖 AI

TwinIR: Coordinated Invisible Dual-Point Attacks on Online HD Map Construction

The paper proposes TwinIR, a coordinated physical attack methodology that uses invisible near-infrared illumination on dual points to disrupt online HD map construction by suppressing compensating geometric cues, thereby significantly degrading autonomous driving perception and planning performance while remaining inconspicuous to the human eye.

Original authors: Haibo Hu, Jianghuai Deng, Chen Tang, Yang Lou, Qian Xu, Jianping Wang

Published 2026-08-06

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Haibo Hu, Jianghuai Deng, Chen Tang, Yang Lou, Qian Xu, Jianping Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to drive a car. To do this safely, the robot doesn't just "see" the road like a human does; it builds a perfect, digital blueprint of the world right in its brain. This blueprint, called an "HD map," draws every lane line, curb, and crosswalk in real-time as the car moves. If this blueprint is wrong, the robot might think a sharp turn is a straight highway, or that a dead end is an open road, leading to a very bad crash. For a long time, scientists have worried about how to trick these robots into seeing things that aren't there, like putting a sticker on a stop sign to make the robot think it's a speed limit sign. But there's a tricky problem: sometimes, even if you mess up one part of the road, the robot is so smart that it looks at the other side of the road and says, "Oh, I know what this looks like," and fixes its mistake all by itself.

This paper, titled "TwinIR," dives into that specific problem. The researchers discovered that if you try to trick a self-driving car's map with just one "trick" (like a single hidden light), the car often ignores it because it can still see the rest of the road clearly. To solve this, they created a new method called TwinIR. Instead of using just one trick, they use a coordinated pair of invisible lights to confuse the robot's brain in two places at once. They found that by using special near-infrared light—which looks like nothing to human eyes but looks like a bright, confusing glare to the car's cameras—they can make the robot completely lose its way, turning a sharp corner into a straight line or making it think a turn is coming much sooner than it actually is. The best part? To a person walking by, the road looks perfectly normal.

The "Cross-Boundary" Glitch

The story starts with a realization about how self-driving cars think. These cars use cameras to build a map of the road. If the road curves, the car needs to see the curve to know to turn. But here's the catch: if you try to trick the car by blinding one side of the road (say, the left lane), the car might just look at the right lane and the center line to figure out, "Oh, the road still curves here," and it will draw the curve correctly anyway. The researchers call this the "cross-boundary compensation effect." It's like if you tried to hide a snake in a garden by covering its head with a hat, but the snake's tail was still wiggling in the open. The garden owner (the car) sees the tail and knows there's a snake, so your hat didn't work.

Previous attempts to hack these maps used a single "blind spot" or a sticker to confuse the car. The authors of this paper found that this single-point approach often fails because the car uses the "compensation boundary" (the other side of the road) to fix the mistake. They argue that to really break the map, you can't just attack one spot; you have to attack the target and the part that tries to fix it.

Enter TwinIR: The Invisible Double-Team

To fix this, the team invented TwinIR. Think of it as a coordinated double-team in basketball. Instead of one player trying to steal the ball, two players work together to trap the opponent. TwinIR uses up to two "attack points."

The Target: The first point attacks the specific part of the road the hacker wants to change (like a curve they want to make look straight).
The Compensation: The second point attacks the "helper" part of the road that usually fixes the mistake.

The magic ingredient is Near-Infrared (NIR) light. Humans can't see this light; it's invisible to our eyes. But the cameras on self-driving cars can see it very clearly. The researchers use small, hidden lasers that shoot this invisible light. To a human walking down the street, the road looks normal. But to the car's camera, it looks like a bright, confusing glare that messes up the map.

The system works in two steps. First, it uses a computer to figure out exactly where to place these invisible lights to cause the most confusion. It checks: "If I put a light here, does the car still see the curve?" If the answer is yes, it adds a second light to the other side of the road to block that view too. It only uses two lights if that second one actually helps; otherwise, it sticks to one. This keeps the attack "sparse" (using as few lights as possible) so it's harder to notice.

What They Found: The Map Breaks

The researchers tested this on a famous dataset called nuScenes, which contains thousands of real-world driving scenes. They tried this attack on three different types of self-driving map systems (MapTR, MGMap, and DAMap).

The results were significant. When they used the old "single-point" method, the car's map accuracy dropped a bit. But when they used TwinIR with two coordinated points:

The map accuracy (called mAP) dropped by 8.18 to 8.96 percentage points for the "Road Straightening" attack (making a turn look straight).
For the "Early Turn" attack (making the car think a turn is coming sooner), the accuracy dropped by 2.84 to 5.62 percentage points.

But the real danger isn't just a bad map; it's what the car does with that map. The researchers measured how often the car's planned path became impossible to reach or unsafe.

With TwinIR, the rate of "unreachable goals" (where the car plans a path it can't actually drive) jumped by 11 to 12 percentage points compared to the single-point attack.
The rate of "unsafe planned trajectories" (paths that would crash into things) increased by 3 to 8 percentage points over the single-point attack.

In simple terms, the car didn't just get a little confused; it started planning paths that were wildly dangerous.

The Real-World Test

To prove this wasn't just a computer simulation, the team took their setup to a real, closed-off test field. They drove a real car equipped with six cameras around a test track. They placed their invisible lasers on the side of the road, hidden behind bushes and signs.

When the car drove by:

To a human: The video looked normal. The road looked exactly as it should.
To the car's camera: The invisible lasers created a massive, bright interference pattern.
The Result: The car's map changed exactly as the researchers predicted. In one test, a sharp turn was drawn as a straight line. In another, the car thought a turn was coming much earlier than it actually was.

The researchers also tested if using more than two lights helped. They tried using three, four, or even five lights. They found that adding a second light made a huge difference, but adding a third, fourth, or fifth barely improved the attack. This confirmed that two lights are the "sweet spot"—enough to break the car's ability to fix its own mistakes, but not so many that it becomes obvious or hard to set up.

Why This Matters

This paper doesn't say that self-driving cars are broken or that they will crash tomorrow. Instead, it highlights a specific weakness in how these cars "think" about the world. It shows that these systems rely heavily on seeing the whole picture, and if you can coordinate a few invisible tricks to hide the parts they use to "double-check" their work, you can fool them.

The authors suggest that future self-driving cars need to be smarter about this. They need to be able to realize when their map is being "compensated" for by other parts of the road, or they need to be able to ignore these invisible infrared tricks. Until then, this "TwinIR" method shows that a very small, invisible, and coordinated effort can have a surprisingly large impact on the safety of autonomous driving.

Technical Summary: TwinIR

Problem Statement

Online High-Definition (HD) map construction is a critical component for autonomous driving, directly feeding prediction and planning modules. While recent models have improved map consistency by aggregating geometric relations and temporal context, their security under physical-world interference remains insufficiently understood.

Previous research identified a "symmetry bias" in online map construction, where disrupting geometric cues in asymmetric scenes (e.g., forks) can cause the model to predict symmetric structures. Existing physical attacks, such as single-point blinding or adversarial patches, aim to exploit this by perturbing a specific "target boundary." However, the authors observe that these single-point attacks often fail due to a cross-boundary compensation effect. In this phenomenon, while the target boundary is perturbed, a spatially separated "compensation boundary" remains visible and retains sufficient geometric cues (e.g., curvature or turning location) for the model to reconstruct the original road geometry. Consequently, searching for a stronger single-point perturbation does not reliably suppress all evidence supporting the original prediction.

Methodology: TwinIR

To address the limitations of single-point attacks, the authors propose TwinIR, a mechanism-guided physical attack framework that utilizes at most two coordinated attack positions. The methodology consists of two primary components:

1. Objective-Conditioned Attack-Position Localization

TwinIR formulates the attack as a sparse optimization problem aiming to maximize targeted map deformation while minimizing the number of physical sources ( $K \le 2$ ). The process involves:

Boundary Role Assignment: For a given attack objective, the system assigns road boundaries to three roles:
- Target Boundary ( $B_{tgt}$ ): The boundary to be perturbed.
- Compensation Boundary ( $B_{cmp}$ ): A non-target boundary providing cues that allow the model to recover the original geometry.
- Guiding Boundary ( $B_{g}$ ): A boundary providing cues consistent with the desired deformation (e.g., a straight road for a "straightening" attack), which must remain visible.
Sequential Search:
- Primary Point ( $p^*_1$ ): The system identifies the optimal attack position on the target boundary to minimize the objective-specific loss (e.g., Chamfer distance for Road Straightening Attack (RSA) or directional displacement for Early Turn Attack (ETA)).
- Secondary Point ( $p^*_2$ ): If the primary point alone is insufficient, the system searches the compensation boundary for a secondary position. This search is conditioned on the objective: for RSA, it seeks a curved segment with similar curvature to the target; for ETA, it targets the straight segment of the reference boundary.
- Conditional Retention: The secondary point is retained only if the combined two-point configuration yields a strictly lower attack loss than the single-point configuration. This ensures sparsity and avoids unnecessary physical deployments.

2. Near-Infrared Realization

To mitigate the visibility issues associated with deploying multiple physical light sources, TwinIR utilizes near-infrared (NIR) illumination.

Spectral Stealth: The attack exploits the spectral sensitivity gap between human vision (which drops sharply beyond the visible spectrum) and CMOS camera sensors (which remain sensitive to NIR light, 700–1100nm).
Digital-to-Physical Transfer: During the offline black-box search, the attack positions are optimized using digital rendering of NIR markers (synthesized as white cores with purple halos). These optimized coordinates are then transformed from the ego-vehicle's Bird's-Eye-View (BEV) frame to physical roadside locations.
Deployment: The attacker places NIR laser emitters at these calculated locations. The system ensures line-of-sight to the relevant onboard cameras while remaining inconspicuous in full-color visible spectrum views.

Key Contributions

Identification of Cross-Boundary Compensation: The authors demonstrate that existing single-point attacks on online map construction can fail because unaffected compensation boundaries preserve the original road geometry. This insight shifts the attack paradigm from finding a single "strongest" point to identifying a minimal set of points that suppress competing geometric cues.
TwinIR Framework: A novel, mechanism-guided attack framework that dynamically selects up to two attack positions (target and compensation boundaries) based on the specific attack objective. It integrates a sparse optimization strategy with near-infrared illumination to maintain effectiveness while minimizing visual detectability.
Comprehensive Evaluation: The framework is validated against state-of-the-art online map construction models (MapTR, MGMap, DAMap) on the nuScenes dataset and in controlled physical-world experiments.

Experimental Results

The evaluation was conducted on an asymmetric-scene subset of nuScenes and a real-world testbed autonomous vehicle (AV).

Map Construction Performance:
- Road Straightening Attack (RSA): TwinIR reduced the mean Average Precision (mAP) by 8.18–8.96 percentage points across the three victim models, significantly outperforming the single-point "Blind" attack (which reduced mAP by 6.51–6.99 points). The degradation was most pronounced in road-boundary predictions.
- Early Turn Attack (ETA): TwinIR reduced mAP by 2.84–5.62 percentage points, compared to 0.81–3.85 points for the single-point attack.
Downstream Planning Impact:
- RSA: The unreachable-goal rate (UGR) increased by 25–28 percentage points over clean inputs, and by 11–12 percentage points over the single-point attack, reaching 51–55% across models.
- ETA: The unsafe planned trajectory rate (UPTR) increased by 19–20 percentage points over clean inputs, and by 3–8 percentage points over the single-point attack, reaching 30–33%.
Physical Validation: In real-world tests, TwinIR successfully induced road straightening and early-turn deformations. Crucially, the NIR sources produced clear responses in infrared-sensitive cameras while remaining virtually invisible in full-color views, confirming the stealthiness of the multi-point deployment.
Ablation on Point Count: Experiments varying the number of attack points ( $k=1$ to $5$) showed that the marginal benefit saturates after $k=2$ . Adding a second point provided substantial gains (1.22–2.24 points for RSA), whereas points 3–5 yielded negligible additional reduction (0.58–0.70 points), justifying the "at-most-two-point" design.

Significance and Claims

The paper claims that TwinIR exposes a critical vulnerability in online HD map construction: the reliance on multiple geometric cues across different boundaries makes the system susceptible to coordinated, sparse physical attacks. By identifying the cross-boundary compensation effect, the authors demonstrate that single-point attacks are often insufficient for reliable map manipulation.

The significance of TwinIR lies in its ability to:

Overcome Model Robustness: It successfully bypasses the model's ability to recover original geometry from residual cues, which single-point attacks fail to do.
Maintain Stealth: By leveraging near-infrared illumination, it achieves coordinated multi-point interference without the high visual conspicuousness typically associated with multiple physical light sources.
Propagate to Safety-Critical Decisions: The induced map errors directly translate to significant disruptions in motion planning, increasing the likelihood of unreachable goals and unsafe trajectories.

The authors position this work as a necessary step in understanding the security of online mapping systems, highlighting that future defenses must account for coordinated, multi-boundary physical interference rather than just localized perturbations.