🤖 AI

Physically Real-time Infrared Attack against Optical Flow Estimation Networks

This paper presents a novel, non-invasive physical-world attack on Optical Flow Estimation Networks that utilizes stealthy infrared lights and pre-computed adversarial examples to generate real-time, targeted disruptions across diverse environmental conditions without modifying the victim system.

Original authors: Shen You, Wei Jiang, Jiarui Liu, Yijian Ye, Qiuzhen Lin, Xiangtao Li, Ka-Chun Wong

Published 2026-07-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shen You, Wei Jiang, Jiarui Liu, Yijian Ye, Qiuzhen Lin, Xiangtao Li, Ka-Chun Wong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the world is full of invisible eyes watching everything. These aren't human eyes, but cameras hooked up to super-smart computer brains called "Optical Flow Estimation Networks." Think of these networks as the ultimate motion detectors. They don't just see a picture; they watch how every single pixel in that picture moves from one frame to the next. If a car drives by, the network calculates exactly how fast and in what direction every part of that car is traveling. This is the secret sauce behind self-driving cars dodging obstacles and security systems tracking thieves. But here's the catch: these computer brains have a secret weakness. Just like a magician can fool a human eye with a sleight of hand, hackers can trick these networks with "adversarial examples"—tiny, almost invisible glitches that make the computer see things that aren't there or miss things that are. The big question scientists are asking is: Can we trick these motion detectors in the real world, not just on a computer screen, without anyone noticing?

This paper introduces a sneaky new way to do exactly that, using a tool that is practically invisible to us humans: infrared light. The authors, a team of researchers, propose a method they call a "Physically Real-time Infrared Attack." Instead of trying to paint a weird pattern on a car (which you could easily see and wipe off), they stick small infrared lights onto the target. To our eyes, the car looks normal. But to the camera, which can see infrared, the car is flashing a secret, chaotic code. The researchers found that by turning these lights on and off in a very specific, rapid rhythm, they can confuse the computer's motion detector. It's like if you were trying to track a runner, but every time they took a step, someone flashed a strobe light that made them look like they were teleporting or standing still. The computer gets so confused by the sudden changes in brightness that it stops tracking the object entirely.

The team didn't just guess this would work; they built a physical test track to prove it. They set up a camera, a moving object (a plastic plate with lights), and a controller. First, they used a "Genetic Algorithm"—think of it as a digital evolution simulator—to figure out the perfect pattern of light flashes. This part took time, like breeding the perfect racehorse. Once they found the winning pattern, they trained a fast, simple neural network (an "Adversarial Generative Network") to copy that pattern instantly. This allowed them to switch the lights on and off in real-time, faster than the camera could blink.

The results were quite convincing. When they tested this against two popular motion-tracking systems (RAFT and PWC-Net), the attack was highly effective. The computer's "vision" of the moving object turned into a blank, chaotic mess. In many tests, the system failed to see the object moving at all, or it saw a completely wrong path. The researchers showed that this trick worked even when the object moved at different speeds, when the room lighting changed from dim to bright, and even when the camera was moved further away. They found that the attack was most effective when the room wasn't too bright, as strong sunlight can overpower the infrared signal, but it still held its own in a wide range of conditions.

Crucially, the paper argues against older methods that tried to print sticky, visible stickers on cars or use digital simulations that don't work in the real world. The authors suggest that those methods often fail because real cameras pick up noise and because the printed patterns look too obvious to humans. Their infrared approach, however, stays hidden from human eyes while wreaking havoc on the machine's vision. They also discovered that the "flash" of the light—how different the brightness is between one frame and the next—was the most important part of the trick. Without that rapid change, the computer could still figure out the motion.

In short, the paper suggests that the security systems relying on motion tracking might be more fragile than we thought. By using a simple, invisible flashlight and a bit of smart timing, it is possible to make a moving object disappear from a computer's view. While this doesn't mean self-driving cars are about to crash tomorrow, it does suggest that the "eyes" of our automated world can be blinded by a little bit of invisible light, and that we need to build stronger defenses against these kinds of physical tricks.

Technical Summary: Physically Real-time Infrared Attack against Optical Flow Estimation Networks

1. Problem Statement

Optical Flow Estimation Networks (OFENs) are critical upstream models in autonomous driving, motion detection, and obstacle avoidance, providing motion vectors that feed into downstream decision-making systems. However, like other Deep Neural Networks (DNNs), OFENs are vulnerable to Adversarial Examples (AEs).

Existing physical-world attacks face four primary limitations:

Digital-to-Physical Gap: Many methods evaluate performance in the digital domain, failing to account for complex physical transformations (noise, lighting, sensor angles) required for real-world deployment.
Static Implementation: Some physical attacks simply print digital AEs, ignoring that cameras inevitably capture noise which degrades AE effectiveness.
Lack of Dynamics: Previous works often fail to account for target object movement, leading to attack failures when the object changes angle or position.
Visibility: Most attacks rely on visible patches or lights, violating the principle of imperceptibility and making them easily detectable by humans.

The paper addresses the need for a stealthy, real-time, physical attack that can dynamically disrupt OFENs as objects move, without modifying the victim system or alerting human observers.

2. Methodology: Real-time Infrared Light Attack (RILA)

The proposed method, RILA, leverages infrared (IR) lights attached to the surface of a target object. Since most cameras designed for low-light environments lack IR filters, they capture the IR light, while humans remain imperceptive to it. The attack process is divided into two stages:

A. Training Stage (Offline)

To overcome the non-differentiable nature of the physical world (the conversion function $C(\cdot)$ from perturbation to physical light), the authors employ a Genetic Algorithm (GA).

Initialization: Instead of uniform distributions, the GA uses two preset strategies based on the "Brightness Constancy" assumption of optical flow. It aims to maximize brightness differences between adjacent frames by setting initial perturbations to specific intensity levels ( $\gamma = 0.2$ or $0.8$) depending on the frame parity.
Evolution: The GA iteratively generates perturbations (light intensity vectors). For each candidate, the hardware controller sets the IR lights, the camera captures the frame, and the OFEN estimates the optical flow.
Fitness Function: A multi-objective fitness function guides the evolution using three loss terms:
1. Invisibility Loss ( $L_{Vis}$ ): Maximizes the "blankness" of the optical flow (making the object undetectable).
2. Imperceptibility Loss ( $L_{Per}$ ): Minimizes the variance of the optical flow to create chaos, preventing accurate motion perception.
3. Flash Loss ( $L_{Flash}$ ): Maximizes the difference in brightness between consecutive frames to disrupt the temporal continuity assumption.

B. Attacking Stage (Real-time)

Since GA is too slow for real-time application, the trained data (pairs of previous and optimal perturbations) is used to train an Adversarial Generative Network (AGN).

Architecture: A simple fully connected network that takes the previous perturbation $P_{i-1}$ as input and outputs the next perturbation $P_i$ .
Execution: In the physical world, the AGN generates the perturbation in milliseconds, sends it to the hardware controller, and the IR lights update instantly. This allows the attack to keep pace with the camera's frame rate (e.g., 30fps).

3. Key Contributions

Paradigm Shift: Proposes a "physical-train and physical-deploy" approach, moving away from digital-attack/physical-deploy models to directly optimize for physical constraints.
Stealth Mechanism: Utilizes infrared lights to ensure the attack is imperceptible to the human eye while remaining visible to standard cameras.
Dynamic Adaptation: Introduces a real-time generative network (AGN) trained on GA-generated data, enabling the attack to adapt to moving objects without modifying the victim model.
Specialized Loss Functions: Designs three specific loss terms (Invisibility, Imperceptibility, Flash) to effectively guide the generation of AEs against OFENs.

4. Experimental Results

Experiments were conducted in a physical test environment using a track, ambient lighting controls, 840nm IR lights, and an ESP32 controller. The victim models were RAFT and PWC-Net.

Ablation Study: Removing any of the three loss functions degraded performance. The Flash Loss was identified as the most critical component, with its removal causing the largest drop in attack efficacy (AIV decreased by 3.22%).
Ambient Lighting: The method performed robustly across varying ambient light intensities (0.1 to 1.0). While extremely bright conditions (>0.7) reduced relative IR contribution, the attack remained effective.
Object Speed: Performance (measured by Average Invisibility - AIV and Average Imperceptibility - AIP) remained high for speeds up to 33 cm/s, though a gradual decline was observed as speed increased.
Camera Distance: The attack maintained consistent performance across distances from 30cm to 120cm, attributed to the fixed coverage area of the IR lights relative to the object surface.
Model Iterations: The attack remained effective against RAFT regardless of the number of iterations (5 to 15), demonstrating robustness against the model's internal computational complexity.

5. Significance and Claims

The paper claims that RILA successfully demonstrates a practical, real-time physical attack against OFENs. Its significance lies in:

Stealth: The use of IR lights makes the attack invisible to humans, bypassing the "imperceptibility" constraint that limits many visible-light attacks.
Robustness: By training directly in the physical environment using a non-gradient-based approach (GA) and deploying via a generative network, the method overcomes the "sim-to-real" gap and noise issues inherent in digital-to-physical transfers.
Threat Assessment: The results indicate a "heavy threat" to current security systems relying on OFENs, as the attack can impair the network's ability to estimate optical flow across diverse lighting, speeds, and distances.

The authors conclude that while the current work focuses on optical flow, the methodology could be extended to other autonomous driving tasks like object detection to create more comprehensive multimodal attacks.