💻 computer science

Follow My Eyes: Backdoor Attacks on VLM-based Scanpath Prediction

This paper presents the first study of backdoor attacks on VLM-based scanpath prediction models, demonstrating that input-aware variable-output attacks can effectively manipulate gaze fixation sequences and durations while evading detection and surviving deployment on commodity smartphones, with no existing defense currently capable of simultaneously mitigating these threats and preserving model utility.

Original authors: Diana Romero, Mutahar Ali, Momin Ahmad Khan, Habiba Farrukh, Fatima Anwar, Salma Elmalaki

Published 2026-04-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Diana Romero, Mutahar Ali, Momin Ahmad Khan, Habiba Farrukh, Fatima Anwar, Salma Elmalaki

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, futuristic pair of glasses (like high-tech AR glasses) that can predict exactly where you are going to look next.

How it works normally:
You put on the glasses and say, "Find the fork." The glasses look at the messy table, guess that you will look at the plate first, then the knife, and finally the fork. Based on this prediction, the glasses only draw high-quality, sharp images where you will look, and blurry images where you won't. This saves battery and makes the world look super crisp. This is called Scanpath Prediction.

The Problem (The Backdoor Attack):
The researchers in this paper discovered a scary new way to hack these glasses. They found that if someone "poisons" the training data (the lessons the glasses learn from) before the glasses are even built, they can install a secret backdoor.

Think of it like a spy teaching a student a secret handshake. The student behaves perfectly in class, but if the teacher whispers a specific code word (the trigger), the student suddenly starts doing something completely different.

The Two Types of Hacks:

The researchers tried two different ways to hack the system, moving from a clumsy attempt to a sneaky, invisible one.

1. The "Clumsy" Hack (Fixed Path)

The Idea: The hacker forces the glasses to always look at the exact same spot (like the center of the screen) whenever the secret code is present.
The Analogy: Imagine a tour guide who usually shows you around a city. But if you say the magic word "Banana," the guide suddenly stops, turns around, and stares at a specific brick on a wall for the rest of the tour, ignoring everything else.
The Result: This is easy to catch. If you look at the guide's behavior, you'd say, "Hey, why are they staring at that brick every time someone says 'Banana'? That's weird!" The researchers found that this "staring at a brick" behavior creates obvious patterns that security systems can spot.

2. The "Sneaky" Hack (Variable Path)

This is the real danger. The researchers realized that to hide, the glasses need to act normally, just with a wrong goal. They designed two sneaky attacks:

Attack A: The "Wrong Target" Switch
- The Idea: When the secret code is present, the glasses still scan the room naturally, but they look for the wrong object.
- The Analogy: You tell the glasses, "Find the fork." The secret code is hidden in the word "fork." The glasses still scan the table, but instead of looking for the fork, they start looking for a knife. They scan the table logically, looking for a knife, and stop when they find it.
- Why it's dangerous: To an observer, the glasses look like they are working perfectly! They are scanning the table and finding something. But they are looking for the wrong thing. If the glasses are controlling a robot arm or a laser pointer, it might grab the knife instead of the fork, or point the laser at a dangerous object.
Attack B: The "Slow Motion" Delay
- The Idea: The glasses still find the right object, but they pretend to take forever to find it.
- The Analogy: You ask the glasses to find the fork. They find it, but they make you wait 5 extra seconds for every step of the search. It's like a GPS that knows the route but decides to drive in slow motion, making you late for your appointment.
- Why it's dangerous: In a fast-paced situation (like a self-driving car or a surgical robot), a delay of a few seconds can cause a crash or a mistake.

The "Trigger" (The Secret Code):
How do the hackers activate this? They use tiny, invisible tricks:

Visual: A tiny white square on the image that you can't really see.
Text: A "zero-width space" (a space character that takes up no room) hidden inside the word you type. It's like a ghost in the machine.
Multimodal: A combination of both.

Can We Stop It? (The Defenses)
The researchers tried to fix these hacked glasses using standard security tools (like retraining the model or cutting out bad parts of the brain).

The Result: It's a disaster. None of the defenses worked perfectly.
- Some defenses stopped the hack but broke the glasses (they couldn't find forks anymore).
- Some defenses kept the glasses working but failed to stop the hack.
- It's like trying to remove a poison from a cake without ruining the taste or the texture. You can't do it easily.

The Final Nail in the Coffin (Mobile Phones)
The researchers tested these hacked glasses on real smartphones (both new and old ones). Even after shrinking the software to fit on a phone (a process called "quantization"), the secret backdoor still worked. The glasses still looked for knives instead of forks, or slowed down the search, even on your pocket device.

The Big Takeaway
This paper is a wake-up call. We are building AI systems that control how we see the world (in AR, VR, and mobile devices). If the people who build these systems don't secure the "lessons" the AI learns, a hacker can secretly reprogram the AI to look at the wrong things or move too slowly, and we might not even notice until it's too late. The "eyes" of our technology can be tricked to follow a different path.

1. Problem Statement

The paper addresses a critical security gap in Vision-Language Model (VLM)-based scanpath prediction. Unlike gaze estimation (which determines where a user is currently looking), scanpath prediction forecasts the sequence and duration of future eye fixations based on scene semantics and search queries. These models are increasingly deployed in mobile and AR/VR systems to drive foveated rendering (reducing GPU load by rendering high resolution only where the user looks) and intent-based interaction.

The core problem is that the supply chain for these models is vulnerable. Since collecting large-scale eye-tracking data is expensive, practitioners often fine-tune on public datasets or use third-party pretrained models. This creates an attack surface where an adversary can inject backdoors via data poisoning or compromised model distribution. The paper investigates whether attackers can manipulate these models to:

Redirect visual attention: Force the system to focus on the wrong object.
Delay task completion: Artificially inflate fixation durations to slow down visual search.
Evade detection: Avoid standard statistical defenses that look for anomalous output patterns.

2. Methodology

A. Threat Model

Attacker: Can poison a small fraction of the training dataset (e.g., 2.5%–10%) or distribute a pre-poisoned model.
Trigger: Can be visual (patches), textual (hidden tokens), or multimodal.
Defender: Has access to a trained model and a small trusted clean dataset but no access to the original training data or knowledge of the trigger.
Goal: The model must perform normally on clean inputs (high utility) but execute malicious behavior only when a trigger is present.

B. Preliminary Study: The Failure of Fixed-Path Attacks

The authors first tested a naive adaptation of classification backdoors: forcing the model to output a fixed trajectory (e.g., always looking at the center) when a trigger is detected.

Result: While effective at hijacking the output, this approach creates detectable statistical clustering in the continuous output space. Because scanpaths are expected to vary based on the input scene, a rigid, fixed trajectory stands out as an anomaly in heatmaps and coordinate histograms, making it easily detectable via activation clustering or simple statistical analysis.

C. Proposed Variable-Output Attacks

To evade detection, the authors designed two variable-output attacks where the malicious behavior is conditioned on the input scene, producing diverse and plausible scanpaths.

Input-Aware Spatial Attack (Semantic Misdirection):
- Mechanism: When a trigger is present, the model is trained to search for an attacker-chosen "poison target" (e.g., a knife) instead of the user's query (e.g., a fork).
- Label Construction: Since the dataset lacks human scanpaths for arbitrary target swaps, the authors use a clean reference model to generate the "poisoned" scanpath for the target object.
- Stealth: The output remains a valid, input-dependent scanpath for the wrong object, avoiding the rigid clustering of fixed-path attacks.
Scanpath Duration Attack (Temporal Inflation):
- Mechanism: The model preserves the correct spatial path but manipulates the timing.
- Variants:
  - Duration Inflation: Adds a fixed delay ( $\Delta t$ ) to every fixation.
  - Extra-Fixation Injection: Inserts additional fixation points into the sequence.
- Impact: This delays the completion of visual search tasks, which can degrade the performance of time-critical AR interfaces or foveated rendering pipelines.

D. Evaluation Setup

Model: GazeFormer (a state-of-the-art Transformer-based VLM for scanpath prediction).
Dataset: COCO-Search18.
Triggers: Zero-width space (text), white patches (visual), and combinations.
Poisoning Ratios: 2.5%, 5%, and 10%.
Defenses Evaluated: Fine-tuning, Fine-pruning, Neural Attention Distillation (NAD), SecureGaze (trigger inversion), and Contrastive Fine-tuning.
Deployment: Tested on commodity smartphones (Samsung Galaxy S24 Ultra and Note9) with various quantization levels (FP32, FP16, INT8).

3. Key Contributions

First Systematic Study: This is the first work to analyze backdoor attacks specifically on VLM-based scanpath prediction, highlighting the unique challenges of continuous, sequential output spaces compared to discrete classification tasks.
Variable-Output Attack Design: The authors demonstrate that naive fixed-path attacks are detectable and propose two sophisticated input-aware attacks (spatial misdirection and temporal inflation) that evade standard clustering-based detection.
Defense Evaluation: A comprehensive evaluation of five post-training defenses reveals a critical gap: no existing defense simultaneously suppresses the attacks and preserves clean model performance.
- Fine-pruning removes the attack but destroys clean utility.
- Fine-tuning and Contrastive Learning offer inconsistent mitigation.
- NAD offers the best trade-off but leaves significant residual attack influence.
- SecureGaze fails to detect the attacks due to the multimodal nature and variable outputs.
Edge Viability: The study confirms that backdoor behavior survives quantization and deployment on mobile devices, proving the threat is practical for real-world edge systems.

4. Key Results

Attack Success:
- Spatial Attack: Successfully redirected fixations to the poison target. On triggered inputs, the BBox hit ratio (accuracy) dropped significantly (e.g., from ~0.86 to ~0.32 for visual triggers at 10% poisoning), while clean performance remained high (>0.78).
- Duration Attack: Successfully inflated viewing times. At 10% poisoning with text triggers, delays reached up to +416 ms (fixed 200ms variant) and +224 ms (fixation insertion).
- Evasion: The input-aware attacks produced zero flagged samples under Activation Clustering, whereas fixed-path attacks were easily detected.
Defense Failure:
- Fine-pruning: Reduced attack success but caused a massive drop in clean accuracy (e.g., BBox hit ratio dropped from 0.835 to 0.673), rendering the model useless for its primary task.
- Fine-tuning: Inconsistent; sometimes strengthened the backdoor (e.g., under text triggers at 10% poisoning).
- NAD: Preserved clean utility better than pruning but failed to fully suppress the attack, leaving residual delays and misdirections.
- SecureGaze: Failed to detect the backdoor because the multimodal, variable-output nature of the attack did not induce the "output collapse" pattern the defense relies on.
Mobile Deployment:
- Backdoored models maintained 84–90% fidelity to server-side outputs after quantization (INT8/FP16) on mobile devices.
- The attacks remained effective on both flagship (S24 Ultra) and legacy (Note9) hardware, with inference latencies remaining within interactive bounds for AR applications.

5. Significance and Implications

Security of Edge AI: The paper demonstrates that the shift of VLM workloads to the edge (mobile/AR) introduces severe security risks. Once a backdoored model is deployed, there is no server-side interception point to catch the malicious behavior.
Beyond Classification: It highlights that security research for ML cannot simply copy-paste defenses from classification tasks. The continuous, sequential nature of scanpath prediction requires new detection and mitigation strategies that account for output diversity and temporal structure.
Systemic Impact: Manipulating scanpaths is not just a data accuracy issue; it directly impacts system safety and efficiency.
- Foveated Rendering: Misdirected gaze could cause the system to render high-resolution textures in the wrong area, wasting battery and increasing latency.
- AR Interaction: Inflated fixation durations could cause critical delays in object recognition or instruction following in collaborative AR environments.
Open Challenge: The failure of current defenses establishes an open research challenge: developing mitigation techniques that can distinguish between legitimate input-dependent variations and malicious backdoor behaviors without sacrificing the model's primary utility.