💻 computer science

Adversarial Video Promotion Against Text-to-Video Retrieval

This paper introduces ViPro, the first adversarial attack framework designed to promote video rankings in text-to-video retrieval systems by leveraging a Modal Refinement strategy to enhance cross-modal interactions and black-box transferability, thereby exposing a critical vulnerability that could be exploited for financial gain or misinformation dissemination.

Original authors: Qiwei Tian, Chenhao Lin, Zhengyu Zhao, Qian Li, Shuai Liu, Chao Shen

Published 2026-04-15

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Qiwei Tian, Chenhao Lin, Zhengyu Zhao, Qian Li, Shuai Liu, Chao Shen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Cheating the Algorithm

Imagine you are running a massive video library, like YouTube or TikTok. You have a search bar where people type in what they want to see (e.g., "funny cat videos"). Behind the scenes, a super-smart AI (a Text-to-Video Retrieval model) reads your text and scans millions of videos to find the best matches. It puts the best matches at the very top of the list.

The Problem:
Most security research focuses on how to hide a video. Imagine someone trying to make a video disappear from the search results so no one sees it. That's like trying to sneak a spy into a building by making them invisible.

The New Threat (ViPro):
This paper introduces a much sneakier and more dangerous trick: Video Promotion. Instead of hiding a video, the attacker wants to force a specific video to the very top of the search results, even if it has nothing to do with the search term.

The Analogy: Imagine a librarian (the AI) who usually puts the most relevant books on the front desk. A hacker doesn't just try to hide a book; they try to glue a specific, irrelevant book (maybe a scam or a bad movie) to the very top of the "Best Sellers" shelf, right next to the books people actually want. If they succeed, that video gets millions of views, clicks, and money.

How They Did It: The "Video Promotion Attack" (ViPro)

The researchers created a tool called ViPro (Video Promotion). Here is how it works, broken down into simple steps:

1. The Goal: The "Sweet Spot"

Think of the AI's brain as a giant map.

Search Queries are like "cities" on the map.
Videos are like "houses."
Normally, the AI puts houses near the city that matches them.
The Attack: The attacker wants to take a video (a house) and physically drag it into the "city" of a specific search query, even if it's a terrible fit.
The Challenge: It's harder to push a video into a specific city (Promotion) than to push it out of a city (Suppression). You have to hit a tiny, specific target.

2. The Secret Sauce: "Modality Refinement" (MoRe)

To make this work, the researchers invented a special technique called MoRe. Think of this as a "Smart Glue" that helps the video stick to the search term without falling off.

Temporal Clipping (The "Scene" Cutter):
Videos are made of many frames (pictures) played quickly. Sometimes, a video has a weird jump or a sudden cut that confuses the AI.
- Analogy: Imagine trying to push a heavy sofa through a doorway. If you push the whole sofa at once, it gets stuck. MoRe cuts the sofa into smaller, manageable pieces (clips), pushes each piece through the door perfectly, and then puts them back together. This ensures the video flows smoothly into the "target city."
Semantic Weighting (The "Focus" Filter):
Sometimes a video has parts that don't match the search term at all. If the AI tries to push the whole video, the "bad parts" fight against the "good parts," and the attack fails.
- Analogy: Imagine you are trying to convince a judge (the AI) that a video is about "baking." If the video shows a dog eating a cake, the judge gets confused. MoRe tells the AI: "Ignore the dog part; focus only on the baking part." It suppresses the confusing parts and amplifies the parts that match the search, making the video look like a perfect match.

The Results: How Well Did It Work?

The researchers tested ViPro against the top 3 video search models in the world. They tried three different levels of difficulty:

White-Box: The attacker knows exactly how the AI works (like having the blueprints).
Grey-Box: The attacker knows some things but not everything.
Black-Box: The attacker knows nothing about the AI's internal code (like a stranger trying to hack a locked door).

The Outcome:

ViPro crushed the competition. It was significantly better than previous methods at forcing videos to the top of the list.
It works even when the attacker knows nothing. Even in the "Black-Box" scenario, ViPro was about 4% to 30% more effective than other methods.
It's invisible. The changes made to the video are so tiny that human eyes can't see them. It looks like a normal video, but the AI thinks it's a perfect match for the search term.

Why Should We Care?

This isn't just a math puzzle; it's a real-world danger.

Money: If a scammer can make their "Get Rich Quick" video appear at the top of "How to invest" searches, they can steal millions.
Misinformation: If a fake news video is promoted to the top of "Election results" searches, it can spread lies faster than the truth.
The Snowball Effect: Once a video is at the top, the platform's recommendation algorithm sees it getting clicks and pushes it even further, creating a viral loop of bad content.

The Takeaway

The researchers found a hole in the security of video search engines. They showed that it is surprisingly easy to trick these powerful AI systems into promoting the wrong videos to the top of the list.

The Solution?
The paper suggests that future video search engines need to be smarter. They shouldn't just look at the video picture; they should also listen to the audio, read the transcript, and check the user's history. By looking at more clues (modalities), the AI will be harder to trick, just like a detective who checks multiple sources of evidence instead of just one.

In short: The paper warns us that in the world of AI video search, "fake it till you make it" is now a viable strategy for hackers, and we need to build better defenses before it becomes a major problem.

1. Problem Statement

Text-to-Video Retrieval (T2VR) systems, powered by Vision-Language Models (VLMs), are critical for searching and recommending video content. While existing research has focused on adversarial attacks that suppress video ranks (pushing relevant videos down in search results), the threat of adversarial video promotion remains largely unexplored.

The Threat: Attackers could manipulate video content to artificially boost their ranking for specific queries. This allows malicious actors to gain financial benefits (via views/clicks) or spread misinformation by ensuring their content appears at the top of search results.
The Challenge: Unlike suppression attacks (which simply push a video out of a retrieval boundary), promotion attacks require pushing a video into the intersection of retrieval boundaries for multiple target queries simultaneously. This is a significantly harder optimization problem, especially in black-box scenarios where the attacker lacks full knowledge of the victim model.

2. Methodology

The authors propose ViPro (Video Promotion), the first adversarial attack framework designed specifically to promote video ranks in T2VR systems. The framework consists of two main components:

A. Attack Objective & Loss Function

Goal: To perturb a candidate video $X$ into $X'$ such that it appears in the top- $L$ results for a set of target queries $Q$ .
Retrieval Boundary Hypothesis: The authors define a "retrieval boundary" as a hypersphere around a query. To succeed, the perturbed video must enter the intersection of these boundaries for all target queries.
Exponential Loss ( $L_{exp}$ ): Instead of using a naive negative similarity loss (which treats all targets equally), ViPro employs an exponential loss function: $L_{exp}(S) = \exp(-S)$ . This provides adaptive gradients, applying stronger gradients to queries where the video is currently far away (low similarity) and weaker gradients to those already close, facilitating multi-target optimization.

B. Modality Refinement (MoRe)

To enhance black-box transferability (the ability of an attack generated on one model to work on another), the authors introduce MoRe, which refines the interaction between visual and textual modalities through two steps:

Temporal Clipping: Videos are not treated as a flat sequence. Frames are clustered into clips based on temporal similarity. This addresses "temporal abruptness" where averaging gradients across all frames can lead to suboptimal convergence.
Semantic Weighting:
- Inter-modality: Calculates frame-to-query similarity. Frames with low similarity to a specific query are down-weighted to suppress "anomalous" queries that cause gradient conflicts.
- Intra-modality: Uses frame-to-frame similarity to guide optimization within a clip, resolving conflicts between different frames.
- Result: The optimization is "tightened" towards the target area, avoiding local minima caused by conflicting gradients.

3. Key Contributions

Novel Attack Paradigm: Introduction of ViPro, the first targeted attack against T2VR designed to promote video ranks rather than suppress them.
Modality Refinement (MoRe): A novel technique combining temporal clipping and semantic weighting to handle fine-grained interactions, significantly improving black-box transferability.
Comprehensive Evaluation: Extensive experiments across 3 leading T2VR models (Singularity, DRL, Cap4Video), 3 datasets (MSR-VTT, DiDeMo, ActivityNet), and 3 threat models (White-box, Grey-box, Black-box).
Theoretical Analysis: A qualitative analysis of the upper and lower bounds of video promotion attacks, linking performance to the distribution of model embeddings and dataset characteristics.

4. Experimental Results

The authors evaluated ViPro against baselines adapted from image-to-text retrieval (Co-Attack, SGA) across various settings:

White-Box Performance: ViPro significantly outperforms baselines, achieving an average improvement of 30% in R@1 (Recall at 1) over other methods.
Grey-Box Performance: Under restricted knowledge (access to encoders but not cross-modal interaction), ViPro maintains a 10% lead over baselines.
Black-Box Performance: In the most realistic scenario (only category knowledge available), ViPro still outperforms baselines by 4% on average. Notably, it is the only method that successfully promotes videos in the Singularity model's black-box setting, where others fail or degrade performance.
Robustness: ViPro remains effective even under defenses like JPEG compression and Temporal Shuffling, outperforming baselines by significant margins (e.g., +10% over Co-Attack in white-box defense scenarios).
Imperceptibility: User studies with 17 experts and metrics (SSIM, PSNR, LPIPS) confirm that ViPro perturbations are stealthier than SGA and comparable to Co-Attack, with no obvious visual artifacts.

5. Significance and Implications

Uncovering a Critical Vulnerability: The paper highlights that T2VR systems are vulnerable to "rank promotion" attacks, a threat vector previously ignored in favor of suppression attacks.
Real-World Impact: The ability to manipulate search rankings has severe implications for content monetization, misinformation spread, and platform integrity.
Defense Insights: The study suggests that current defenses are insufficient. It proposes that future robust T2VR models should incorporate multi-modal fusion (e.g., audio, metadata) to reduce reliance on visual features alone, thereby increasing the difficulty of visual-only adversarial attacks.
Theoretical Bounds: The analysis reveals that the success of promotion attacks is fundamentally limited by the "retrieval boundary" density of the dataset and the model's embedding compactness, offering a theoretical framework for understanding attack limits.

In conclusion, ViPro demonstrates that promoting videos adversarially is not only possible but highly effective, necessitating immediate attention to defensive strategies in cross-modal retrieval systems.