Technical Summary: ReShift

Problem Statement

Vision–Language Models (VLMs) are increasingly deployed in safety-critical domains such as autonomous driving and medical analysis. While these models have advanced through reasoning-oriented techniques like Chain-of-Thought (CoT), they remain vulnerable to backdoor attacks. Existing backdoor methods against VLMs primarily operate at the output level, utilizing token injection or response rewriting to forcibly override the final answer once a trigger is activated.

The paper identifies a critical limitation in these output-level attacks: they often neglect the internal reasoning dynamics. Consequently, the generated CoT remains logically aligned with the correct answer while the final prediction is arbitrarily altered. This creates internal inconsistencies and logical incoherence that are easily detectable by human inspection or uncertainty-based monitoring mechanisms (e.g., via perplexity distribution divergence). The authors pose the question of whether a backdoor mechanism can operate directly at the reasoning level, subtly redirecting the internal inference trajectory while preserving trajectory-level plausibility and surface-level coherence.

Methodology

The authors propose ReShift, a novel aha-moment-driven reasoning-level backdoor framework. ReShift explicitly manipulates the internal CoT dynamics of VLMs by leveraging reinforcement learning (RL)-enhanced cognitive behaviors, specifically the "aha moment"—a behavior where a model revisits earlier reasoning steps and adjusts its trajectory.

The framework consists of two core components:

1. Poisoned Reasoning-Aware Data Construction (PRDC)

This pipeline constructs a poisoned dataset designed to induce guided "aha moments."

• Process: Given a clean CoT, the system prepends a trigger phrase (e.g., "Wait, let me think") and generates guiding steps that smoothly redirect the reasoning trajectory toward a predefined target answer.
• Goal: To embed these guided shifts into the training data such that the reasoning flow remains coherent on the surface while being steered toward the malicious target.

2. Supervised–Reinforcement Joint Optimization (SRJO)

This is a one-stage optimization framework that jointly integrates Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to stabilize and amplify the trigger-induced trajectory shift.

• Hybrid Strategy: SFT is applied to the prefix (the initial correct reasoning steps) to anchor the reasoning structure and backdoor pattern. RL is applied to the suffix (post-trigger generation) to enforce convergence toward the target answer.
• Reward Functions: The RL stage utilizes three specific rewards:
  • Target Reward ($R_{target}$): Ensures the final extracted answer matches the predefined target.
  • Shift Reward ($R_{shift}$): Encourages the emergence of "entropy rebound" (defined below) to facilitate genuine reasoning shifts.
  • Format Reward ($R_{format}$): Prompts the generation of specific "aha moment" patterns (e.g., "let me think...").
• Dynamic Training: The proportion of SFT vs. RL is dynamically regulated. SFT dominates initially to inject the backdoor pattern; once stable target activation and significant entropy rebound are observed, the RL coverage increases to refine the trajectory shift.

Entropy Rebound and Theoretical Analysis

The authors introduce Entropy Rebound as a principled signal for characterizing reasoning redirection.

• Definition: They define Windowed Entropy Difference (WED) to monitor token-level entropy variations during generation. A reasoning shift is characterized by a sudden increase in window-averaged entropy (an "entropy rebound") near the convergence stage, contrasting with the stable low entropy of clean reasoning.
• Theoretical Guarantee: The paper provides a theorem establishing a lower bound on the Kullback-Leibler (KL) divergence between clean and trigger-injected reasoning trajectories. This bound is determined by the entropy gap, theoretically linking the observed entropy rebound to significant trajectory-level divergence.

Key Contributions

1. Reasoning-Level Attack Paradigm: ReShift is the first framework to explicitly redirect the internal CoT trajectory of VLMs rather than merely overriding the final output token. This approach preserves the logical consistency of the reasoning steps leading up to the shift.
2. Aha-Moment Mechanism: The work leverages RL to induce "aha moment" behaviors, allowing the model to naturally revisit and adjust its reasoning path, thereby maintaining stealthiness.
3. SRJO Framework: The authors propose the first backdoor optimization framework that jointly integrates SFT and RL within a unified process, utilizing prefix-level SFT and suffix-level RL to balance structure and target alignment.
4. Entropy Rebound Signal: The formalization of Entropy Rebound provides a theoretical and empirical signal for detecting and inducing reasoning redirection, linking entropy gaps to trajectory divergence.

Experimental Results

Extensive experiments were conducted on VLMs (Qwen2.5-VL-7B and InternVL3.5-8B) across in-domain (A-OKVQA, ScienceQA) and out-of-domain (MathVista, MMMU) benchmarks.

• Attack Success Rate (ASR): ReShift achieves high ASR (up to 1.00 on ScienceQA and 0.97 on A-OKVQA), outperforming baselines like BadToken, BadVision, and Rewrite.
• Stealthiness:
  • Reasoning Quality: ReShift maintains high coherence (Coh) and rationale (Rat) scores, comparable to the clean base model, whereas output-level attacks often degrade these metrics.
  • Perplexity: Unlike output-level attacks which show pronounced divergence in perplexity distributions between clean and trigger samples, ReShift's trigger samples remain closely aligned with clean samples.
  • Detection Resistance: When evaluated against backdoor detectors (BYE and BkdAttr), ReShift yields significantly lower detection accuracy (DACC), often approaching random guessing (0.09–0.17), compared to baselines which show moderate detectability.
• Utility Preservation: The attack maintains high clean-task accuracy and low ASR-C (the rate of clean samples being mistakenly redirected), indicating minimal impact on benign inputs.

Significance and Claims

The paper claims that ReShift represents a significant advancement in backdoor attack capabilities by moving from output-level manipulation to stable reasoning-level deviation. By preserving early-stage coherence and leveraging natural cognitive behaviors (aha moments), the attack becomes substantially harder to detect via statistical monitoring or human inspection.

The authors emphasize that reasoning dynamics constitute a critical yet underexplored attack surface. They argue that their work underscores the need for trajectory-aware defenses in multimodal models, as traditional defenses focusing on final outputs or static token distributions may fail to detect attacks that manipulate the internal inference process while maintaining surface-level plausibility. The work does not claim to solve the problem of defense but rather highlights the vulnerability of the reasoning process itself.