Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring

The paper introduces V-Skip, a dual-path token pruning mechanism that prevents "Visual Amnesia" in Multimodal Large Language Models by anchoring compression to visual salience rather than linguistic redundancy, achieving a 2.9× speedup with negligible accuracy loss.

The Gist

Imagine you have a very smart, but slightly chatty, robot assistant. You show it a picture of a red apple on a table and ask, "What color is the apple?"

The robot starts thinking out loud (this is called Chain-of-Thought reasoning). It says:
"Okay, I see an image. It looks like a fruit. It is round. It is sitting on a table. The fruit is red. It is an apple."

This thinking process helps the robot get the right answer, but it takes a long time and uses a lot of battery because the robot is saying so many words.

The Problem: "Visual Amnesia"

To make the robot faster, engineers tried to make it "shut up" and skip the boring words. They used a rule that says: "If a word is easy to guess from the words before it, delete it."

So, when the robot thought, "It is a red apple," the rule said:

• "It" is easy to guess? Delete.
• "is" is easy to guess? Delete.
• "a" is easy to guess? Delete.
• "red"? Well, if you say "apple," people usually guess "red." So, Delete.

The Result: The robot now just says, "Apple."
It's fast! But it's wrong. It forgot the most important part: the color. Because the robot deleted the word "red" thinking it was obvious, it lost its connection to the actual picture. It's like the robot suddenly forgot what it was looking at. The authors call this "Visual Amnesia" (forgetting the visual world).

The Solution: V-Skip (The "Double-Check" System)

The paper introduces a new method called V-Skip. Instead of just listening to the robot's internal grammar, V-Skip gives the robot a second pair of eyes.

Imagine the robot has two managers checking its work before it speaks:

1. The Grammar Manager (Text Path): Looks at the words and asks, "Is this word necessary for the sentence to make sense?"
   • Verdict: "Red" is predictable. Delete it.
2. The Eye Manager (Visual Path): Looks at the picture and the robot's brain activity. It asks, "Is this word pointing to something important in the picture?"
   • Verdict: "Red" is pointing directly to the apple in the photo. KEEP IT!

The Rule: V-Skip uses a "Union" strategy. If either manager says "Keep it," the word stays.

• If the Grammar Manager says "Delete" but the Eye Manager says "Keep," the word stays.
• This ensures the robot never deletes the crucial details (like colors or shapes) just because they were easy to guess linguistically.

How They Made It Fast (The "Training" Trick)

Usually, having two managers check every word would make the robot slower because it takes extra time to calculate the scores.

To fix this, the authors used a clever trick called Distillation:

1. They ran the "Double-Check" system on thousands of examples offline (while the robot was sleeping/learning).
2. They taught the robot a new, tiny habit (using a technique called LoRA) to know instinctively which words to keep and which to skip, without needing to do the math every time.
3. Now, the robot is fast like a sprinter but smart like a detective. It skips the fluff but remembers the visual details.

The Results

• Speed: The robot is 2.9 times faster.
• Accuracy: It doesn't lose its mind. On tests involving reading text in images (like menus or documents), it was 30% better than other methods that tried to cut corners.
• Hallucinations: It stopped making up things that weren't there. Because it kept the "visual anchors" (the words tied to the picture), it didn't start guessing wildly.

In a Nutshell

V-Skip is like editing a movie script.

• Old Way: Cut out every word that isn't a new plot twist. (Result: The movie makes no sense because you cut out the descriptions of the scenery).
• V-Skip Way: Cut out the boring filler words, but never cut a word if it describes something you can see on the screen.

It makes the AI faster without making it blind.

Technical Summary

Here is a detailed technical summary of the paper "Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring."

1. Problem Statement: Visual Amnesia in Multimodal CoT

Multimodal Large Language Models (MLLMs) utilize Chain-of-Thought (CoT) reasoning to decompose complex visual queries into step-by-step rationales, significantly improving performance. However, the autoregressive nature of Transformers generates extensive token sequences, leading to prohibitive latency and high computational costs (specifically, linear growth in Key-Value cache).

Existing solutions attempt to mitigate this via token compression (e.g., TokenSkip, LLMLingua-2), which prune tokens based on linguistic surprisal (predictability from preceding text). The authors identify a critical failure mode in applying these text-centric methods to multimodal tasks, termed Visual Amnesia:

• The Mechanism: Text-centric pruners discard tokens that are linguistically predictable (low entropy) even if they are critical visual anchors (e.g., the word "red" in "red apple" is predictable given "apple," but essential for visual grounding).
• The Consequence: Severing the connection to the input image leads to object hallucinations and a collapse in fine-grained reasoning accuracy, particularly in tasks requiring precise visual details (e.g., DocVQA).

2. Methodology: V-Skip and the VA-IB Framework

The paper proposes V-Skip, a novel framework that reformulates token compression as a Visual-Anchored Information Bottleneck (VA-IB) optimization problem. Instead of relying solely on language probability, V-Skip employs a dual-path gating mechanism to balance linguistic efficiency with cross-modal grounding.

A. Theoretical Foundation: VA-IB Objective

The authors define the compression goal as maximizing joint utility subject to a length budget:
$$\max_{\hat{C}} [ I(\hat{C}; A) + \lambda I(\hat{C}; V | Q) ]$$

• $I(\hat{C}; A)$ (Sufficiency): Ensures the compressed chain retains enough semantic content to predict the answer.
• $I(\hat{C}; V | Q)$ (Anchoring): Ensures high mutual dependence between the compressed chain and the visual input ($V$) to prevent hallucinations. This term specifically targets tokens that are unpredictable from text alone but deterministic given the image.

B. Dual-Path Scoring Mechanism

To operationalize the VA-IB objective, V-Skip calculates two scores for every generated token $c_t$:

1. Textual Path ($S_{text}$): Measures linguistic redundancy using negative log-likelihood (self-information). High probability (low surprise) tokens are candidates for pruning.
2. Visual Path ($S_{vis}$): Measures visual grounding by aggregating cross-modal attention weights.
   • It calculates the total attention mass directed from a text token to visual patches.
   • It employs a max-pooling strategy over specific "salient" layers and attention heads to isolate the strongest grounding signals, avoiding noise from syntactic heads.

C. The V-Skip Gating Mechanism (Union-of-Saliency)

A token is retained if it is salient in either modality. The retention mask $m_t$ is determined by:
$$m_t = \mathbb{I}(S_{text} \geq \tau_{text}) \lor \mathbb{I}(S_{vis} \geq \tau_{vis})$$

• This Union strategy ensures that tokens which are linguistically redundant but visually critical (high $S_{vis}$) are rescued from pruning.
• Tokens are only pruned if they are deemed redundant in both linguistic and visual contexts.

D. Efficient Inference via Distillation

To avoid the runtime overhead of calculating attention maps and scores during inference:

1. Offline Policy Learning: The V-Skip pruning policy is applied to a dataset to generate "Compressed CoT" examples.
2. LoRA Distillation: A lightweight Low-Rank Adaptation (LoRA) adapter is fine-tuned on this distilled dataset.
3. Result: The model learns to internalize the pruning logic, generating concise, visually grounded rationales directly without explicit online scoring or filtering.

3. Key Contributions

1. Identification of Visual Amnesia: The paper formally defines and analyzes the failure mode where text-centric compression discards essential visual anchors, leading to hallucinations.
2. VA-IB Framework: Introduces an information-theoretic objective that explicitly balances linguistic efficiency with visual grounding constraints.
3. V-Skip Architecture: Proposes a dual-path scoring mechanism (Linguistic Surprisal + Cross-Modal Attention) and a Union-of-Saliency gating strategy to selectively preserve visual anchors.
4. Efficient Distillation: Demonstrates a method to distill the pruning policy into a LoRA adapter, enabling zero-overhead inference speedups.

4. Experimental Results

The method was evaluated on Qwen2-VL (2B, 7B, 72B) and Llama-3.2-11B-Vision across MMMU (complex reasoning) and DocVQA (fine-grained OCR/spatial grounding).

• Performance vs. Compression:
  • On DocVQA, V-Skip outperforms text-centric baselines (LLMLingua-2, ASCoT) by over 30-45% in accuracy (ANLS) at high compression ratios ($\gamma=0.5$).
  • While baselines suffer massive accuracy drops (e.g., LLMLingua-2 drops 53% on DocVQA), V-Skip limits the drop to ~8%.
  • On MMMU, V-Skip maintains high accuracy, limiting loss to ~5.9% compared to >20% for baselines.
• Efficiency:
  • Achieves a 2.9× speedup in inference latency compared to full CoT generation.
  • Outperforms baselines in speed even when generating longer sequences than text-only compressors, due to the elimination of online filtering overhead.
• Visual Attribute Retention:
  • V-Skip retains 89.4% of color tokens and 91.2% of object tokens at $\gamma=0.5$, whereas LLMLingua-2 retains only ~42.5% and ~64.8% respectively.
• Hallucination Mitigation:
  • On the POPE benchmark (object hallucination), V-Skip maintains a balanced "Yes-Ratio" (~51%), closely matching the uncompressed baseline. In contrast, baselines exhibit severe "Yes-Bias" (64-66%), indicating they hallucinate objects when visual anchors are stripped.
• Scalability:
  • Demonstrates a Positive Scaling Law: Larger models (72B) show greater resilience to pruning than smaller models (2B), as larger models generate more redundant CoT chains that are easier to compress without logical fracture.

5. Significance

• Paradigm Shift: Moves multimodal compression away from purely text-centric metrics toward multimodal-aware strategies that respect the unique information distribution of vision-language tasks.
• Practical Deployment: By distilling the pruning policy into LoRA, V-Skip offers a practical solution for deploying efficient MLLMs in latency-constrained environments without sacrificing the "visual grounding" required for factual accuracy.
• Safety & Reliability: The method significantly reduces object hallucinations, a critical safety concern in MLLMs, by ensuring the reasoning chain remains tethered to the visual input.

In conclusion, V-Skip proves that efficient multimodal reasoning requires "seeing" what is being said, not just predicting the next word, effectively solving the trade-off between inference speed and visual fidelity.