Technical Summary: Visual Attribution Distillation (VAD) for Multimodal On-Policy Distillation

1. Problem Statement

Multimodal Large Language Models (MLLMs) often fail due to minor perceptual errors (e.g., overlooking a word or misreading a spatial relation), which can derail the entire generation trajectory. On-Policy Distillation (OPD) addresses this by supervising student-generated trajectories using a "privileged-view" teacher (a model with access to better visual evidence, such as high-resolution crops).

However, current privileged-view OPD methods suffer from source-mixed corrections. When a teacher corrects a student's token, the correction is a blend of:

1. Visual signals: Evidence derived from the privileged view.
2. Linguistic priors: General language patterns.
3. Teacher-specific effects: Idiosyncrasies of the teacher model.

Existing approaches fail to distinguish which part of the teacher's correction is actually supported by the visual evidence.

• Vision-OPD directly distills the teacher's full distribution, transferring source-mixed noise.
• VA-OPD and V-Zero use "visual advantage" to weight tokens or trajectories but fail to explicitly reconstruct the target based on what the evidence supports. Crucially, positive visual advantage metrics can miss refutation: revealing evidence might suppress a wrong token without necessarily boosting the correct one, a dynamic standard advantage metrics fail to capture effectively.

The core challenge is to estimate which component of a teacher's correction is attributable to a controlled visual intervention, rather than merely determining where or how strongly to distill.

2. Methodology: Visual Attribution Distillation (VAD)

VAD introduces a counterfactual target-reconstruction algorithm that isolates the visually attributable part of a teacher correction. The method operates during training using a fixed teacher and a student policy, but does not require auxiliary views during inference.

2.1 On-Policy Counterfactual Views

For every student-generated prefix $y_{<t}$, VAD queries the same fixed teacher under three conditions:

1. Student View ($p^0_S$): The student's distribution given the full image $x^0$.
2. Evidence-Present View ($p^+_T$): The teacher's distribution given a cropped, evidence-rich view $x^+$.
3. Evidence-Removed View ($p^-_T$): The teacher's distribution given a degraded or removed view $x^-$.

2.2 Attribution via Intervention Proxy

VAD defines two vectors in the space of centered log-probabilities:

• Teacher Correction ($r_t$): The difference between the privileged teacher ($p^+_T$) and the student ($p^0_S$).
• Visual Evidence Proxy ($u_t$): The difference between the evidence-present and evidence-removed teacher views ($p^+_T - p^-_T$).

The vector $u_t$ serves as a signed proxy for the visual evidence direction. A positive value indicates the evidence supports a token; a negative value indicates the evidence refutes it.

2.3 Target Reconstruction

Instead of matching the full teacher distribution, VAD reconstructs a new target by projecting the original correction $r_t$ onto the visual proxy $u_t$:

1. Decomposition: The correction is split into a visually attributed component ($r^{vis}_t$) and a residual ($r^{res}_t$).
   • $r^{vis}_t = \beta_t u_t$, where $\beta_t$ is a non-negative coefficient ensuring only agreement with the visual direction is preserved.
   • $r^{res}_t = r_t - r^{vis}_t$ (treated as a mixed residual, not purely linguistic).
2. Support and Refutation Branching: The method separates the support (positive $u_t$) and refutation (negative $u_t$) branches. It allocates a "visual correction budget" to these branches, capping the support share to prevent uncertain evidence from dominating.
3. Student-Anchored Target: The final target $q^{VAD}_{T,t}$ is constructed by applying the refined, signed visual shift to the student's current distribution ($p^0_S$), rather than the teacher's. This ensures the target remains anchored to the student's state while correcting only the evidence-supported tokens.

2.4 Training Objective

The primary supervision signal is the Jensen-Shannon Divergence (JSD) between the student and the reconstructed target ($q^{VAD}_{T,t}$).

• Weak Regularizer: To prevent language drift (e.g., unstable formatting or length) caused by focusing solely on visual corrections, a weak regularizer derived from the original privileged teacher distribution is added. This anchors semantics and formatting without overriding the visual reconstruction.

3. Key Contributions

1. Identification of Source-Mixed Limitations: The paper identifies that direct teacher matching and positive visual-advantage weighting fail to explicitly estimate which correction components are supported by controlled visual interventions.
2. VAD Algorithm: Introduction of a method that uses counterfactual visual interventions to:
   • Construct a proxy direction for visual evidence.
   • Decompose the teacher correction into visually attributable and residual components.
   • Reconstruct a student-anchored target that separates support and refutation.
3. Empirical Validation: Demonstration of consistent gains across six fine-grained visual benchmarks at 4B and 9B parameter scales, outperforming direct distillation and visual-advantage weighting.

4. Experimental Results

The authors evaluated VAD on Qwen3.5-4B and Qwen3.5-9B models across six benchmarks (V*, ZoomBench, HR-Bench 4K/8K, MME-RealWorld EN/CN).

• Performance Gains: VAD achieved the highest average accuracy (Avg6) at both scales, outperforming the best scale-matched baselines by 2.40 points (4B) and 2.80 points (9B).
• Comparison to SOTA: The 4B VAD checkpoint outperformed significantly larger models, including Gemini 3 Flash, Gemini 3.1 Pro, and the 397B Qwen3.5 model.
• Semantic Analysis: Decomposition analysis confirmed that the attributed component ($r^{vis}_t$) is enriched in visual attributes, objects, and decision tokens (e.g., "vertical," "metal," "A/B/C/D"), while the residual component ($r^{res}_t$) contains linguistic scaffolding and meta-content.
• Generalization: Unlike other post-training methods that sometimes degrade held-out generalization, VAD preserved or slightly improved performance on held-out tasks (MMVP, CV-Bench, MMStar, POPE), showing that visual specialization did not sacrifice broader capabilities.
• Ablation Studies:
  • Target Reconstruction: The branch-aware reconstruction provided the largest gain (+1.00 point over one-sided projection without regularization).
  • Regularization: The weak teacher anchor was crucial for stabilizing output format and length.
  • Divergence: JSD was found to be the most balanced default divergence compared to Forward or Reverse KL.

5. Significance and Claims

The paper claims that counterfactual target reconstruction is a more effective alternative to source-mixed supervision in multimodal OPD. By explicitly attributing corrections to visual evidence and reconstructing the learning target around the student's distribution, VAD ensures that the model learns what the image supports rather than blindly mimicking a teacher's mixed signals.

The authors position VAD as a method that improves parameter efficiency (small models outperforming much larger ones) and perceptual grounding without requiring complex inference-time search or tool use. The method is presented as a robust solution to the "visual-correction attribution problem," offering a principled way to filter noise from privileged supervision.

Limitations Acknowledged by the Authors:

• The visual evidence is represented by a single contrastive vector from one view pair, which may bias compositional evidence.
• The current projection yields semantic enrichment rather than a perfectly identifiable separation; the attributed component may still retain some non-visual teacher effects.