Responses Fall Short of Understanding: Revealing the Gap between Internal Representations and Responses in Visual Document Understanding

1. Problem Statement

Visual Document Understanding (VDU) is a complex task for Large Vision-Language Models (LVLMs) that requires integrating visual perception, text recognition, and structured reasoning. Currently, LVLM performance in VDU is evaluated almost exclusively based on the correctness of generated responses.

The authors identify a critical limitation in this evaluation paradigm: response accuracy does not necessarily reflect whether the model has internally captured the necessary information. There is a potential "gap" where a model may possess the correct information in its hidden states (internal representations) but fail to express it in its final text output. Furthermore, it is unclear where within the model's layers this information is most accessible (e.g., final layer vs. intermediate layers) and whether standard fine-tuning (all-layer) effectively bridges this gap.

2. Methodology

The study employs a two-phase approach: Linear Probing Analysis to diagnose the gap, followed by Targeted Fine-Tuning to mitigate it.

A. Linear Probing Analysis

To investigate internal representations, the authors trained binary linear classifiers on the hidden states of an LVLM at every layer during inference.

• Models: Qwen2.5-VL 32B, Gemma3 27B, and LLaVA-NeXT 13B.
• Token Types: Classifiers were trained on four token inputs per layer:
  1. Image-token: Visual embeddings.
  2. Text-token: Question text embeddings.
  3. All-token: Mean-pooled sequence.
  4. Last-token: The final token of the sequence.
• Tasks: Four binary classification tasks were constructed to test specific VDU capabilities:
  1. Visual Attributes Recognition: (e.g., "Is the background black?")
  2. Word Recognition: (e.g., "Is the text 'Spencerlan'?")
  3. Structure Understanding: (e.g., "Is the red boxed region a title?")
  4. Figure Understanding: (e.g., "Is Pale Green the minimum?")
• Metric: The Gap is defined as the difference between the maximum linear probing accuracy across all layers and the actual response accuracy of the model.

B. Fine-Tuning Strategies

Based on the probing results, the authors hypothesized that targeting intermediate layers could reduce the gap. They compared:

1. All-layer Fine-tuning: Standard full fine-tuning.
2. Single-step Intermediate Fine-tuning: Fine-tuning specific layer groups (Lower, Middle, Upper) or combinations (e.g., Lower-Middle).
3. Two-step Fine-tuning: Sequentially fine-tuning different layer groups (e.g., Lower then Middle).

• Configuration: The vision encoder and projector were frozen; only LLM layers and the LM head were tuned.

3. Key Findings & Results

A. The Existence of the Gap

• Internal vs. External: There is a significant gap between internal representations and generated responses. In many cases, linear probing accuracy on hidden states (up to 90%+) far exceeds the model's actual response accuracy (often 60–70%).
• Layer Dynamics: The information required to solve VDU tasks is not best represented in the final layer. Instead, the most linearly separable information often peaks in intermediate layers.
  • Observation: Sharp increases in probing accuracy occur at specific intermediate layers, suggesting a shift from visual processing to linguistic reasoning.
  • Token Analysis: The "last-token" representation generally holds the strongest signal, but image tokens often encode weak linearly separable information compared to text tokens.

B. Fine-Tuning Effectiveness

• All-layer Fine-tuning is Insufficient: While all-layer fine-tuning improves response accuracy, it fails to fully close the gap between internal representations and responses. The internal representations remain more informative than the output.
• Intermediate Layer Fine-tuning is Superior:
  • Fine-tuning intermediate layers (specifically the "Middle" group or "Lower-Middle" combinations) consistently outperformed all-layer fine-tuning in both response accuracy and linear probing accuracy.
  • This strategy significantly narrowed the gap (reducing the difference between probing accuracy and response accuracy) more effectively than full fine-tuning.
• Task-Specific Layer Importance:
  • Object-level tasks (Visual attributes, Word recognition): Benefit most from fine-tuning Middle layers or a two-step M→L (Middle then Lower) approach.
  • Relational reasoning tasks (Structure, Figure understanding): Benefit from L→M (Lower then Middle) or including Upper layers, reflecting the need for global understanding before specific extraction.
• Efficiency: Targeted intermediate fine-tuning achieved these gains while reducing trainable parameters by ~20% and training time by ~35% compared to full fine-tuning.

4. Key Contributions

1. Comprehensive Layer-wise Analysis: The first detailed investigation of internal representations in LVLMs for VDU using linear probing across all layers and token types.
2. Gap Quantification: Demonstrated a significant, persistent gap between internal knowledge and external responses in VDU, which cannot be resolved by standard all-layer fine-tuning.
3. Intermediate Layer Discovery: Revealed that the most informative representations for VDU tasks are located in intermediate layers, not the final layer.
4. Novel Fine-tuning Strategy: Proposed and validated a strategy of fine-tuning selected intermediate layers, which improves performance, reduces the representation-response gap, and increases training efficiency.

5. Significance

This work challenges the assumption that "better responses equal better understanding." It highlights that LVLMs often "know" the answer internally but fail to "speak" it due to the dynamics of autoregressive generation (where representations may be optimized for next-token prediction rather than preserving specific answer signals).

The findings suggest that:

• Evaluation: Relying solely on response accuracy is insufficient for assessing model reliability and safety.
• Optimization: Future LVLM training should consider layer-specific strategies. Targeting intermediate layers is a more efficient and effective way to align internal representations with outputs, particularly for complex multimodal reasoning tasks.
• Interpretability: Understanding where information is encoded helps in debugging model failures and improving the interpretability of multimodal reasoning.