Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

1. Problem Statement

Multimodal Large Language Models (MLLMs) often fail on complex visual tasks (e.g., charts, high-resolution perception, visual math) due to missing or poorly localized visual evidence. While external tools (cropping, search, generation) can provide this evidence, they introduce latency, complexity, and generalization issues. An alternative is visual latent reasoning, where the model emits continuous latent tokens to represent intermediate visual evidence internally.

However, existing latent reasoning methods suffer from unstable gains and often yield negligible improvements after post-training. The authors identify a fundamental feature-space mismatch as the primary cause:

• Pre-norm Architecture Issue: Modern MLLMs (e.g., Qwen2.5-VL, Gemma 4) use pre-norm transformers. In these architectures, hidden state norms accumulate significantly across layers, resulting in final-layer states that are orders of magnitude larger (e.g., ~8.7× for vision, ~546× for text) than the input embeddings the model was trained to consume.
• The Mismatch: Current latent methods typically reuse these high-norm output hidden states directly as inputs for the next latent token ("output-as-input"). This creates a feedback loop where the model receives inputs from a distribution it was not trained to process, destabilizing optimization and rendering the latent channel unreliable.

2. Methodology: The GAP Framework

The authors propose GAP (Granular Alignment Paradigm), a three-level alignment strategy to stabilize and improve visual latent reasoning.

A. Feature-Level Alignment: PCA-Aligned Latent Head

To resolve the norm and subspace mismatch, GAP does not feed raw decoder states back into the model. Instead:

• Projection: The backbone's final hidden states (after native RMSNorm) are mapped via a lightweight PCA-aligned latent head into the principal subspace of auxiliary-image vision embeddings.
• Reconstruction: The model predicts low-dimensional PCA coefficients ($c_t$) rather than full-dimensional vectors. These are reconstructed using the empirical mean ($\mu$) and top-$k$ eigenvectors ($P_k$) of the training set's vision embeddings: $\hat{v}_t = P_k c_t + \mu$.
• Effect: This forces generated latents to reside in the same norm regime and empirical subspace as the input vision embeddings, correcting the feature-space mismatch without requiring retraining of the backbone.

B. Data-Level Alignment: Context-Grounded Supervision

To ensure latent targets are interpretable and inspectable, the authors construct a dataset of 49,309 curated examples where:

• Auxiliary Images: Each query is paired with a generated "auxiliary image" representing the missing visual evidence.
• Structured Output: The training response includes a <latent> span (supervised by frozen ViT features of the auxiliary image) and a <parser> span (a textual description of the intended visual signal).
• Goal: This links continuous latent vectors to specific, human-readable visual intents, making the training process transparent.

C. Model-Level Alignment: Difficulty-Aware Assignment

To prevent noise from applying latent supervision to examples the base model already solves:

• Selection Strategy: The system estimates the base model's accuracy on a query via repeated sampling.
• Conditional Supervision: Latent targets are assigned only when the base model fails (accuracy $\le \tau$). Easy examples receive standard text-only supervision.
• Benefit: This focuses the latent capacity on difficult cases where visual reasoning is actually needed, avoiding distribution shifts on easy tasks.

3. Key Contributions

1. Diagnosis of Feature-Space Mismatch: The paper empirically demonstrates that in pre-norm MLLMs, output hidden states occupy a different norm regime than input embeddings, causing instability in output-as-input latent feedback. Simple norm rescaling (EMA) improves performance, validating this hypothesis.
2. The GAP Paradigm: A lightweight recipe combining:
   • PCA-aligned projection to map latents back to the input vision subspace.
   • Context-grounded supervision for inspectability.
   • Difficulty-aware assignment for training stability.
3. Open Dataset: Release of a 49K high-quality multimodal latent-supervision dataset covering visual CoT, charts, geometry, and math.
4. Empirical Validation: Demonstration that aligned latents carry task-relevant visual signals beyond mere token slots.

4. Experimental Results

Evaluated on Qwen2.5-VL 7B, the full GAP model (LH+PCA+DA) was compared against the base model, dense captioning baselines, and prior latent methods (Monet, LVR).

• Performance: GAP achieved the best mean aggregate performance across perception (HRBench4K, MMStar, MME-RealWorld-Lite) and reasoning (MathVista, WeMath) benchmarks.
  • It improved the reasoning aggregate (Avg-R) by +5.98 over Monet and +6.31 over LVR.
  • It improved the perception aggregate (Avg-P) by +1.74 over Monet.
• Ablation Studies:
  • PCA Alignment: Essential for recovering reasoning accuracy lost by naive latent heads.
  • Difficulty-Aware Assignment: Crucial for preventing performance degradation on easy examples.
  • Token Budget: Performance peaks at 36 latent tokens (6x6 grid); increasing further yields diminishing returns or saturation.
• Intervention Probing:
  • Zero-Latent Checkpoint: Even without generating latents at inference, the model trained with latent supervision outperformed dense captioning, suggesting the training process itself shapes better representations.
  • Noise Injection: Replacing predicted PCA coefficients with Gaussian noise significantly degraded performance, confirming that the generated latents contain task-relevant visual signals, not just formatting artifacts.

5. Significance and Claims

The paper claims that visual latent reasoning can be made robust and effective in MLLMs by addressing the specific architectural incompatibilities of pre-norm transformers.

• Modest Claims: The authors do not claim to solve hallucination entirely or to be universally applicable to all backbones without further validation. They note that the "difficulty threshold" is heuristic and that the approach relies on the ability to generate meaningful auxiliary images during data construction.
• Core Insight: The primary contribution is the identification of the norm/subspace mismatch as a failure mode and the demonstration that a lightweight, PCA-based reconstruction path can align continuous visual feedback with the model's input distribution, enabling stable and interpretable visual reasoning without external tools.