Decompose, Look, and Reason: Reinforced Latent Reasoning for VLMs

The paper proposes "Decompose, Look, and Reason" (DLR), a reinforced latent reasoning framework that dynamically decomposes queries into textual premises and extracts premise-conditioned continuous visual latents to achieve superior performance and interpretability in complex visual reasoning tasks compared to existing methods.

The Gist

Imagine you are trying to solve a very tricky puzzle, like a complex math problem drawn on a whiteboard or a "spot the difference" game with a crowded room.

If you ask a standard AI (a Vision-Language Model) to solve this, it often tries to do everything in its head at once. It looks at the picture, tries to describe it in words, and then guesses the answer. The problem is, words are a bad translator for pictures. When the AI turns a visual detail into a sentence, it loses the nuance. It's like trying to describe a delicious pizza to someone over the phone; you can say "it has cheese and pepperoni," but you can't convey the smell, the texture, or exactly how the pepperoni is arranged.

This paper introduces a new method called DLR (Decompose, Look, and Reason) to fix this. Think of DLR not as a single brain trying to do everything, but as a team of three specialists working together.

The Three Specialists: Decompose, Look, and Reason

1. The Decomposer (The Project Manager):
   Instead of staring at the whole messy image and panicking, this specialist breaks the big question down into tiny, manageable steps.

   • Analogy: Imagine you are looking for a specific red sock in a giant, messy laundry pile. The Decomposer doesn't say, "Find the sock!" Instead, it says, "Okay, step one: Look only at the pile of clothes on the left side. Step two: Ignore the blue shirts, look only for red." It creates a checklist.

2. The Looker (The Detective with a Magic Lens):
   This is the most unique part. Previous AI methods either tried to "crop" the image (like taking a photo of just one part) or just guessed. The Looker uses a Magic Lens that doesn't just cut the image; it creates a "mental snapshot" (a latent embedding) of exactly what the Decomposer asked for.

   • Analogy: If the Decomposer says, "Look for the red sock," the Looker doesn't just zoom in randomly. It uses a special filter that highlights only the red textures and ignores the blue jeans or the white towels. It captures the "essence" of the red sock without needing to cut the picture out. It's like having a superpower to instantly focus your eyes on exactly what matters, ignoring the rest of the room.

3. The Reasoner (The Detective Writing the Report):
   Now that the Looker has found the specific evidence, the Reasoner writes down the logic. Because the Looker gave it a perfect, focused "mental snapshot," the Reasoner can say, "I see the red sock is under the blue shirt," and deduce the answer with confidence.

The Secret Sauce: "Reinforced Latent Reasoning"

How do we teach this team to work so well? The authors used a three-stage training camp:

• Stage 1: The Warm-up (Pretraining): They teach the "Looker" how to match words to pictures. It's like teaching a dog to sit when you say "sit." They make sure the AI understands that the word "red" actually connects to the visual idea of red.
• Stage 2: The Classroom (Supervised Fine-Tuning): They show the team examples of how to break down a problem and find the answer. The AI learns the format: "First, make a checklist. Second, look. Third, write the answer."
• Stage 3: The Gym (Reinforcement Learning): This is the big innovation. In the classroom, the AI just copies the teacher. But in the real world, the teacher might be wrong, or the problem might be new.
  • The authors introduced a Spherical Gaussian Latent Policy. This is a fancy way of saying they gave the "Looker" a controlled way to be creative.
  • Analogy: Imagine the "Looker" is a dart player. In the classroom, it just throws darts at a fixed spot. In the Gym, the AI is allowed to throw darts slightly off-center to see what happens. If it hits a bullseye (gets the right answer), it gets a treat. If it misses, it learns not to throw that way again. This allows the AI to explore different ways of looking at the image, rather than just sticking to one rigid way.

Why is this better than what we have now?

• Old Way (Text-only CoT): The AI talks to itself in circles. "Is it red? Maybe. Is it blue? Maybe." It gets confused and gives a wrong answer after writing a huge paragraph.
• Old Way (Image Editing): Some AI tries to draw boxes or zoom in on the image. This is slow and requires extra tools (like a camera app).
• DLR (The New Way): It's fast, internal, and precise. It doesn't need to edit the image or write a novel. It breaks the problem down, focuses its "eyes" exactly where needed, and solves it.

The Result

When they tested this on hard math puzzles, visual logic games, and complex image questions, DLR won. It beat the best existing models, even those that are huge and expensive.

In summary:
This paper teaches AI to stop trying to "guess" the whole picture at once. Instead, it teaches the AI to break the problem down, focus its attention like a laser, and then solve it step-by-step. It's the difference between a student frantically guessing answers and a detective methodically gathering evidence to solve a case.

Technical Summary

1. Problem Statement

Vision-Language Models (VLMs) struggle with complex visual reasoning tasks due to visual information loss when relying solely on textual Chain-of-Thought (CoT). Existing approaches suffer from specific limitations:

• Text-only CoT: Translates visual inputs into text, inevitably losing critical visual details.
• Interleaved MCoT / "Thinking with Images": Relies on localized visual signals (e.g., cropped patches, bounding boxes) or external tool calls. These methods often over-include irrelevant context within a patch or under-include non-local/global relationships (e.g., dominant color, cross-patch logic). They also incur high computational costs due to external tool dependencies.
• Existing Latent Reasoning: Typically inserts a single continuous visual embedding once or relies on Region-of-Interest (ROI) supervision. This fails to support multi-step reasoning where different reasoning steps require attending to different, dynamic regions of the image.

2. Methodology: The DLR Framework

The authors propose "Decompose, Look, and Reason" (DLR), a reinforced latent reasoning framework that mimics human cognitive processes. It dynamically interleaves textual decomposition with continuous visual latent extraction.

Core Architecture

The framework consists of two learnable components:

1. VLM Policy ($P_\theta$): Generates discrete text tokens (premises, rationales, answers).
2. Latent Visual Grounder ($P_\phi$): Generates continuous latent embeddings ($z$) conditioned on the textual premise.

The reasoning trajectory $\tau$ follows a three-step iterative loop:

1. Decompose: The VLM generates a textual premise ($p$) (a sub-question or hypothesis) that needs visual verification, enclosed in <premise> tags.
2. Look: The Visual Grounder attends to the image features conditioned on the hidden state of the premise. It extracts premise-conditioned continuous visual latents ($z$) enclosed in <vis_thought> tags. Unlike patch-based methods, these latents capture both localized and non-local semantic information.
3. Reason: The VLM generates a textual rationale ($r$) grounded in the injected visual latents ($z$) to deduce the final answer.

Three-Stage Training Pipeline

To train this dynamic system effectively, the authors propose a progressive pipeline:

• Stage I: Pretraining (Cross-Modal Alignment)

  • Goal: Establish alignment between the continuous visual space and discrete text space.
  • Method: A lightweight visual grounder is trained using InfoNCE contrastive loss. It learns to extract visual evidence ($z$) that aligns semantically with the target answer embedding ($h_a$) in the latent space.

• Stage II: Supervised Fine-Tuning (SFT)

  • Goal: Teach the model the structured DLR format (Decompose $\to$ Look $\to$ Reason).
  • Method: The model is trained on annotated data where it learns to generate premises and rationales. The visual grounder acts as a deterministic feature extractor to maximize the likelihood of the ground-truth rationale.
  • Limitation: SFT relies on teacher forcing, which bounds the visual grounder to deterministic extraction without active exploration.

• Stage III: Reinforcement Learning (RL) with SGLP

  • Goal: Break the deterministic bottleneck of SFT and enable active exploration in the continuous latent space.
  • Innovation: Spherical Gaussian Latent Policy (SGLP):
    • Standard Gaussian noise in Euclidean space is geometrically mismatched for vision-language features, which lie on a hyperspherical manifold (where direction encodes semantics, not magnitude).
    • SGLP predicts an $L_2$-normalized mean direction ($\mu_\phi$) and injects isotropic noise, then re-projects the sample back onto the unit hypersphere: $z = \frac{\mu_\phi + \epsilon}{\|\mu_\phi + \epsilon\|_2}$. This prevents magnitude collapse and ensures exploration occurs in the angular (semantic) space.
  • Optimization: Uses a modified GRPO (Group Relative Policy Optimization) to jointly optimize the text policy and the latent policy.
  • Reward Design:
    1. Outcome Reward: Binary reward for correct final answers.
    2. Focus Reward ($R_{focus}$): Encourages the visual grounder's attention map to align with an "oracle" attention map (derived from a strong frozen VLM) conditioned on the premise. This reward is only active if the outcome is correct, preventing the reinforcement of hallucinated attention patterns.

3. Key Contributions

1. DLR Framework: A novel architecture that dynamically decomposes queries into premises and extracts premise-conditioned continuous visual latents. This allows for multi-step, iterative visual verification without relying on fixed patches or external tools.
2. Spherical Gaussian Latent Policy (SGLP): A geometrically aware policy optimization method that aligns with the hyperspherical nature of vision-language representations, enabling effective latent exploration without magnitude collapse.
3. Three-Stage Training Pipeline: A comprehensive strategy (Pretraining $\to$ SFT $\to$ RL) that bridges the gap between cross-modal alignment, structured reasoning, and active latent exploration.

4. Experimental Results

The authors evaluated DLR on four vision-centric benchmarks: V Bench* (visual detail), MathVista (math reasoning), MMMU-Pro (multidisciplinary reasoning), and MMStar (general multimodal).

• Performance: DLR consistently outperformed strong baselines, including:
  • Text-only baselines (Qwen3-VL-8B-Thinking).
  • Interleaved MCoT methods (ICoT).
  • "Thinking with images" methods (PixelReasoner).
  • Existing latent reasoning methods (LVR).
  • Notable: DLR surpassed the proprietary GPT-4o (approx. 200B parameters) on several benchmarks.
• Key Findings:
  • MathVista: DLR showed significant gains (+5.0% over backbone), proving its ability to handle multi-step diagram inspection better than single-pass latent methods.
  • Interpretability: Case studies showed DLR avoids the "overthinking" and hallucination seen in text-only baselines (e.g., generating 15k+ tokens with incorrect logic) by grounding each step in specific visual evidence.
  • Ablation: Removing the RL stage (SFT only) or the SGLP policy caused significant performance drops, confirming the necessity of active latent exploration.

5. Significance

• Efficiency & Effectiveness: DLR achieves state-of-the-art performance without the computational overhead of external tool calls or image editing, utilizing internal latent spaces instead.
• Stepwise Interpretability: By explicitly linking textual premises to visual latents, the model provides a transparent reasoning path where each step can be verified against specific image regions.
• Theoretical Advancement: The introduction of SGLP addresses a critical geometric mismatch in latent space reasoning, offering a new paradigm for optimizing continuous variables in RL for multimodal tasks.
• Human-Cognitive Alignment: The "Decompose $\to$ Look $\to$ Reason" loop mirrors human problem-solving strategies, suggesting a more robust path for future VLM development.