💬 NLP

Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment

This paper introduces Perception-RFT, a training framework that applies Group Relative Policy Optimization to multimodal document QA without intermediate reasoning tokens, demonstrating that direct visual grounding alignment outperforms reasoning-centric approaches by reducing inference costs by over 60% while avoiding the "Grounding Divergence" trade-off between semantic robustness and geometric precision.

Original authors: Harikrishnan P M, Goutham Vignesh, Ganesh Parab, Saisubramaniam Gopalakrishnan, Vishal Vaddina, Varun V, Rohit Agrawal

Published 2026-07-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Harikrishnan P M, Goutham Vignesh, Ganesh Parab, Saisubramaniam Gopalakrishnan, Vishal Vaddina, Varun V, Rohit Agrawal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to read a messy, handwritten receipt and point its finger exactly at the total price. This isn't just about the robot saying the right number; it's about the robot physically locating that number on the paper. This field is called "Multimodal Document Question Answering." Think of "multimodal" as the robot using two senses at once: eyes (to see the image of the document) and a brain (to understand the text). For a long time, scientists have been stuck on a big question: To get the robot to point at the right spot, does it need to "think" out loud first? Should it say, "Okay, I see a dollar sign, then a number, so the total must be here," before pointing? Or is that extra chatter just slowing it down? This matters because in the real world—like when a bank checks a loan application or a lawyer audits a contract—speed and accuracy are everything. If the robot wastes time "thinking" or points at the wrong spot, it could cost people money or miss a crucial detail.

The authors of this paper decided to test a bold idea: What if we tell the robot to stop thinking and just look? They built a training system they call "Perception-RFT." Instead of letting the robot generate a long, wordy explanation (like a student showing their work on a math test), they forced it to jump straight to the answer and the coordinates of where that answer lives on the page. They used a special training method called Group Relative Policy Optimization (GRPO), which is like a coach giving a team of robots a group of answers and saying, "You did better than the others, so keep doing that," without needing to explain why step-by-step.

Here is the surprising twist they found: The robots didn't need the "thinking" part at all. In fact, when the researchers did let the robots try to think out loud, the robots eventually stopped doing it on their own. During training, the models started generating long reasoning chains, but as they got better at the task, they compressed those thoughts down until they were almost gone. By the end, the robots that were allowed to think actually performed worse than the ones that were forced to just "look" and point immediately. The "thinking" robots were slower, used more computer power, and made more mistakes. The paper suggests that for this specific job—finding a spot on a document—direct perception is a superpower, and adding a "reasoning" step is just a distraction.

The researchers also discovered something tricky called "Grounding Divergence." When they tested their super-trained robots on documents they had never seen before (like switching from financial receipts to scientific charts), the robots got really good at pointing at the right spot, but they sometimes got a little worse at understanding the words. It's like a student who gets so good at finding the answer key on a page that they stop reading the question carefully. However, the paper notes that for most practical uses, getting the pointer right is the most important thing.

Finally, they found a shortcut. Usually, you have to teach a robot with thousands of examples before you can start the fancy training. But this team showed that if you switch to this "just look" training method very early—after teaching it with only a tiny fraction of the usual data—the robot learns just as well. They managed to get the same high-quality results using 65% less training data. So, the main takeaway is simple: For reading documents and pointing at answers, stop overthinking. The best way to teach a machine is to make it look, not to make it talk.

Technical Summary: Stop Thinking, Start Looking

Problem Statement

Multimodal Document Question Answering (DocQA) requires not only generating correct text but also performing Document Visual Grounding (DVG): the precise localization of the visual region supporting an answer. Current approaches face two primary limitations:

Supervised Fine-Tuning (SFT): While effective, SFT often reaches an optimization plateau where further scaling of annotated datasets yields diminishing returns on geometric precision.
Reasoning-Centric Reinforcement Learning (RL): Recent paradigms (e.g., DocThinker) posit that explicit intermediate reasoning traces (Chain-of-Thought) are necessary to bridge semantic understanding and spatial localization. However, these verbose traces inflate inference token costs and introduce latency without clear empirical benefit for inherently perceptual tasks.

Furthermore, the dynamics of RL in multimodal settings remain underexplored, particularly regarding whether reasoning is truly necessary for grounding or if it introduces instability and unnecessary computational overhead.

Methodology: Perception-RFT

The authors introduce Perception-RFT, a training framework designed to align visual features directly with structured grounding outputs, bypassing intermediate reasoning tokens.

Core Framework

Task Formulation: The task is defined as a generative Visual QA problem where the policy $\pi_\theta$ outputs a structured JSON containing the answer and a normalized bounding box ( $[x_1, y_1, x_2, y_2]$ ).
Direct Perception Constraint: Unlike standard RL approaches that encourage verbose reasoning, Perception-RFT enforces a system prompt explicitly forbidding explanations or intermediate steps ("Do NOT include explanations... Output ONLY the final valid JSON"). This forces the model to map visual features directly to coordinate regression.
Optimization Algorithm: The framework utilizes Group Relative Policy Optimization (GRPO). Instead of maximizing logical consistency via reasoning chains, the objective maximizes the advantage of valid outputs based on a joint reward signal, constraining policy drift via KL divergence.

Gated Dense Reward Mechanism

To stabilize training without intermediate guidance, the authors propose a composite reward function $R = R_{format} + R_{sem} + R_{loc}$ :

Format Consistency ( $R_{format}$ ): A binary switch (0 or 1) ensuring the output is valid JSON with the correct schema.
Semantic Precision ( $R_{sem}$ ): Uses Average Normalized Levenshtein Similarity (ANLS) with quadratic scaling to aggressively penalize near-misses.
Gated Localization Reward ( $R_{loc}$ ): A "cliff-edge" mechanism.
- Shaping State: If IoU is below a threshold ( $\tau=0.6$ ), the reward is scaled down ( $\lambda \cdot \text{IoU}$ ) to prevent convergence on suboptimal partial overlaps.
- Success State: If IoU $\geq 0.6$ , the reward jumps to 1.0, encouraging a geometric safety margin.

Experimental Design

The study employs Qwen3-VL-4B and compares several configurations:

SFT: Standard supervised fine-tuning on ~23k financial document samples.
RFTs (SFT $\to$ RL): Perception-RFT applied after SFT initialization.
RFTb (Cold-Start RL): Perception-RFT applied directly to the base model without SFT.
Reasoning-RFTb: A controlled variant allowing intermediate reasoning traces under identical RL conditions to isolate the role of reasoning.
Datasets: Training on curated financial documents (DocILE, FormNLU) and evaluation on In-Distribution (ID) sets and two Out-of-Distribution (OOD) benchmarks: DOGR-Bench (800 samples) and MMDocBench (4,028 samples).

Key Results

1. Reasoning is Redundant and Detrimental

The most significant finding is that explicit reasoning does not improve performance in DVG tasks under RL.

Performance Gap: The reasoning-free variant (RFTb) consistently outperforms the reasoning-enabled variant (Reasoning-RFTb) on both ID and OOD datasets. For instance, on OOD data, RFTb achieves an F1all of 0.600 compared to 0.382 for Reasoning-RFTb.
Trace Compression: Even when reasoning is enabled, models progressively compress and eliminate reasoning traces during training. At convergence, the average reasoning token length drops by 62% (from ~191 to ~72 tokens), effectively converging to a direct perception policy.
Stability: Reasoning-enabled training exhibits higher variance and less stable convergence compared to the direct perception approach.

2. RL Surpasses SFT Saturation

Breaking the Ceiling: On In-Distribution data, SFT reaches a performance plateau (F1all = 0.668). Applying Perception-RFT (RFTs) breaks this ceiling, improving F1all to 0.718, with significant gains in both text extraction and localization.
Importance of Initialization: Cold-start RL (RFTb) improves over the base model but remains substantially below RFTs, confirming that supervised initialization provides a stable foundation for geometric learning under RL.

3. The Grounding Divergence Phenomenon

The authors identify a previously uncharacterized trade-off termed Grounding Divergence. Under joint RL optimization on OOD data:

Localization Improves: Geometric precision transfers robustly to unseen domains (e.g., DOGR-Bench localization jumps from 0.416 to 0.759).
Semantic Robustness Degrades: Semantic extraction capabilities often fail to transfer or degrade compared to the zero-shot base model.
Implication: RL induces asymmetric generalization; it excels at learning transferable geometric patterns but may over-specialize semantic extraction, leading to a selective trade-off between geometric precision and semantic flexibility.

4. Data Efficiency

An early transition from SFT to RL (RFTs-Early) achieves comparable precision to full SFT+RL while using 65% less training data. This suggests that prolonged supervised learning is not strictly necessary for high-precision grounding if RL is introduced early.

Significance and Claims

The paper claims to establish that Document Visual Grounding is a perception-dominant task where direct alignment between visual features and spatial outputs is more effective than reasoning-mediated generation.

Efficiency: By bypassing reasoning tokens, the approach reduces per-query inference token length by over 60%, offering a more cost-effective and faster deployment strategy.
Optimization Dynamics: The work extends findings from text-domain post-training (SFT saturation and cold-start RL instability) to the multimodal domain and identifies the specific "Grounding Divergence" trade-off.
Practical Guidance: The authors argue that for DVG, models should "Stop Thinking, Start Looking." The optimal policy relies on direct perception rather than explicit reasoning, and early SFT-to-RL transitions offer the best balance of data efficiency and performance.

The study is modest in its scope, noting that these findings are established at the 4B parameter scale and that scaling behavior to larger models remains a subject for future work. Additionally, the lack of reasoning-annotated supervised data limits the ability to construct a reasoning-enabled SFT $\to$ RL pipeline for further comparison.