💬 NLP

Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs

This paper proposes a training-free framework that enhances spatial reasoning in Multimodal Large Language Models by constructing a Spatial Evidence Graph to verify the faithfulness of reasoning chains against visual evidence, identify contradictions, and guide the model to correct errors, thereby significantly improving accuracy across diverse benchmarks.

Original authors: Yang Yang, Jiawei Chen, Tairan Chen, Zhaoxia Yin

Published 2026-08-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yang Yang, Jiawei Chen, Tairan Chen, Zhaoxia Yin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a computer that can look at a photograph and answer questions about it, much like a human observer. These systems, known as multimodal large language models, have become remarkably skilled at describing scenes, identifying objects, and even solving puzzles. However, when asked to reason about space—determining if one object is to the left of another, or if a cup is sitting behind a book—they often stumble. The problem is not always that the computer cannot see the image; rather, it is that the computer's internal thought process, the step-by-step logic it uses to reach an answer, can drift away from what is actually visible. If the system makes a small mistake early in its reasoning, such as misidentifying the position of a single item, that error can snowball. The computer might treat this incorrect observation as a fact, use it to build further conclusions, and eventually arrive at a final answer that is confidently wrong, even though the image clearly shows otherwise.

Researchers have long tried to fix this by teaching these models more about space or by adding extra layers of data to help them see better. But a new study suggests that the most effective solution lies not in teaching the model new facts, but in checking its work as it happens. The team behind this research, led by Yang Yang and colleagues, developed a method to verify the computer's reasoning against the image in real time, without needing to retrain the model or change its underlying code. They found that when a model's reasoning chain is faithful to the visual evidence, its answers are significantly more accurate. In fact, on a standard test of real-world questions, models that stuck to the visual facts got the right answer about 51 percent of the time, while those that drifted into unsupported guesses only succeeded about 34 percent of the time.

To solve the problem of drifting logic, the researchers created a framework that acts like a rigorous editor for the computer's thoughts. When the model generates a chain of reasoning, the system breaks that text down into tiny, atomic claims about the image, such as "the apple is present" or "the bowl is to the right of the plate." It then builds a structured map, which they call a Spatial Evidence Graph, linking each of these claims to the specific part of the image it refers to. This map allows the system to trace every statement back to its source, ensuring that no piece of reasoning is floating in a vacuum.

The next step is the most critical: checking the reliability of the visual evidence supporting each claim. The system does not blindly trust its own detection tools. Instead, it assesses whether the object is clearly visible, whether its location is unambiguous, and whether measurements like depth are stable. If the system detects that an object is partially hidden or the image is too blurry to be sure, it marks that piece of evidence as uncertain rather than forcing a decision. This prevents the computer from making a confident correction based on shaky data. The system then scans the chain of reasoning from the very beginning to find the first point where a claim contradicts the reliable visual evidence.

Once this earliest error is located, the system guides the model to rewrite its reasoning starting from that specific mistake. It tells the model, "You said the apple was to the right of the bowl, but the visual evidence shows the apple is actually to the left. Please correct this step and update your conclusion." By fixing the root cause of the error and regenerating the subsequent steps, the model avoids the cascade of mistakes that would have led to a wrong answer. This process is entirely training-free, meaning it works with existing models without requiring them to learn new skills or access new data sets.

The results of this approach were tested across fifteen different combinations of models and datasets, covering a wide range of visual reasoning tasks. The method improved the average accuracy of the models to nearly 69 percent, outperforming previous state-of-the-art techniques by a significant margin. More importantly, the study showed that the models became much more consistent in their reasoning. The fraction of their intermediate steps that were faithful to the image increased dramatically, proving that the method successfully stopped errors from spreading. The researchers also found that this process-oriented correction works well alongside other methods that try to enhance spatial awareness, suggesting that verifying the logic is just as important as providing better tools for seeing.

While the system is highly effective, the researchers acknowledge its limits. It can only correct errors that are explicitly written out in the model's reasoning steps. If the model makes a mistake that remains hidden or implicit within its internal processing, the system cannot find it to fix it. Furthermore, the current tests were conducted on static images, so it remains to be seen how well this approach handles moving video or complex, multi-view scenarios. Nevertheless, the study offers a clear path forward: by treating the reasoning process as something that can be audited and corrected in real time, we can make these powerful artificial intelligence systems more reliable and trustworthy, ensuring that their answers are grounded in the reality of what they see.

Technical Summary: Trace, Verify, and Correct

Problem Statement

Multimodal Large Language Models (MLLMs) have advanced significantly in visual question answering and reasoning but continue to struggle with spatial reasoning. A critical failure mode identified in this work is perceptual unfaithfulness: intermediate spatial judgments within a Chain-of-Thought (CoT) reasoning chain that are inconsistent with the input image. When MLLMs generate such unsupported intermediate steps, these errors propagate through the reasoning chain, leading to incorrect final answers.

Existing approaches to improve spatial reasoning generally fall into two categories:

Training-stage enhancements: These methods use spatial QA data or introduce spatial priors but require parameter access and task-specific adaptation, making them difficult to apply to new models or tasks without retraining.
Inference-stage augmentations: These methods introduce depth cues, 3D priors, or structured scene representations at inference time. However, they remain outcome-oriented, enhancing the input or generation process without explicitly verifying or correcting intermediate spatial evidence. Consequently, local reasoning errors often remain undetected.

The authors argue that improving final-answer accuracy requires a process-oriented strategy that verifies and corrects intermediate spatial evidence before the final answer is generated.

Methodology

The paper proposes Trace, Verify, and Correct, a modular, training-free framework designed to ensure perceptual faithfulness in spatial reasoning. The framework operates in three primary stages:

1. Spatial Evidence Graph (SEG) Construction

To address the difficulty of extracting and tracing spatial judgments in free-form CoT, the framework converts natural language reasoning into a structured Spatial Evidence Graph (SEG).

Extraction: An MLLM is prompted to extract atomic spatial evidence units from each reasoning step. These units cover four types: object existence, absolute image position, 2D relative spatial relations, and relative depth.
Structure: Each unit is represented as a tuple $s_m = \langle o_i, r_m, o_j, t_m, k_m \rangle$ , linking visual entities ( $o$ ), spatial relations ( $r$ ), evidence type ( $t$ ), and the source reasoning step ( $k$ ).
Binding: The graph associates these units with visual evidence required for verification, such as bounding boxes from Grounding DINO and depth masks from SAM2 combined with a monocular depth estimator.

2. Reliability-Aware Spatial Verification (SERA)

Recognizing that visual evidence (e.g., object detection, depth estimation) can be uncertain due to occlusion or clutter, the framework employs SERA to assess the reliability of verification evidence before making correction decisions.

Reliability Assessment: For each visual entity, SERA evaluates three components:
- Existence Reliability ( $R_E$ ): Based on the confidence of object presence.
- Localization Reliability ( $R_L$ ): Based on the confidence of the top grounding score and the margin over the second-best candidate.
- Measurement Reliability ( $R_M$ ): Based on mask area sufficiency and depth consistency (for depth relations).
Aggregation: These components are aggregated into a reliability score $g(o)$ . Entities are categorized into HIGH, MEDIUM, or LOW reliability tiers.
Verification Logic: Only evidence supported by HIGH reliability entities is used to make decisive SUPPORTED or CONTRADICTED judgments. Units involving uncertain or low-reliability evidence are marked as INCONCLUSIVE to prevent introducing new errors during correction.

3. Targeted Revision and Regeneration

The framework identifies the earliest spatial evidence unit in the reasoning chain that is contradicted by reliable visual evidence.

Verification Report: A report is generated containing the contradictory unit, its source step, and the precomputed visual judgments (existence, layout, depth).
Regeneration: The original MLLM is guided to revise the reasoning starting from the source step of the earliest contradiction, preserving correct prior steps and regenerating the subsequent reasoning and final answer. If no reliable contradiction is found, the original answer is retained.

Key Contributions

Process-Oriented Framework: The authors propose a training-free framework that shifts the focus from outcome enhancement to evidence-level verification and correction, explicitly locating and fixing perceptually unfaithful reasoning.
SEG and SERA Modules: The development of the Spatial Evidence Graph for structured provenance tracing and SERA for hierarchical, reliability-aware verification of visual evidence.
Empirical Validation: The framework is model-agnostic and consistently improves performance across multiple MLLMs (Qwen, InternVL, Llama) and five diverse benchmarks (RealWorldQA, GQA, POPE, etc.).

Experimental Results

The method was evaluated across 15 model-dataset settings. Key findings include:

Accuracy Improvement: The framework achieved an average accuracy of 68.94%, outperforming compared baselines by 8.55 percentage points on average.
Faithfulness Correlation: Analysis showed that "Fully Faithful" CoTs (where all evidence is supported) achieved 51.10% accuracy on RealWorldQA, whereas "Unfaithful" CoTs achieved only 34.41%, demonstrating a strong statistical link between faithfulness and final accuracy.
Error Accumulation: The study found that spatial unfaithfulness accumulates along the reasoning chain; as the chain progresses, the cumulative faithfulness of evidence decreases.
Ablation Studies: Removing either the SEG (structured tracing) or SERA (reliability gating) resulted in significant performance drops, confirming the necessity of both components.
Efficiency: While the method incurs higher inference latency (approx. 2.8x Vanilla) due to the verification and regeneration steps, it does not require parameter updates or fine-tuning.

Significance and Claims

The paper claims that perceptual faithfulness is not merely a metric for evaluating reasoning quality but a practical lever for improving final-answer accuracy. By explicitly verifying intermediate spatial evidence against reliable visual data and correcting the earliest detected errors, the framework prevents error propagation without requiring model retraining.

The authors position their work as a complement to existing inference-time augmentation methods (like SpatialPIN), noting that combining their correction framework with spatial augmentation yields further performance gains. They conclude that evidence-level verification provides an effective, generalizable approach to enhancing spatial reasoning in MLLMs, particularly in high-stakes domains where incorrect spatial judgments could lead to safety risks.

Limitations Acknowledged:
The framework relies on explicit CoT generation; it cannot correct spatial errors that remain implicit or are omitted from the reasoning chain. Furthermore, current experiments are limited to static image QA and open-source models, with effectiveness in video understanding, multi-view reasoning, or closed-loop embodied interaction remaining to be investigated.