When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning

1. Problem Statement

Visual Spatial Reasoning (VSR)—the ability to understand relations such as left/right, front/back, overlap, and proximity—remains a significant bottleneck for modern Vision-Language Models (VLMs). While recent multimodal architectures excel at general image-text tasks, they struggle when spatial relations are ambiguous, frame-dependent, or require precise grounding.

A common strategy to mitigate this is information injection at inference time, where models are provided with:

• Explicit spatial cues (e.g., bounding boxes, depth).
• External commonsense knowledge.
• Chain-of-Thought (CoT) reasoning instructions.

The Core Question: It is unclear when such additional information genuinely improves reasoning versus when it introduces noise, redundancy, or cognitive overload. The paper challenges the intuition that "richer input always leads to better reasoning."

2. Methodology

The authors conducted a hypothesis-driven empirical analysis treating VLMs as fixed black-box systems. Instead of proposing new architectures, they systematically varied the type and quantity of injected information to isolate causal effects.

• Models Evaluated: Three representative VLMs with diverse architectures:
  • Qwen-2-VL-7B
  • LLaVA-Next-1.6-7B
  • BLIP-3-5B
• Benchmarks: Two public datasets:
  • VSR: Focuses on fine-grained spatial relations across 66 categories.
  • EmbSpatial: Focuses on reasoning in cluttered, object-dense scenes.
• Intervention Variables:
  1. Spatial Contexts (SC): Extracted via off-the-shelf perception models (Grounding DINO, Segment Anything 2, Depth Anything v2, Orient Anything). Types included:
     • Numerical: Bounding boxes, orientation angles, metric depth values.
     • Verbalized/Interpretable: Lateral (left/right), vertical, orientation, depth (close/far), overlap, and relative size.
  2. Commonsense Knowledge (CK): Retrieved from the ATOMIC2020 knowledge base. Statements were filtered by semantic similarity thresholds (0.1 to 0.9) and categorized into Physical-Entity (PE), Event-Centred (EC), and Social-Interaction (SI).
  3. Spatial Reasoning (SR) Instructions: CoT-style prompts encouraging step-by-step reasoning using the provided context.

3. Key Hypotheses

The study tested three specific hypotheses regarding information injection:

• H1 (Spatial Context): Not all spatial contexts contribute equally. Single, targeted cues should outperform aggregated multi-context inputs due to integration burdens.
• H2 (Commonsense Knowledge): Excessive or weakly relevant commonsense knowledge degrades performance due to information overload. Performance should peak at selective relevance thresholds rather than maximal injection.
• H3 (CoT Prompting): Chain-of-Thought improves reasoning only when spatial grounding is sufficiently precise and unambiguous. In ambiguous or frame-dependent scenarios, CoT may amplify errors.

4. Key Results & Findings

A. The "More is Less" Phenomenon

Across all models and datasets, injecting more information did not consistently lead to better reasoning. In many cases, adding multiple signals (SC + CK + SR) resulted in lower accuracy than the baseline or single interventions, indicating cognitive overload.

B. Analysis of Spatial Contexts (Supporting H1)

• Targeted vs. Aggregated: A single, task-relevant spatial cue yielded the largest gains. As the number of injected contexts increased (from 1 to 9), performance peaked at 1 context and generally declined.
• Representation Matters: Verbalized spatial contexts (e.g., "left of," "close") significantly outperformed numerical counterparts (e.g., raw depth values, angle degrees). Numerical inputs introduced integration complexity, causing performance drops of up to 9.2%.
• Model Sensitivity: Different models benefited from different context types (e.g., Qwen favored depth cues; LLaVA favored orientation; BLIP favored lateral relations), suggesting no universal "best" context.

C. Analysis of Commonsense Knowledge (Supporting H2)

• Relevance Thresholds: Performance peaked at high similarity thresholds (0.7–0.9), where only highly relevant knowledge was injected.
• Noise Effect: Lower thresholds (0.1–0.4) admitted loosely related knowledge, which acted as noise, causing accuracy to drop below zero-shot baselines.
• Model Variance: While Qwen and BLIP benefited from selective CK, LLaVA showed slight degradation even with injected knowledge, highlighting model-specific sensitivities to external data.

D. Analysis of Chain-of-Thought (Supporting H3)

• Grounding Dependency: CoT prompting improved accuracy significantly (up to +12.2%) on EmbSpatial (cluttered scenes) where premises were clear. However, it degraded performance (up to -2.0%) on VSR tasks involving ambiguous frames of reference.
• Error Amplification: In cases where the frame of reference was unspecified (e.g., "left" relative to the camera vs. the object), CoT forced the model to construct a coherent but incorrect reasoning chain, reinforcing the error rather than correcting it.

5. Significance and Contributions

• Paradigm Shift: The paper challenges the prevailing assumption that "more context equals better reasoning." It demonstrates that indiscriminate context accumulation leads to diminishing returns and performance degradation.
• Practical Guidelines:
  • Selectivity: Information injection must be task-aligned. A single, high-quality, verbalized spatial cue is superior to a bundle of raw data.
  • Thresholding: Commonsense knowledge must be filtered strictly by relevance to avoid overload.
  • Conditional CoT: Chain-of-Thought should only be applied when spatial grounding is unambiguous; otherwise, it risks amplifying hallucinations.
• Design Implications: Future VLM pipelines should prioritize controlled, selective information injection over the accumulation of external signals. This highlights the need for better mechanisms to filter and format external data before it reaches the reasoning engine.

Conclusion

The study concludes that effective Visual Spatial Reasoning in VLMs relies on precision and selectivity rather than volume. The most robust performance is achieved not by feeding the model everything it might need, but by providing exactly what it needs in a format it can reliably integrate.