💬 NLP

SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

SpatialThinker is a novel multimodal large language model that unifies Scene Graph Generation and visual reasoning within a single pass using online reinforcement learning with dense spatial rewards, achieving state-of-the-art performance on spatial benchmarks with limited training data.

Original authors: Hunar Batra, Haoqin Tu, Hardy Chen, Yuanze Lin, Cihang Xie, Ronald Clark

Published 2026-07-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hunar Batra, Haoqin Tu, Hardy Chen, Yuanze Lin, Cihang Xie, Ronald Clark

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: AI is "Blind" to Space

Imagine you show a picture of a messy desk to a very smart but slightly clumsy robot. You ask, "Is the red light directly above the laptop?"

Current AI models (like the ones in the paper's examples) often guess based on general vibes. They might say, "Well, lights are usually above things, so yes," or they might get confused about left and right. They struggle to understand where things are in 3D space relative to each other, even if they can describe the objects perfectly. They are like someone who knows the names of all the players on a soccer team but can't tell you who is standing in front of whom.

The Solution: "SpatialThinker"

The researchers built a new AI called SpatialThinker. Think of this AI not just as a chatbot, but as a cartographer who draws a map before answering a question.

Instead of just looking at the picture and guessing, SpatialThinker follows a strict, four-step process every time it sees an image:

Observe: It looks at the picture.
Map It (The Scene Graph): It draws a mental "connect-the-dots" map. It identifies objects (like "laptop," "light") and draws lines between them with labels like "above," "left of," or "behind."
Think: It uses this map to reason through the question.
Answer: It gives the final answer based on the map, not a guess.

The Secret Sauce: "Dense Rewards" (The GPS Trainer)

How did they teach the AI to draw these maps correctly? They didn't just show it thousands of pictures and say "Good job" when it got the final answer right. That's like teaching a driver by only telling them "You arrived!" at the end of a trip, without correcting them if they drove the wrong way down the street.

Instead, they used Dense Rewards, which is like a GPS trainer that gives constant feedback:

Format Reward: "Did you draw the map in the right format?" (Yes/No)
Count Reward: "Did you count the right number of objects?" (If you say there are 3 chairs but there are only 2, you lose points).
Accuracy Reward: "Did you get the final answer right?"
Spatial Reward: "Did you place the objects in the right spots on your map?"

The AI gets points for every step of the process being correct, not just the final answer. This forces it to learn the "where" and "how" of the world, not just the "what."

The Training Data: A Tiny, High-Quality Library

Most AI models need to read millions of books (or look at millions of images) to learn. SpatialThinker is different.

The researchers created a special dataset called STVQA-7K.
It only has 7,000 examples (a drop in the bucket compared to the billions other models use).
However, every single example is high-quality and verified by two different AI systems to ensure the "maps" are perfect.

The Analogy: Imagine teaching a chess player. Instead of making them play 1,000,000 random games, you give them 7,000 perfectly analyzed master games where every move is explained. They learn the principles of the game much faster and better than someone who just played millions of sloppy games.

The Results: Small Data, Big Brains

The paper claims some impressive results:

Beating the Giants: The 7-billion-parameter version of SpatialThinker (a relatively small model) performed as well as or better than massive, proprietary models like GPT-5 and GPT-4o on spatial tasks.
The 30-Billion Powerhouse: The larger version (30B) beat GPT-5 and Claude 4 Sonnet on average across 14 different tests.
Generalization: Because it learned the structure of space (the "map" concept) rather than just memorizing patterns, it got better at general real-world questions too, not just the specific training questions. It reduced "hallucinations" (making things up) because it had to point to specific spots on its map to justify its answer.

Summary

SpatialThinker is an AI that learns to "see" space by forcing itself to draw a mental map of relationships between objects before answering a question. By using a strict "GPS-style" training system that rewards every correct step of the mapping process, it learned to understand 3D space and object locations using only a tiny fraction of the data other models need, outperforming much larger and more expensive AI systems.

Technical Summary: SpatialThinker

Problem Statement

Multimodal Large Language Models (MLLMs) have achieved significant progress in vision-language tasks but continue to struggle with spatial reasoning, particularly in 3D space. This capability is crucial for embodied AI, navigation, and augmented reality. Existing approaches to improve spatial reasoning typically rely on:

Data-Intensive Pipelines: Generating massive synthetic datasets (hundreds of thousands to millions of samples) from 3D scene graphs or point clouds.
Architectural Modifications: Adding explicit 3D inputs (depth maps, point clouds) or auxiliary spatial tokens.
Sparse Reinforcement Learning (RL): Using simple accuracy-only rewards that provide insufficient guidance for visually-grounded reasoning.

A critical gap exists in integrating Scene Graph Generation (SGG) directly into the reasoning process. Current methods often treat SGG as a disjoint preprocessing step for data curation or as an isolated generation target, failing to use structured scene representations to guide the reasoning chain end-to-end. Furthermore, existing RL methods lack the dense, structured feedback necessary to enforce precise spatial localization and relational understanding.

Methodology: SpatialThinker

The authors propose SpatialThinker, the first MLLM to unify Scene Graph Generation (SGG) and visual reasoning in a single pass via online Reinforcement Learning (RL). The framework simulates human-like spatial perception by constructing a "mental scene graph" of task-relevant objects and relations, then reasoning over this structured representation to derive an answer.

1. Core Architecture and Training

Base Models: Built upon Qwen2.5-VL (3B, 7B) and Qwen3-VL (30B).
Training Paradigm: The model is trained using Group-Relative Policy Optimization (GRPO), an online RL method that avoids critic networks by estimating advantages through intra-group comparisons.
No SFT: The models are trained directly on the RL objective without prior Supervised Fine-Tuning (SFT) on the training dataset.
Inference: The model generates a structured response containing <observe>, <scene> (a JSON-formatted scene subgraph), <thought>, and <answer>.

2. Multi-Objective Dense Reward Design

The core innovation is a dense, lexicographically gated multi-objective reward function ( $R_{total}$ ) that guides the model through a structured reasoning process: observe → localize → reason → answer. The reward comprises four components:

Format Reward ( $R_f$ ): Enforces a structured reasoning template. It validates the JSON inside the <scene> tag, ensuring parseability, the presence of required fields (ID, bounding box), and valid subject-predicate-object triplets.
Count Reward ( $R_c$ ): Penalizes deviations between the predicted and ground-truth counts of objects and relations. This prevents "reward hacking" where models generate excessive objects to maximize random spatial matches.
Accuracy Reward ( $R_a$ ): A binary score based on exact string match with the ground-truth answer. This carries the highest weight ( $w=0.5$ ) to prioritize task correctness.
Spatial Reward ( $R_s$ ): Computed only when the final answer is correct (via lexicographic gating). It uses the Hungarian algorithm to match predicted and ground-truth objects based on Complete IoU (CIoU) and semantic similarity. This provides dense supervision on object localization.

Lexicographic Gating: The rewards are ordered as Format ≻ {Count, Accuracy} ≻ Spatial. The model must satisfy formatting first, then optimize count and accuracy, and only receives the spatial reward if the answer is correct. This ensures spatial grounding reinforces valid reasoning rather than overriding it.

3. Dataset: STVQA-7K

To facilitate reward-aligned training, the authors constructed STVQA-7K, a high-quality spatial VQA dataset:

Source: Derived from human-annotated scene graphs in Visual Genome.
Scale: 7,587 spatially grounded multiple-choice VQA pairs.
Content: Covers nine reasoning types (relations, size, orientation, distance, depth, reach, location, count, existence) and includes 3D-reasoning questions (e.g., depth, orientation).
Quality Control: Uses a dual-LLM verification pipeline (Claude Sonnet 4 generates, GPT-4o validates) to ensure high quality, retaining only the top 7,587 samples from an initial pool of 56,224.
Scalability: The pipeline is capable of generating up to ~108K samples.

Key Contributions

Unified SGG and Reasoning: SpatialThinker is the first MLLM to integrate scene graph construction directly within the reasoning chain via online RL, rather than treating it as a preprocessing step.
STVQA-7K Dataset: A high-quality, scene-graph-grounded spatial VQA dataset with a scalable synthesis pipeline, enabling effective training with only 7K samples.
Dense Spatial Reward Design: A novel multi-objective reward system with lexicographic gating that enforces structured grounding. This design improves both in-distribution and out-of-distribution (OOD) generalization, nearly doubling the performance gains of sparse RL baselines.

Experimental Results

The authors evaluated SpatialThinker across 14 benchmarks (8 spatial, 6 real-world/generic VQA) and compared it against proprietary models (GPT-5, GPT-4o, Claude 4 Sonnet) and open-source baselines.

Performance with Limited Data: Trained on only 7K samples, SpatialThinker-7B matches the performance of GPT-5 and outperforms GPT-4o and Claude 4 Sonnet on average across spatial benchmarks.
Scaling Effects: SpatialThinker-30B surpasses both GPT-5 and Claude 4 Sonnet on average, achieving state-of-the-art results on CV-Bench 3D (93.6%, +3.3% over GPT-5) and 3DSRBench.
Generalization: SpatialThinker demonstrates superior OOD generalization on real-world VQA tasks (e.g., RoboSpatial-Home, HallusionBench).
- Sparse RL vs. Dense RL: While sparse RL (Vanilla GRPO) improves base models by ~4.4%, the dense spatial reward design achieves ~7.7% improvement (a 1.7× gain).
- Real-World Transfer: SpatialThinker-7B achieves +5.3% improvement on real-world benchmarks compared to +1.9% for sparse RL, confirming that structured spatial grounding enhances broad visual perception and reduces hallucinations.
Ablation Studies: Removing the count reward leads to reward hacking (23.7% accuracy). Adding lexicographic gating and RoI filtering recovers performance to 76.3%, and dual-LLM dataset filtering boosts it to 87.9% on the validation set.

Significance and Claims

The paper claims that structured spatial grounding via dense rewards enables robust spatial understanding with limited data, challenging the notion that massive-scale pretraining or explicit 3D inputs are strictly necessary for high-level spatial reasoning.

Efficiency: The model achieves state-of-the-art results using only 7K synthetic samples, a fraction of the data used by existing Spatial MLLMs (which often use millions of samples).
Emergent 3D Reasoning: Despite being trained on 2D RGB images with 2D rewards, the relational scene-graph supervision encodes cues linked to depth and orientation, allowing the model to develop emergent 3D reasoning capabilities.
Generalization: The approach improves not just spatial tasks but also generic visual perception and real-world VQA, suggesting that enforcing structured reasoning reduces hallucinations and sharpens fine-grained visual distinction.

The authors conclude that their method demonstrates that reward-aligned reasoning combined with scene graph grounding is a viable and efficient path to robust spatial understanding in MLLMs.