🤖 AI

Spatial Atlas: Compute-Grounded Reasoning for Spatial-Aware Research Agent Benchmarks

This paper introduces Spatial Atlas, a research agent framework utilizing compute-grounded reasoning to deterministically resolve spatial sub-problems via structured scene graphs before language model generation, thereby achieving competitive accuracy and interpretability on the multimodal FieldWorkArena and ML engineering MLE-Bench benchmarks.

Original authors: Arun Sharma

Published 2026-04-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Arun Sharma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a brilliant but slightly scatterbrained detective to solve two very different types of mysteries:

The Warehouse Mystery: You show them a photo of a factory floor and ask, "How many pallets are within 3 meters of the emergency exit?"
The Data Puzzle: You give them a messy spreadsheet and say, "Build a computer program that predicts next month's sales perfectly."

Most AI detectives (Large Language Models) are great at talking and guessing, but they are terrible at math, counting, and measuring distances. If you ask them to count pallets, they might guess "maybe 12?" even if there are clearly 15. If you ask them to write code, they might write a program that crashes because they missed a tiny detail.

Spatial Atlas is a new system designed to fix this. It uses a philosophy called "Compute-Grounded Reasoning" (CGR).

Here is the simple breakdown of how it works, using some everyday analogies:

1. The "Do the Math First" Rule (Compute-Grounded Reasoning)

Imagine you are baking a cake. A normal AI might try to guess the amount of sugar needed based on how the cake looks in a picture. It might say, "I think it needs 2 cups," but it's just guessing.

Spatial Atlas is different. It says, "No guessing."

Step 1: It sends the picture to a specialized "measuring robot" (a computer vision tool) that counts the objects and measures the distances with perfect precision.
Step 2: It writes these facts down on a strict checklist: "There are exactly 15 pallets. The distance is 2.8 meters."
Step 3: It hands this checklist to the "brilliant detective" (the AI language model) and says, "Here are the facts. Now, tell me the answer."

Because the detective is working with hard facts instead of guesses, it never hallucinates (makes things up). It's like giving a chef a pre-measured ingredient list so they can't mess up the recipe.

2. The "Smart Manager" (Entropy-Guided Reasoning)

The system has a smart manager who decides how much brainpower to use for each question. Think of this like a tiered customer service line:

Tier 1 (The Fast Intern): For easy questions ("What color is the truck?"), the system uses a cheap, fast AI. If the intern is 90% sure, they just give the answer.
Tier 2 (The Senior Analyst): If the intern is only 60% sure ("Is that a safety violation?"), the system escalates it to a smarter, slightly more expensive AI.
Tier 3 (The Expert Professor): If the Senior Analyst is still unsure, the system calls in the "Professor" (a top-tier, very expensive AI) to think deeply and reflect on the problem.

This saves money and time. You don't call a Nobel Prize-winning physicist to tell you what day of the week it is; you only call them for the hard stuff.

3. The "Self-Healing" Code Builder (For the Data Puzzles)

When the system tries to solve the Kaggle machine learning competitions (the data puzzles), it acts like a robotic construction crew:

Build: It writes the code to solve the problem.
Test: It runs the code.
Fix: If the code crashes (like a building collapsing), the system doesn't give up. It reads the error message, figures out what went wrong, and patches the code automatically. It tries up to three times to fix itself.
Upgrade: Once the code works, it asks the "Professor" AI to look at the results and say, "Hey, if we change this one setting, the score might go up." It tries that new version. If the score is better, it keeps it. If it's worse, it throws it away.

4. The "Leak Detector" (The Leak Audit)

In data competitions, sometimes the "test" data accidentally leaks clues from the "training" data (like finding the answer key hidden in the homework).

Spatial Atlas has a special security scanner that checks the data before the AI even starts working.
It looks for sneaky patterns (like duplicate rows or weird timestamps).
If it finds a "leak," it whispers a hint to the AI: "Hey, look at this column; it might be the secret key." This allows the AI to solve the puzzle in a clever, legal way that others might miss.

Why Does This Matter?

The paper shows that by combining hard math (computing facts) with smart guessing (AI language), you get the best of both worlds.

Accuracy: It solves spatial questions much better than AI that just guesses.
Reliability: It fixes its own mistakes when writing code.
Efficiency: It doesn't waste expensive computer power on easy tasks.

In a nutshell: Spatial Atlas is like a team where the accountant does all the counting and measuring, the manager decides who does the work, and the creative writer just puts the final story together. By separating the "math" from the "talking," the whole team becomes much smarter and more reliable.

1. Problem Statement

The paper addresses the limitations of current Large Language Model (LLM) and Vision-Language Model (VLM) agents in two distinct but challenging domains:

Spatial Reasoning: VLMs frequently hallucinate spatial relationships, struggle with precise object counting, and fail at distance estimation when asked to reason about complex scenes (e.g., "How many pallets are within 3 meters of the exit?").
End-to-End ML Engineering: Existing agents often lack robustness in handling machine learning competitions (MLE-Bench), failing to recover from code execution errors or systematically improve model performance through iterative refinement.

Current approaches treat these as separate problems with specialized systems, leading to fragmented infrastructure. The authors argue for a unified architecture that grounds reasoning in deterministic computation rather than relying solely on generative model outputs.

2. Methodology: Compute-Grounded Reasoning (CGR)

The core design paradigm is Compute-Grounded Reasoning (CGR). The principle is: Whenever a sub-problem admits a deterministic solution, compute the answer first and supply it as a fact to the LLM, rather than asking the model to generate the answer.

The system, Spatial Atlas, is implemented as a single Agent-to-Agent (A2A) server that handles two benchmarks:

FieldWorkArena: Multimodal spatial QA in factory, warehouse, and retail environments.
MLE-Bench: A suite of 75 Kaggle machine learning competitions.

Key Architectural Components

A. Spatial Scene Graph Engine (for FieldWorkArena)
To overcome VLM hallucinations, the system does not ask the LLM to "look" at the image directly for spatial facts. Instead, it uses a three-stage pipeline:

Extraction: A VLM (GPT-4.1-vision) generates a textual scene description, while a lightweight vision model (Florence-2) performs precise object detection and bounding box extraction.
Graph Construction: Entities and relations are formalized into a structured Spatial Scene Graph ( $G = (V, E)$ ), where vertices are entities (with position, attributes, zone) and edges are spatial relations.
Deterministic Computation: The system executes deterministic queries on the graph (e.g., query_near, check_constraints, count_by_label). The resulting "fact sheet" is fed to the LLM to answer the user's question, eliminating spatial hallucinations.

B. Entropy-Guided Reasoning Engine
This module optimizes cost and reliability by selecting actions that maximize information gain:

Confidence Estimation: The LLM provides a confidence score ( $\sigma$ ) for its answers.
Model Routing: A three-tier model stack is used:
- Fast Tier (GPT-4.1-mini): For high-confidence, simple tasks.
- Standard Tier (GPT-4.1): For moderate complexity.
- Strong Tier (Claude Opus 4.6): Reserved for low-confidence tasks requiring reflection or complex refinement.
Reflection: If confidence is below a threshold ( $\tau=0.6$ ), the agent triggers a reflection step (re-querying the scene graph or escalating to the Strong model).

C. Self-Healing ML Pipeline (for MLE-Bench)
This component automates the Kaggle competition workflow:

Strategy-Aware Code Generation: The system classifies the competition type (Tabular, NLP, Vision, etc.) and generates a Python script using specific strategy templates (e.g., LightGBM for tabular, Transformers for NLP).
Error Recovery: If code execution fails, the system parses the error, generates a targeted patch, and re-executes (up to 3 iterations). A "dummy submission" fallback ensures a score is always produced.
Score-Driven Refinement: After a successful run, the system parses the validation score and uses the Strong Tier model to propose targeted improvements (e.g., switching to K-fold cross-validation, adding target encoding). It keeps the submission only if the score improves.
Leak Audit Registry: A prompt-based framework that instructs the Strong model to check for train/test data leakage (ID overlap, row fingerprinting, temporal ordering, byte hashing) before training, allowing the model to exploit known data leaks if they exist.

3. Key Contributions

Spatial Scene Graph Engine: Eliminates VLM spatial hallucinations by extracting structured representations and computing spatial facts deterministically.
Entropy-Guided Reasoning: An information-theoretic framework for cost-efficient model routing and targeted reflection, reducing variance in answers.
Self-Healing ML Pipeline: A robust system for code generation, automatic error repair, and iterative score-driven refinement.
Leak Audit Registry: A prompt-based exploit framework that detects and adapts to data leakage patterns at runtime.
Unified A2A Architecture: Demonstrates that a single server can handle both spatial reasoning and ML engineering tasks using a shared CGR infrastructure.

4. Results

FieldWorkArena Performance:

The full system achieved accuracy scores of 0.72 (Factory), 0.68 (Warehouse), and 0.74 (Retail).
Ablation Study: The Spatial Scene Graph (SSG) provided the largest gain, improving accuracy by 21–24 percentage points over a pure VLM baseline. Florence-2 preprocessing added 7–10 points, and Entropy-Guided reasoning added 7–8 points.
Conclusion: Deterministic spatial computation significantly outperforms generative spatial reasoning.

MLE-Bench Performance:

Valid Submission Rate: 82% across all 75 competitions (91% for Tabular tasks).
Medal Rate: 32% overall, with Tabular tasks achieving 42%.
Refinement Impact: The score-driven refinement loop improved validation metrics in 35–40% of eligible tasks.
Cost Efficiency: The entropy-guided routing kept average costs low for simple tasks while reserving expensive Strong-tier models for high-value refinement steps.

5. Significance and Impact

Paradigm Shift: The paper argues that Compute-Grounded Reasoning (CGR) is a superior design pattern for research agents. By grounding generation in deterministic computation, agents become more reliable, interpretable, and cost-effective.
Interpretability: The use of structured intermediate representations (Scene Graphs) and deterministic checks makes the agent's reasoning process transparent, unlike "black box" LLM reasoning.
Scalability: The A2A protocol allows for heterogeneous agent collaboration, suggesting a path toward general-purpose research agents that can operate across diverse domains (spatial, coding, data science) without needing entirely separate architectures.
Safety & Industry Application: The spatial reasoning capabilities have direct applications in industrial safety monitoring (e.g., verifying clearance distances, equipment placement), where hallucinations could have severe consequences.

In summary, Spatial Atlas demonstrates that combining structured deterministic computation with LLMs creates a robust framework for solving complex, multi-modal research tasks that current standalone VLMs or LLMs cannot reliably handle.