🤖 AI

SheetCompass: Hierarchical Relation Graphs for Agentic Spreadsheet Reasoning

SheetCompass is a graph-guided, memory-driven agentic framework that addresses the limitations of existing LLM-based spreadsheet reasoning by explicitly modeling hierarchical structural relationships and spatial contexts within and across worksheets to enable more effective automation of complex workbooks.

Original authors: Panjing He, Mingyue Cheng, Yucong Luo, Li Li, Xiaohan Zhang

Published 2026-08-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Panjing He, Mingyue Cheng, Yucong Luo, Li Li, Xiaohan Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Puzzle of the Digital Spreadsheet

Imagine you are trying to teach a super-smart robot how to read a messy, multi-page spreadsheet. In the world of Artificial Intelligence, this is a bit like asking a detective to solve a crime by reading a list of words that were ripped out of a book and shuffled into a single line. Spreadsheets are special because they aren't just lists of data; they are visual maps. They have rows, columns, and hidden connections between different pages that tell a story about money, sales, or inventory. For a human, looking at a spreadsheet is like scanning a city map: you instantly see how the "Sales" street connects to the "Prices" avenue. But for a Large Language Model (LLM)—the kind of AI that writes poems and answers questions—spreadsheets are often a nightmare. When we feed these models a spreadsheet, we usually flatten it into a long string of text, like turning a 3D building into a flat piece of paper. In doing so, we lose the "where" and the "how" of the data. The robot forgets that a number in the top left belongs to a category in the bottom right, or that two different sheets are actually talking to each other. This paper, titled SheetCompass, asks a simple but crucial question: How do we teach an AI to see the map again, rather than just reading the list of words?

The Paper's Story: Building a Compass for the AI

The authors of this paper, working at the University of Science and Technology of China, realized that the problem wasn't that the AI wasn't smart enough; it was that the AI was looking at the spreadsheet the wrong way. They argue that treating a spreadsheet like a simple text document strips away the very structure that makes it useful. To fix this, they built a new system called SheetCompass. Think of SheetCompass not as a robot that just reads, but as a team of three expert explorers working together with a custom-made map.

First, the team had to solve the "flat text" problem. Instead of feeding the AI a long, confusing paragraph, SheetCompass turns the spreadsheet into a hierarchical graph. Imagine a spreadsheet as a messy room full of furniture. Traditional methods try to describe the room by listing every object in a single sentence: "chair, table, lamp, chair, table..." SheetCompass, however, builds a 3D model of the room. It creates a "Table Node" for the whole table and "Column Nodes" for the headers, then draws lines connecting them. It even draws special "semantic lines" between columns that look different but mean the same thing (like "Price" on one sheet and "Cost" on another), even if they are far apart. This creates a stable, structured skeleton that the AI can navigate without getting lost.

But having a map isn't enough; the explorers need to know how to use it. SheetCompass introduces a dual-level memory system. One part of the memory is like a library of expert rules (Expert Knowledge), teaching the AI things like "don't delete rows, just hide them" or "how to calculate a pivot table." The other part is a diary of past mistakes (Reasoning Experience), where the system remembers what went wrong in previous attempts so it doesn't repeat the same error. This is like a student who not only has a textbook but also keeps a notebook of their own test corrections.

Finally, the system brings in the multi-agent team. Instead of one AI trying to do everything, SheetCompass splits the job into three roles:

The Navigator (Explorer): This agent looks at the graph map, finds the right columns and tables, and figures out the order of steps needed to solve the problem.
The Builder (Programmer): This agent takes the plan and writes the actual code to do the math or create the charts, making sure it sticks strictly to the map so it doesn't invent fake data.
The Inspector (Reflector): This is the safety guard. After the code runs, the Inspector checks the results against a checklist. If the math looks weird (like subtracting a tiny decimal from a huge price), the Inspector stops the process, says "Wait, that doesn't make sense," and sends the team back to fix it.

What They Found

The researchers tested SheetCompass on several difficult datasets that included complex spreadsheets with multiple tables and tricky instructions. They compared their new system against other popular AI methods, including those that just write code in one go or use simpler agent loops. The results were clear: SheetCompass was significantly better at getting the job done right the first time.

In their tests, when using a powerful AI model (GPT-4), SheetCompass achieved a success rate of 63.2% on one major benchmark (SCB), beating the next best method which only got 61.1%. On another tough test (SB) that required the AI to pass every single check in a series of trials, SheetCompass improved the success rate to 18.3%, a big jump from the previous best of 13.5%. When they used an even more advanced model (GPT-5), the gap widened, with SheetCompass reaching 71.3% on the SCB benchmark and 22.0% on the hard SB benchmark.

The paper also ran "ablation studies," which are like taking apart a machine to see which part is most important. They found that if they removed the graph map, the system's performance dropped sharply, proving that the map is the most critical piece. They also found that the Inspector (Reflector) was vital; without it, the system made more mistakes because it couldn't catch its own errors.

Why It Matters

The authors suggest that the key to making AI good at spreadsheets isn't just making the AI smarter, but giving it a better way to see the data. By turning flat text into a structured graph and using a team of specialized agents to check each other's work, SheetCompass bridges the gap between how humans naturally read tables and how computers process information. The paper doesn't claim to have solved every spreadsheet problem in the world, but it shows that this new approach of "structural perception" combined with "collaborative reasoning" is a powerful way forward. It suggests that for AI to truly master the messy, real-world tools we use every day, it needs to stop reading lists and start reading maps.

Technical Summary: SheetCompass

Problem Statement

Automated spreadsheet reasoning remains a significant bottleneck for Large Language Models (LLMs), particularly when handling complex, real-world workbooks. While LLMs excel at semantic understanding, their performance degrades sharply on spreadsheets due to the prevailing "information-flattening paradigm." Existing methods typically serialize multi-dimensional grids into linear strings (e.g., Markdown or JSON), which fundamentally strips away:

Intra-sheet Topology: The orthogonal positioning system of rows and columns is shattered, causing complex layouts to lose their spatial neighborhood within the token stream.
Inter-sheet Semantics: Implicit data dependencies and business logic across different worksheets are obscured, breaking critical correspondences (e.g., between cost and price sheets) into isolated text fragments.

This flattening creates a mismatch with the visual, grid-based nature of spreadsheets, preventing LLMs from leveraging the spatial cues human experts use for rapid scanning and logical alignment. Consequently, models struggle to reconstruct dynamic data flows, leading to structural misalignment in cross-table reasoning tasks.

Methodology: The SheetCompass Framework

To address these limitations, the authors propose SheetCompass, a graph-guided and memory-driven agentic framework. The core philosophy is to transform raw tabular data into structured evidence spaces that guide agent planning, execution, and reflection. The framework consists of three primary components:

1. Hierarchical Relation Graph Construction

Instead of linear sequences, SheetCompass models spreadsheets as a hierarchical graph $G = (V, E)$ to preserve structural dependencies.

Node Hierarchy: The graph organizes vertices into a two-tier hierarchy: Table Nodes (representing worksheets) and Column Nodes (representing headers paired with sample data entries). This avoids dense cell-level modeling while capturing semantic identity.
Edge Types:
- Structural Edges ( $E_{str}$ ): Deterministically extracted from the layout to define physical boundaries. These include inclusion edges (linking tables to columns) and adjacency edges (connecting neighboring columns horizontally).
- Semantic Edges ( $E_{sem}$ ): Dynamically constructed to link related columns across different sheets, even if spatially distant. This is achieved by calculating a combined score of spatial-semantic similarity (via a pre-trained transformer) and logical relationship verification (via an LLM checking for primary/foreign key logic). A confidence threshold filters out low-quality connections.

2. Dual-Level Memory Mechanism

To enhance agent reliability and support long-horizon tasks, SheetCompass employs two memory systems:

Expert Knowledge Memory: A static repository containing domain knowledge (procedural rules, common formulas) and tool-use knowledge (historical code-execution logs, error root causes, and repair solutions). This provides a stable cognitive baseline to prevent hallucinations.
Reasoning Experience Memory: A dynamic, short-term record of the current reasoning session. It logs episodic details such as graph traversal paths, sandbox error messages, and state mismatches. Crucially, successful trajectories and high-value lessons from this memory are periodically distilled into the long-term Expert Knowledge Memory.

3. Multi-Agent Workflow

The framework coordinates three specialized agents to handle perception, execution, and verification in a closed loop:

Navigational Explorer: Decomposes high-level instructions into atomic steps. It uses the hierarchical graph to perform topological sensing, extracting relevant subgraphs via semantic similarity filtering and Breadth-First Search (BFS) to anchor targets and reduce context length.
Logical Programmer: Acts as a constrained code generator. It translates the explorer's steps into executable Python scripts, strictly adhering to the verified node sets of the graph to ensure all variables are anchored to the correct physical layout.
Critical Reflector: Implements a state-check feedback mechanism. It translates user instructions into a checklist and validates the spreadsheet's metadata state after code execution. If discrepancies are found, it generates diagnostic notes to trigger iterative refinement by the Explorer and Programmer.

Key Contributions

The paper claims three primary contributions:

Structural-Reasoning Bridging: The proposal of SheetCompass, which effectively bridges structural space perception with multi-agent reasoning, overcoming the information loss inherent in conventional flat sequences.
Hierarchical Contextual Graph: The introduction of a graph structure that preserves both intra-sheet spatial layouts and inter-sheet semantic linkages, combined with a dual-level memory system and coordinated multi-agent workflow to guarantee high-fidelity execution.
Empirical Validation: Extensive experiments demonstrating that SheetCompass achieves state-of-the-art performance on complex spreadsheet benchmarks, confirming the critical impact of structural and semantic cross-table reasoning.

Experimental Results

The framework was evaluated on three benchmarks: SCB (SheetCopilotBench), SB (SpreadsheetBench), and SheetRM.

Performance: SheetCompass consistently outperformed strong baselines (including Binder, VBA generation, OS-Copilot, SheetCopilot, and SheetAgent) across all metrics.
- On SCB, it achieved a 63.2% Pass@1 (GPT-4 backbone) and 71.3% (GPT-5 backbone), surpassing the best baseline by significant margins.
- On SB, it achieved a 22.0% Hard Restriction score (GPT-4) and 24.7% (GPT-5), representing absolute improvements of 6.4% to 6.9% over the strongest baselines.
- On SheetRM, it reached a 43.5% Pass@1 (GPT-4) and 52.3% (GPT-5).
Ablation Studies: Removing the hierarchical graph caused the sharpest performance drop (e.g., -14.9% Pass@1 on SCB), confirming that structural grounding is foundational. Similarly, disabling the multi-agent workflow or dual-level memory led to significant degradation, validating the necessity of collaborative execution and memory retention.
Complexity Analysis: SheetCompass demonstrated superior robustness in multi-table scenarios compared to single-table settings. While baselines suffered steep performance drops when crossing tables, SheetCompass maintained high accuracy, attributed to its ability to use semantic edges to track cross-table dependencies.

Significance and Claims

The paper positions SheetCompass as a solution to the fundamental mismatch between linear text processing and the visual, grid-based design of spreadsheets. By explicitly modeling the "intrinsic grid properties" through a hierarchical graph, the framework allows downstream agents to navigate multi-dimensional spaces without losing spatial context.

The authors claim that this approach resolves a major bottleneck in industrial productivity by enabling high-fidelity spreadsheet automation. The work establishes a scalable foundation for next-generation enterprise tools, moving beyond simple pattern matching to robust, interpretable, and self-correcting reasoning systems that can handle complex, non-linear data dependencies. The framework's ability to transition from error detection to logical self-repair via its closed-loop cognitive ensemble is highlighted as a key advancement in agentic reasoning.