🤖 AI

GeoContra: From Fluent GIS Code to Verifiable Spatial Analysis with Geography-Grounded Repair

GeoContra is a verification and repair framework that enhances LLM-driven GIS workflows by enforcing geospatial contracts through static inspection, runtime validation, and semantic feedback loops, significantly improving the spatial correctness of generated Python code across diverse models and real-world tasks.

Original authors: Yinhao Xiao, Rongbo Xiao, Yihan Zhang

Published 2026-05-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yinhao Xiao, Rongbo Xiao, Yihan Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very talented, fast-talking assistant who loves to write computer code for maps. This assistant is great at sounding confident and producing code that looks perfect on the surface. However, if you ask them to calculate how long it takes to walk from a park to a hospital, they might accidentally write code that says the walk takes -5 minutes (negative time!) or measures the distance in "degrees" instead of "meters."

In the world of maps and geography, these mistakes are like a chef serving a delicious-looking cake that is actually made of soap. It looks right, but it's geographically impossible.

This paper introduces GeoContra, a new system designed to act as a strict, geography-savvy "quality control inspector" for these AI assistants. Here is how it works, broken down into simple concepts:

1. The Problem: "Fluent" but Wrong

Currently, AI models can write code that runs without crashing (it's "fluent"), but they often break the fundamental rules of geography. They might mix up map projections, ignore traffic rules, or calculate distances in ways that make no sense in the real world. Before this paper, there wasn't a good way to catch these specific geographic errors automatically before the results were used.

2. The Solution: The "Geographic Contract"

Think of GeoContra as a contract or a rulebook that the AI must sign before it starts working.

The Old Way: You ask the AI, "How do I find the nearest hospital?" and it just guesses the code.
The GeoContra Way: Before the AI writes a single line of code, the system gives it a detailed contract. This contract says: "You must use this specific map grid (CRS), you cannot use negative numbers for time, you must check the walking path (not a straight line), and your output must look like this specific list."

It's like giving a construction worker a blueprint that explicitly says, "Do not build the roof with ice," rather than just hoping they know better.

3. The Three-Step Inspector Team

Once the AI writes the code based on the contract, GeoContra runs it through a three-layer inspection process:

The Static Inspector (The Code Reviewer): This looks at the code before it runs. It checks for obvious red flags, like trying to measure a walking path using a ruler meant for flying birds, or using a field name that doesn't exist in the data.
The Runtime Inspector (The Test Driver): This actually runs the code. It checks if the output file was created, if the numbers make sense (e.g., no negative travel times), and if the file isn't corrupted.
The Semantic Inspector (The Logic Check): This asks, "Did you actually do what was asked?" If the task was to find the nearest hospital, but the code just listed all hospitals, this inspector catches that the AI took a shortcut and didn't solve the real problem.

4. The Repair Loop: "Try Again, But Smarter"

If the code fails any of these checks, GeoContra doesn't just say "Error." It sends a specific note back to the AI: "You tried to measure distance in degrees, but the contract said use meters. Also, you forgot to check the walking network. Please fix these specific errors and try again."

The AI then rewrites the code with these specific instructions in mind. This happens in a loop until the code passes all checks or the system runs out of attempts.

5. The Results: From "Maybe" to "Trustworthy"

The researchers tested this system on 7,079 real-world map tasks using data from 15 different areas around Boston (covering cities, suburbs, parks, and waterways). They tested it on 11 different AI models, including some of the most powerful ones available.

Without GeoContra: The AI models produced code that was geographically correct only about 47% to 57% of the time.
With GeoContra: The accuracy jumped to 77% to 81%.

Even for smaller, less powerful AI models, the system helped them become much more reliable, turning code that would have failed completely into code that worked correctly.

The Big Takeaway

The main point of this paper is that for AI to be useful in geography and city planning, it shouldn't just be judged on whether the code runs. It must be judged on whether the code makes geographic sense.

GeoContra turns AI from a "code generator" that might produce dangerous errors into a "verifiable analyst" that respects the rules of the real world. It ensures that when an AI says a hospital is 10 minutes away, that 10 minutes is actually based on a real walking path, not a mathematical glitch.

1. Problem Statement

The central challenge in automated Geographic Information Science (GIS) is not merely generating syntactically correct code, but ensuring that the generated analysis preserves geographic meaning.

The Gap: Current Large Language Model (LLM) systems for GIS often produce "fluent" scripts that execute successfully but violate critical geographic rules. These violations include:
- CRS Errors: Performing metric operations (e.g., distance, area) on unprojected coordinate systems (degrees vs. meters).
- Semantic Errors: Misinterpreting spatial predicates (e.g., treating Euclidean distance as network accessibility).
- Topological/Logic Errors: Dropping zero-count objects, creating negative travel times, or aggregating over incorrect areal units.
- Schema Violations: Hallucinating field names or casting data types incorrectly.
Limitation of Existing Benchmarks: Most existing evaluations focus on code fluency, tool invocation, or platform-specific execution (e.g., Google Earth Engine). They rarely combine real-world local vector/raster data with executable contracts that verify geographic invariants (topology, units, semantics) at scale.

2. Methodology: The GeoContra Framework

GeoContra is a verification and repair framework that wraps LLMs with machine-checkable geospatial contracts. It transforms the workflow from "generate code" to "generate verifiable spatial analysis."

A. Task Formulation (The Contract)

A geospatial task $\tau$ is defined as a tuple $(q, D, Y, C, M^+, M^-)$ :

$q$ : Natural language request.
$D$ : Input datasets with schemas and Coordinate Reference Systems (CRS).
$Y$ : Expected output specifications (columns, row counts, metric ranges).
$C$ : Spatial Constraints: Geographic validity rules (e.g., "CRS must be projected for metric ops," "travel time $\ge$ 0").
$M^+$ / $M^-$ : Sets of Expected Methods (e.g., shortest_path) and Forbidden Shortcuts (e.g., direct Euclidean distance for network tasks).

B. The Three-Layer Verification Engine

GeoContra employs a tripartite verification system to detect violations before accepting a result:

Static Verifier ( $V_s$ ): Inspects the Abstract Syntax Tree (AST) before execution.
- Checks for metric operations on unprojected CRS.
- Detects spatial predicate direction mismatches (e.g., point-in-polygon vs. polygon-in-point).
- Flags forbidden shortcuts (e.g., using .distance() in a network task).
Runtime Verifier ( $V_r$ ): Executes the code and validates the output artifacts.
- Ensures output files exist and are readable (CSV/GeoJSON).
- Validates row counts, duplicate IDs, and non-negative metrics (distances, times).
- Checks for schema consistency (no hallucinated columns).
Semantic Verifier ( $V_t$ ): Checks if the logic matches the task intent.
- Verifies that specific GIS operations were actually used (e.g., did the code actually perform a spatial join or buffer aggregation?).
- Distinguishes between "executable but wrong" (e.g., a file was created, but via the wrong algorithm).

C. Bounded Repair Loop

If violations are detected, the system does not simply fail. Instead:

Violations are serialized into a Repair Prompt containing the task context, specific error evidence (line numbers, tracebacks), and hard constraints.
The LLM is prompted to regenerate the code ( $p_{r+1}$ ) to fix the specific violations.
This loop continues up to a maximum budget ( $R_{max}$ ) until the program is executable and violation-free.

3. Key Contributions

GeoContra-Real Benchmark:
- A large-scale benchmark comprising 7,079 real geospatial tasks across 15 Greater Boston areas.
- Covers 9 task families: Buffer counting, spatial joins, nearest-neighbor, network accessibility, raster sampling, zonal statistics, topology quality, overlay areas, and multi-step workflows.
- Includes diverse urban morphologies (dense inner-city to suburban/coastal) and heterogeneous data (vector, raster, network, topology).
Geography-Grounded Formulation:
- The first framework to encode GIScience rules (CRS validity, topology, spatial semantics) as explicit, executable contracts rather than relying on implicit model knowledge.
Multilayer Verification & Repair:
- A novel system that translates GIS rules into static, runtime, and semantic checks, using specific geographic violations to guide model repair rather than generic self-reflection.
Extensive Evaluation:
- Evaluated on 2 closed models (DeepSeek-V4, Kimi-K2.5) and 11 open-source models, totaling over 20,000 runs.

4. Results

The framework significantly improves the reliability of LLM-driven GIS workflows.

Closed-Model Performance:
- DeepSeek-V4: Spatial correctness improved from 47.6% to 77.5% (+29.9 percentage points).
- Kimi-K2.5: Spatial correctness improved from 57.7% to 81.5% (+23.8 percentage points).
- Executability also increased significantly (e.g., DeepSeek-V4 from 90.9% to 98.2%).
Open-Model Performance:
- Across 11 open models, average spatial correctness rose from 18.3% (LLM-only) to 44.9% (GeoContra), a +26.6 percentage point gain.
- The framework is particularly effective for models that can follow structured constraints, even if their baseline coding ability is weak (e.g., Qwen2.5-32B improved from 0.3% to 43.0%).
Failure Reduction:
- Final violations (static, runtime, and semantic) dropped sharply across all models. For instance, runtime violations for DeepSeek-V4 dropped from 0.102 to 0.027 per task.
Case Studies:
- GeoContra successfully corrected specific errors such as negative travel times, invalid output casts (string to integer), and the use of forbidden shortcuts (Euclidean distance in network analysis).

5. Significance and Implications

Methodological Shift: GeoContra argues that trustworthy GeoAI must move from "code assistance" to "verifiable spatial analysis." A model should not be rewarded merely for producing fluent code, but for preserving the geographic integrity of the analysis chain.
Practical Impact: By catching errors like negative travel times or CRS mismatches, the framework prevents the silent distortion of critical planning outcomes (e.g., accessibility, environmental justice, exposure analysis).
Scalability: The framework demonstrates that explicit contracts and automated verification can scale to thousands of real-world tasks, bridging the gap between LLM capabilities and the rigorous requirements of GIScience.
Future Directions: The authors note that while the current verifier is lightweight, future work should incorporate stronger reference solvers, topology-aware geometry comparison, and checks for spatial distributional plausibility (e.g., spatial autocorrelation).

In summary, GeoContra establishes a new standard for evaluating LLMs in GIS by proving that geography-grounded contracts are essential for transforming fluent code generation into reliable, reproducible, and geographically valid spatial analysis.