Learning to Wander: Improving the Global Image Geolocation Ability of LMMs via Actionable Reasoning

1. Problem Statement

Geolocation (identifying the geographic coordinates of an image) is a complex task requiring extensive world knowledge and reasoning. While Large Multimodal Models (LMMs) have shown promise, their performance on geolocation remains under-explored.

The core limitations of current approaches are:

• Static Paradigm: Existing methods (e.g., PlaNet, Im2GPS) treat geolocation as a static classification or retrieval problem based on a single image. They lack the ability to perform active exploration.
• Lack of Interactivity: Humans solve geolocation problems by moving, rotating views, and gathering new evidence when initial clues are ambiguous. Current models cannot "act" to reduce uncertainty.
• Dataset Limitations: Existing datasets (e.g., Im2GPS3K, YFCC) consist of discrete, static images. They lack the navigable graph structures required to simulate embodied agent actions (rotation, movement).

2. Methodology

The paper introduces two main components: a new benchmark dataset (WanderBench) and a novel reasoning framework (GeoAoT).

A. WanderBench: The First Actionable Geolocation Benchmark

WanderBench is an open-access, global geolocation dataset designed specifically for embodied agent interaction.

• Scale & Scope: Contains 32,741 panoramic images across 1,047 unique locations spanning six continents.
• Structure: Data is organized as navigable graphs.
  • Nodes: High-resolution panoramic images (2048×1024) with precise GPS coordinates.
  • Edges: Traversable paths between panoramas, allowing agents to "move forward," "rotate," or "backtrack."
• Generation Process:
  1. Location Proposal: LMMs generate detailed textual descriptions of locations with specific difficulty levels.
  2. Verification: LMMs verify these locations against satellite maps (Zoom 16) to ensure validity.
  3. Retrieval: Panoramas are retrieved via map service APIs.
  4. Question Generation: LMMs generate geolocation questions (multiple-choice, open-ended) paired with the panoramas, calibrated for difficulty.
• Evaluation Protocol: Uniquely supports bidirectional evaluation:
  1. Solving: The model must locate the image.
  2. Proposing: The model must generate new, valid geolocation questions at specific difficulty levels (testing meta-reasoning).

B. GeoAoT (Action of Thought)

GeoAoT is a framework that couples multimodal reasoning with physical actions, moving beyond static Chain-of-Thought (CoT).

• Mechanism: Instead of generating a static reasoning chain, the model enters an iterative loop:
  1. Observation: Analyze the current panorama.
  2. Reasoning: Identify missing or ambiguous visual cues (e.g., "signage is unreadable," "need to check vegetation").
  3. Action: Formulate and execute a concrete environmental action (e.g., "Rotate 180° right," "Move forward 10 meters").
  4. Update: Re-evaluate the hypothesis based on the new observation.
• Inference Only: The framework operates purely at inference time without additional training, leveraging pre-trained LMMs.

3. Key Contributions

1. WanderBench Dataset: The first open-access global dataset supporting interactive agent actions (movement/rotation) and bidirectional evaluation (solving + generating tasks). It bridges the gap between static image recognition and dynamic spatial exploration.
2. GeoAoT Framework: A novel agent paradigm that integrates Action of Thought. It transforms reasoning into actionable plans (e.g., approaching landmarks, adjusting viewpoints) to actively reduce uncertainty, mimicking human problem-solving strategies.
3. New Evaluation Metrics: Introduces a protocol that measures not just geolocation accuracy (Distance, F1, Recall) but also difficulty-aware question generation and spatial diversity of proposed locations.

4. Experimental Results

The authors evaluated 19 leading LMMs (both open-source like Qwen, InternVL, and closed-source like GPT-4o, Gemini, o3) on WanderBench.

• Baseline Performance:

  • Closed-source models generally outperformed open-source ones. Gemini-2.5 Pro achieved the best baseline results (Distance: 134.3 km, Score: 95.28).
  • Open-source models like Qwen2.5-VL and GLM-4.5V showed competitive performance, narrowing the gap with closed systems.
  • Retrieval-based models (e.g., GeoCLIP) struggled with fine-grained (street-level) accuracy compared to LMMs.

• GeoAoT Impact:

  • Universal Improvement: Applying GeoAoT improved performance across all tested models.
  • Error Reduction: Models saw significant reductions in distance error. For example, o3 reduced error from 318.8 km to 180.3 km (−138.5 km), and LLaVA-OneVision reduced error by over 1,000 km.
  • Metric Gains: Consistent improvements in Street, City, and Country-level F1 scores. For instance, o3's Street-level F1 increased from 0.524 to 0.606.
  • Interpretability: The framework produced more transparent reasoning traces, showing exactly how the model gathered evidence to reach a conclusion.

• Robustness Analysis:

  • ANOVA and trend analysis revealed that top-tier models (o3, Gemini-2.5 Pro, GPT-4o) exhibit monotonic performance improvement as data coverage increases, indicating strong generalization.
  • Some models showed non-monotonic behavior, suggesting instability under varying coverage conditions.

5. Significance

• Paradigm Shift: The paper redefines geolocation from a static "image-to-coordinate" mapping task to a dynamic, interactive exploration process.
• Embodied AI: It demonstrates that equipping LMMs with the ability to "act" (move/rotate) significantly enhances their reasoning capabilities in spatial tasks, a crucial step toward truly autonomous embodied agents.
• Meta-Reasoning: By requiring models to generate their own test cases, WanderBench evaluates a deeper level of "meta-understanding" regarding geographical reasoning and difficulty calibration.
• Future Direction: Establishes a foundation for developing LMMs capable of self-evaluative, geographically grounded intelligence, essential for applications in autonomous navigation, emergency response, and fact-checking.