💻 computer science

TOL: Textual Localization with OpenStreetMap

This paper introduces TOL, a large-scale benchmark and the TOLoc framework for textual-to-OpenStreetMap localization, which enables accurate 2-DoF global positioning in urban environments by leveraging semantic and directional information from text descriptions and map tiles without relying on geometric observations or GNSS.

Original authors: Youqi Liao, Shuhao Kang, Jingyu Xu, Olaf Wysocki, Yan Xia, Jianping Li, Zhen Dong, Bisheng Yang, Xieyuanli Chen

Published 2026-04-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Youqi Liao, Shuhao Kang, Jingyu Xu, Olaf Wysocki, Yan Xia, Jianping Li, Zhen Dong, Bisheng Yang, Xieyuanli Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are lost in a massive, unfamiliar city. You don't have a GPS, and you can't take a picture because your phone camera is broken. However, you can talk. You pull out your phone and send a text message to a friend: "I'm standing on top of a parking lot. To my north is a big red building, to my south is a busy road, and to my west is a park."

Your friend, who has a giant, detailed map of the entire city, reads your message and instantly knows exactly where you are.

This paper is about teaching a computer to be that helpful friend. The researchers have created a new way for computers to find your location just by reading a text description of your surroundings, using a free, open-source map called OpenStreetMap (OSM).

Here is a breakdown of their work using simple analogies:

1. The Problem: The "Heavy Backpack" vs. The "Light Pocket"

Most current navigation systems are like hikers carrying a heavy backpack. They rely on 3D laser scans (point clouds) or high-resolution satellite photos. These are incredibly detailed but huge, expensive to store, and hard to update. If a new building goes up, the map might be outdated.

The researchers asked: Why carry the heavy backpack when we can just use a lightweight, free pocket map?
They chose OpenStreetMap (OSM). It's like a Wikipedia for maps—free, constantly updated by volunteers, and very small in file size. The challenge is that OSM is just lines and shapes; it doesn't "speak" English. The goal was to teach a computer to translate your English text description into a specific spot on this simple map.

2. The New Benchmark: "The TOL Library"

To teach the computer, you need a massive library of practice problems. The researchers built TOL (Textual Localization with OpenStreetMap).

The Scale: They created a dataset with 121,000 text-to-map pairs.
The Content: They took real driving routes in Boston, Singapore, and Karlsruhe. For every spot on the road, they automatically generated a text description like: "You are on a road. To your left is a school; to your right is a gas station."
The Magic: They did this without any humans typing a single sentence. They used the map data itself to write the descriptions, making it cheap and scalable.

3. The Solution: "TOLoc" (The Two-Step Detective)

The computer system they built, called TOLoc, works like a detective solving a mystery in two steps:

Step 1: The "Rough Search" (Place Recognition)
Imagine you are looking for a specific house in a city of millions. You wouldn't check every house one by one. First, you ask: "Is the house in the North District or the South District?"

The computer reads your text and looks at the map.
It creates a "fingerprint" for your text (e.g., "Park + Road + Building") and a "fingerprint" for every map tile.
It quickly scans the whole city map and says, "Okay, I found the top 5 map tiles that look most like your description." This narrows the search from the whole world down to a few city blocks.

Step 2: The "Fine Tuning" (Pose Estimation)
Now that the computer has 5 candidate blocks, it needs to find the exact spot.

This is where the TOA Module (Text-to-OSM Alignment) comes in. Think of this as a super-precise magnifying glass.
It takes your text description and the specific map tile it picked, and it aligns them perfectly. It calculates exactly how many meters you are from the center of that tile.
It outputs your precise coordinates (Latitude and Longitude).

4. Why is this a Big Deal?

It's Lightweight: Unlike other systems that need gigabytes of data, this works with tiny map files.
It's Human-Centric: It uses the way humans naturally describe things ("I'm near the park") rather than requiring technical data like laser scans.
It Works Everywhere: The researchers tested it in cities thousands of miles apart (Singapore vs. Boston). The system didn't just memorize the training cities; it learned the concept of matching text to maps, so it worked well in new, unseen cities too.

The Result

In their tests, their system was significantly better than existing methods. If the goal was to find a location within 10 meters, their system succeeded about 18% more often than the next best method.

In summary: This paper gives computers the ability to understand human speech about location and instantly pinpoint where you are on a free, digital map. It's like giving a robot a pair of eyes that can read a story and instantly know where the story is taking place.

1. Problem Definition

The paper addresses the Text-to-OSM (T2O) Global Localization task. The goal is to estimate an accurate 2-degree-of-freedom (2-DoF) position $(x, y)$ in an urban environment given only a natural language textual description of the surrounding scene and a city-scale OpenStreetMap (OSM) database.

Context: Unlike existing methods that rely on dense point clouds or high-resolution aerial imagery (which are costly to store and update), OSM offers a lightweight, free, and continuously updated semantic map.
Challenge: While previous works have explored Text-to-Point-Cloud or Text-to-Image localization, T2O localization remains underexplored. Existing T2O approaches (e.g., GOTPR, CVG-Text) focus primarily on coarse Place Recognition (PR) (retrieving a map tile) rather than precise Pose Estimation (PE) (meter-level localization). Furthermore, existing datasets often rely on Large Language Models (LLMs) for text generation, increasing cost and limiting scalability.
Task Formulation: Given a text query $T = \{h_i\}$ $T = {h_{i}}$ describing visible objects and their directions (North, South, East, West, Center) relative to a query point, the system must:
1. Retrieve the top- $K$ most similar OSM tiles from a database.
2. Regress the precise 2-DoF pose within the top-1 retrieved tile.

2. Methodology: TOLoc Framework

The authors propose TOLoc, a two-stage coarse-to-fine localization framework that explicitly models the semantics and directional information of the scene.

A. Data Representation

OSM Tiles: Geographic elements (nodes, ways, areas) are rasterized into 3-channel grid images. These are processed by a map encoder to generate spatial feature maps.
Text Descriptions: Queries consist of 5 sentences describing the center and four cardinal directions. A text encoder processes these to generate directional feature vectors.

B. Stage 1: Place Recognition (PR)

Direction-Aware Encoding:
- Map Side: The feature map of an OSM tile is partitioned into 5 spatial regions (Top, North, South, West, East) based on polar coordinates relative to the query center. Features in each region are aggregated via masked average pooling to create a direction-aware global descriptor.
- Text Side: Sentence features for the 5 directions are concatenated and passed through an MLP to form a global text descriptor.
Retrieval: The system computes cosine similarity between the text descriptor and all map descriptors in the database to retrieve the top- $K$ candidate tiles.
Backbone: Utilizes pre-trained vision-language models (CLIP or SigLIP) as encoders.

C. Stage 2: Pose Estimation (PE)

Text-to-OSM Alignment (TOA) Module: This is the core innovation for fine-grained localization.
- It takes the text descriptor and the patch-level features of the top-1 retrieved OSM tile.
- Self-Attention: Applied to map features to capture long-range spatial dependencies within the tile.
- Cross-Attention: Fuses the refined map features with the text descriptor, allowing the model to align semantic cues (e.g., "north of a building") with specific local map regions.
Regression: A projection head regresses the 2-DoF offset $(\delta x, \delta y)$ relative to the center of the retrieved tile. The final global position is calculated by adding this offset to the tile's coordinates.

D. Training Strategy

Two-Stage Training:
1. PR Stage: Trained with symmetric cross-entropy loss to maximize the matching probability between text and map tiles.
2. PE Stage: The top-1 tile and text are jointly optimized using an L1 loss on the predicted pose. A validity mask ensures only samples where the top-1 retrieval is close to the ground truth are used for pose regression training.

3. Key Contributions

TOL Benchmark:
- A large-scale dataset containing ~121,000 textual queries paired with OSM tiles across Boston, Singapore, and Karlsruhe (covering ~316 km of road trajectories).
- Fully Automated Pipeline: Unlike prior works relying on LLMs or manual annotation, TOL generates text by detecting visible objects in OSM data and applying predefined templates with visibility masking (ray-casting to handle occlusions).
- Diversity: Covers multiple continents and urban environments to test generalization.
TOLoc Framework:
- A novel two-stage framework that moves beyond simple retrieval to achieve meter-level accuracy.
- Introduces direction-aware feature modeling for both text and maps, significantly improving discriminative power.
- Proposes the TOA module for effective cross-modal fusion and precise pose regression.
State-of-the-Art Performance:
- Demonstrates that TOLoc significantly outperforms existing baselines (GOTPR, CVG-Text) in both place recognition and global localization tasks.

4. Experimental Results

Place Recognition (PR):
- On the TOL-N validation set, TOLoc (SigLIP-384 backbone) achieved 27.60% Recall@1 (at 25m threshold), outperforming the best baseline (CVG-Text) by a significant margin.
- It showed strong generalization when trained on TOL-N and tested on the unseen TOL-K360 (Karlsruhe) dataset.
Global Localization (PE):
- Success Rate (SR): TOLoc achieved a 28.10% success rate within a 25m threshold (using SigLIP-384), outperforming the best existing method by 8.31%.
- Precision: At stricter thresholds, TOLoc achieved 6.83% success at 5m and 18.65% at 10m.
- Ablation Studies: Confirmed that the TOA module is critical; removing it (using simple MLP or CLS baselines) resulted in poor localization accuracy, often predicting the map center.
Efficiency:
- The system is efficient, with the PR stage taking ~70ms per query and the lightweight PE stage taking only ~2.8ms.

5. Significance and Impact

Scalability: By utilizing OpenStreetMap instead of expensive point clouds or proprietary satellite imagery, this approach enables scalable, low-cost localization for global applications.
Human-Machine Interaction: It provides a natural interface for localization, allowing users (e.g., lost pedestrians, emergency responders) to describe their surroundings in plain text to find their location without needing GPS or specialized sensors.
Foundation for Future Work: The TOL benchmark and TOLoc framework establish a new standard for text-driven geospatial understanding, paving the way for active navigation and dialogue-based localization systems.

Conclusion: The paper successfully bridges the gap between natural language understanding and precise geospatial localization using open data, demonstrating that semantic and directional cues from text can be effectively aligned with OSM structures to achieve robust, meter-level localization.