💻 computer science

COMPASS: COmpact Multi-channel Prior-map And Scene Signature for Floor-Plan-Based Visual Localization

The paper presents COMPASS, a visual localization algorithm that estimates a robot's pose by matching a multi-channel radial descriptor derived from architectural floor plans (encoding geometry and semantic features like walls and windows) with a corresponding descriptor generated from fisheye camera imagery using a novel window detection method.

Original authors: Muhammad Shaheer, Miguel Fernandez-Cortizas, Asier Bikandi-Noya, Holger Voos, Jose Luis Sanchez-Lopez

Published 2026-04-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Muhammad Shaheer, Miguel Fernandez-Cortizas, Asier Bikandi-Noya, Holger Voos, Jose Luis Sanchez-Lopez

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find your way inside a massive, empty building under construction. You have two things to help you: a blueprint (the architect's drawing of the building) and a pair of 360-degree cameras strapped to your head.

The problem is that most robots trying to solve this puzzle only look at the shape of the room. They see a wall here and a corner there, but they don't know if that wall is solid concrete or a giant glass window. It's like trying to identify a person in a crowd just by their height, ignoring whether they are wearing a red hat or a blue coat.

The researchers behind COMPASS (a clever name for their new robot brain) decided to fix this by teaching the robot to read both the shape and the identity of the building's features.

Here is how they did it, broken down into simple steps:

1. The "Radial Fingerprint"

Imagine standing in the middle of a room and spinning around in a circle. As you spin, you are taking a snapshot of everything around you.

The Blueprint Side: The robot looks at the floor plan. It shoots 360 invisible laser beams (rays) out from its center. For every single degree of the circle, it records:
- How far the beam traveled before hitting something.
- What it hit: Was it a solid Wall? A glass Window? Or was it an Opening (like a doorway)?
- How the distance changed: Did the wall suddenly get closer (a corner) or stay the same (a flat wall)?

This creates a unique "fingerprint" for that specific spot in the building. It's not just a map of distances; it's a map of what is there.

2. The "Camera Side"

Now, the robot looks at the real world through its fisheye cameras (which see everything around them, like a fish's eye).

The Challenge: The camera sees pixels, not blueprints. It needs to figure out, "Is that bright rectangle a window or just a shiny patch of wall?"
The Solution: The team built a special algorithm that acts like a detective. It looks for straight lines (using a tool called ELSED) and checks for brightness. If it sees a vertical frame with a bright center, it says, "Aha! That's a window!"
The Match: It then translates what the camera sees into the same "fingerprint" format as the blueprint. It marks the spots where it sees windows with a specific code, just like the blueprint did.

3. The "Magic Spin" (Matching)

Once the robot has its "Blueprint Fingerprint" and its "Camera Fingerprint," it tries to match them.

Think of it like two gears with teeth. The robot spins the camera's fingerprint around until the "teeth" (the patterns of walls and windows) line up perfectly with the blueprint's teeth.
When they line up, the robot knows exactly where it is and which way it is facing.

What They Actually Proved

The researchers tested this on a dataset from a construction site (the Hilti-Trimble challenge). They didn't build a full robot that walks around yet; instead, they took a single snapshot from a known location and proved the concept works:

The Match: When they compared the camera's view of the windows against the blueprint's drawing of the windows, the patterns matched up incredibly well (about 95% correlation).
The Heading: The robot correctly figured out which way it was facing just by matching these patterns.
The Tilt: They also showed they could figure out if the robot was leaning left/right or up/down by looking at where vertical lines (like door frames) seemed to converge in the sky (a trick called "vanishing points").

The Catch (What They Admit)

The paper is honest about its current limits. This "window-matching" trick works great near the outside walls of a building where there are lots of windows. However, if you are in the middle of a long, boring hallway with no windows, the robot might get confused because the "window pattern" disappears.

They also noted that in a real construction site, the building might not be finished yet (e.g., a wall in the blueprint hasn't been built in real life), which can cause mismatches.

In summary: COMPASS is a new way for robots to use blueprints and cameras together. Instead of just seeing "shapes," the robot learns to recognize "stories" (walls vs. windows), making it much better at knowing where it is inside a building.

1. Problem Statement

Indoor localization for mobile robots in service, logistics, and construction is a persistent challenge. While LiDAR-based SLAM is mature, it requires expensive sensors and often necessitates building maps from scratch in new environments. Conversely, 2D architectural floor plans are widely available and contain rich geometric and semantic information (walls, doors, windows). However, existing floor-plan-based localization methods suffer from two main limitations:

Underutilization of Semantics: Many methods (e.g., particle filters) treat all structural boundaries identically, failing to distinguish between walls, windows, and openings.
Geometric Limitations: Existing descriptors like Scan Context (used for LiDAR) encode only geometric properties (primarily height/range) but lack semantic element types. This reduces discriminative power in indoor environments where rooms may share similar geometry but differ in their specific wall-window layouts.

The paper aims to bridge the gap between 2D floor plans and visual data by creating a descriptor that jointly encodes geometry and structural semantics to enable robust, cross-modal localization.

2. Methodology: The COMPASS Framework

The authors propose COMPASS, a framework that generates a unified, multi-channel radial descriptor from both floor plans and fisheye camera images. The system consists of three main components:

A. Descriptor Design

The core innovation is a 5-channel radial descriptor ( $D \in \mathbb{R}^{5 \times 360}$ ) representing the environment in 360 azimuth bins (1° resolution). The five channels encode:

Normalized Range: $r_j / r_{max}$ . Distinguishes proximity to walls vs. open space.
Hit Type (Semantic): Encodes the element type: Wall (1.0), Window (0.5), or Opening (0.0). This captures the alternating pattern of building facades.
Range Gradient: Highlights corners and depth discontinuities (sharp changes in distance).
Inverse Range: $1/(1 + r_j)$ . Correlates with the angular extent of elements in fisheye images (nearby objects appear larger).
Local Range Variance: Captures structural complexity within a local window (e.g., distinguishing smooth walls from complex corridor entrances).

B. Floor Plan Descriptor Generation

Input: A 2D floor plan represented as binary masks for walls and windows.
Process: From a candidate pose $(t_x, t_y, \psi)$ , 360 rays are cast. The algorithm records the distance to the first hit and the type of hit (wall, window, or open) for each bin.
Output: The 5-channel descriptor matrix populated with geometric and semantic data derived from the map.

C. Visual Descriptor Generation (Fisheye Images)

Input: Dual fisheye images (front and back) from an Insta360 camera (covering ~360°).
Window Detection Algorithm:
1. Line Segmentation: Uses ELSED (Edge Drawing Line Segment Detector) to detect line segments, suitable for low-texture construction environments.
2. Band Estimation: Identifies the vertical image region where windows are likely to appear using brightness density and segment density.
3. Clustering & Pairing: Clusters vertical segments and pairs them to form window proposals, filtering out artifacts (e.g., safety equipment, fisheye edges).
4. Bearing Projection: Detected window bounding boxes are projected to azimuthal bearings using the Kannala–Brandt fisheye model.
Descriptor Population:
- Hit-Type Channel: Bins corresponding to detected windows are set to 0.5; others default to 1.0 (wall).
- Note: In this proof-of-concept, only the Hit-Type channel is fully populated from images. Range-based channels (0, 2, 3, 4) are subjects of ongoing work.

D. Pose Recovery

Matching: The visual descriptor is matched against the floor plan descriptor database using circular cross-correlation (via FFT). Due to the descriptor's rotation equivariance, a change in heading corresponds to a cyclic column shift.
Optimization: The algorithm maximizes cosine similarity over all candidates and shifts to find the best position and yaw.
Attitude Estimation: Roll and pitch are estimated separately using vanishing point analysis on vertical and horizontal lines detected by ELSED.

3. Key Contributions

Multi-Channel Semantic Descriptor: A novel 5-channel radial descriptor that jointly encodes geometry and structural semantics (wall vs. window) from 2D floor plans, moving beyond purely geometric approaches.
Cross-Modal Window Detection: A robust algorithm for detecting windows in fisheye images using line segment clustering and brightness verification, enabling the generation of the semantic "hit-type" channel for visual descriptors.
Proof of Concept: Successful demonstration of cross-modal matching between a floor plan and fisheye imagery, validating the feasibility of using structural semantic patterns for localization.

4. Experimental Results

The authors tested the system using the Hilti-Trimble SLAM Challenge 2026 dataset (a construction site environment).

Setup: A single known pose was used to generate the floor plan descriptor and extract the visual descriptor from dual fisheye frames.
Window Detection: The algorithm successfully detected ~300 line segments per image, identifying 18 windows (front) and 16 windows (back).
Matching Performance:
- Correlation: The circular cross-correlation between the camera and floor plan hit-type channels peaked at 0.9486 with a 0° shift, correctly recovering the relative heading.
- Agreement: At the correct alignment, 64% (231 out of 360) of the azimuth bins agreed between the two modalities.
- Discrepancies: Mismatches occurred in areas where the "as-built" state differed from the plan (e.g., windows behind walls not yet constructed) or where distant windows were segmented differently.
Attitude Estimation: The vanishing point method achieved approximately 2° agreement with ground truth for the dominant tilt axis.

5. Significance and Limitations

Significance:

Semantic Localization: COMPASS demonstrates that semantic information (distinguishing windows from walls) significantly enhances the discriminative power of localization descriptors in indoor environments.
Cost-Effective: It leverages widely available 2D floor plans and low-cost fisheye cameras, avoiding the need for expensive LiDAR sensors.
Construction Applicability: The method is specifically tailored for construction sites where floor plans are the primary source of truth, even if the physical site is incomplete.

Limitations & Future Work:

Current Scope: The current proof-of-concept relies primarily on the Hit-Type channel. This channel is highly discriminative near facades but may be uniform in interior corridors with no windows.
Range Channels: The geometric channels (range, gradient, inverse range, variance) are not yet fully populated from images. The authors note these are essential for disambiguating positions along the same facade and for robust localization in window-free interiors.
Next Steps: Future work involves completing the extraction of all five channels from imagery, implementing dense grid matching, and evaluating performance across full trajectories.