💻 computer science

From Uncertainty to Determinism: Coarse-to-Fine Visual Floorplan Localization without Ray Matching

This paper proposes a coarse-to-fine visual floorplan localization framework that eliminates the need for resource-intensive ray matching by using an image-conditioned pose diffusion model to resolve multimodal pose ambiguities in a coarse stage, followed by a localized refiner for precise sub-meter adjustments, achieving state-of-the-art performance on S3D and ZInD benchmarks.

Original authors: Shiyong Meng, Bolei Chen, Ping Zhong, Yang Wan, Rongzhi Wang, Jiazhi Xia, Jianxin Wang

Published 2026-07-30

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Shiyong Meng, Bolei Chen, Ping Zhong, Yang Wan, Rongzhi Wang, Jiazhi Xia, Jianxin Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking through a giant, endless maze made entirely of identical white rooms. You pull out your phone to ask, "Where am I?" but the GPS signal is dead. You take a picture of the room, but every single room looks exactly the same: a white wall, a door, maybe a window. If you tried to find your spot by comparing your photo to a map, you'd get confused because the photo matches dozens of different places on the map at once. This is the daily struggle for robots and augmented reality glasses trying to navigate indoors. They need to match a rich, colorful photo of the world with a simple, flat, black-and-white drawing of the floorplan. The problem is that these two things look nothing alike, and the repeating patterns of buildings make it a guessing game with many possible answers. Scientists have been trying to solve this "Where am I?" puzzle for years, hoping to let robots walk through our homes and offices without getting lost or needing massive, heavy 3D maps of every single brick.

The paper you are about to read tackles this exact problem with a clever new strategy called CF2Loc. Instead of trying to force a perfect, one-to-one match between a photo and a map (which often fails when rooms look alike), the authors suggest a two-step "coarse-to-fine" approach. Think of it like finding a specific house in a huge city. First, you don't look for the exact house number; you just guess a few neighborhoods where the house might be. This is the "coarse" stage. Then, once you have a few good guesses, you zoom in on just those neighborhoods to find the exact front door. This is the "fine" stage.

The authors argue that the old way of doing this—trying to predict invisible "rays" of light from the photo to the map—is like trying to solve a puzzle by first turning the picture into a blurry sketch and then comparing that sketch to the map. They say this loses too much detail and gets stuck when there are too many possibilities. Instead, their new method uses a "diffusion model," which is a type of AI that starts with a bunch of random guesses (like throwing darts blindfolded at a map) and slowly cleans them up, one by one, until they land on the most likely spots. It's like starting with a cloud of fog and slowly letting the wind blow the fog away to reveal a few clear islands of truth.

Once the AI has found these few "islands" of possible locations, it switches to a "local refiner." This is a super-focused tool that looks only at the small patch of the map around each island. Because the area is so small, the confusing repeating patterns disappear, and the AI can pinpoint the exact location with sub-meter precision (meaning within a few feet). The result is a system that is faster, more accurate, and doesn't need to pre-calculate massive databases of light rays for every building. In tests on huge datasets of synthetic houses and real-world homes, this new method beat all the previous best attempts, proving that sometimes, guessing a few possibilities and then refining them is much better than trying to calculate the perfect answer immediately.

From "Maybe Here" to "Definitely Here"

The core idea of this paper is to stop trying to force a single, perfect answer out of a messy situation and instead embrace the confusion first. The authors call their method CF2Loc (Coarse-to-Fine Visual Floorplan Localization).

The Problem with the Old Way
Previous methods tried to solve this by predicting "rays." Imagine shining a flashlight from your camera and drawing a line to where the wall is on the map. The computer then tries to match these lines to the map. The paper argues this is a bad idea for two main reasons:

It loses information: Turning a rich, colorful photo into a few simple lines throws away all the subtle details that could help distinguish one room from another.
It gets stuck: If a room looks like ten other rooms, the "ray" method has to guess which one is right immediately. If it guesses wrong, it fails. It treats the problem like a math equation with only one answer, but the real world often has many.

The New Strategy: A Two-Step Dance
The authors propose a workflow that moves from "uncertainty" to "determinism."

Step 1: The Coarse Stage (The Foggy Guess)
Instead of guessing one spot, the AI uses a pose diffusion model. Think of this as a magical fog machine. You start with a cloud of random particles scattered all over the map. The AI then acts like a wind, gently pushing these particles. Over several steps, the particles that are in the wrong places drift away, while the particles that are in the right places (the "modes") clump together.
- How it works: The AI looks at the photo and the floorplan and asks, "If I were standing here, what would I see?" It doesn't pick one spot; it finds all the spots that make sense.
- The result: Instead of one wrong answer, the AI produces a small list of "candidate" locations (like 5 or 10 spots) where the robot could be.
Step 2: The Fine Stage (The Zoom-In)
Now that the AI has a shortlist of possibilities, it switches to a local refiner. For each candidate spot, it crops out a tiny 5-meter by 5-meter piece of the map centered on that spot.
- Why this helps: In a huge map, a hallway might look like ten other hallways. But if you zoom in on just one 5-meter patch, the unique details (like a specific door frame or a corner) become obvious. The confusion disappears.
- The result: The AI calculates a tiny "residual" (a small correction) to move the candidate spot to the exact right place. It's like taking a blurry photo and sharpening just the face in the center.

What They Found
The team tested this on two major datasets: S3D (a huge collection of 3,500 synthetic houses) and ZInD (a real-world dataset of 1,575 homes).

Accuracy: Their method beat the previous best results significantly. On the S3D dataset, they improved the accuracy of finding a spot within 0.5 meters from about 37.5% to 64.4%. On the real-world ZInD dataset, they boosted the 0.5-meter accuracy from 11.1% to 45.5%.
Speed: Because they don't have to pre-calculate millions of "rays" for every building, their system is much faster. They found that they could get great results with just 10 steps of their diffusion process, making it fast enough for real-time use.
Robustness: The system works even when the floorplan is just a simple black-and-white drawing (geometric) or when it includes labels like "kitchen" or "bedroom" (semantic).

What They Rejected
The authors explicitly argue against the idea that we need to predict intermediate "rays" to solve this problem. They show that trying to compress a photo into a ray representation creates a bottleneck that loses too much information. They also reject the idea that we need massive, pre-computed databases of map features for every single building. Their method works "on the fly" without needing to look up tables or render 3D models beforehand.

The Bottom Line
This paper suggests that when you are trying to find your way in a confusing, repetitive world, it's better to cast a wide net, find a few likely spots, and then zoom in to get the details, rather than trying to calculate the perfect answer in one go. By using a "coarse-to-fine" approach, the authors have created a system that is more accurate, faster, and more flexible than the current state-of-the-art methods, proving that sometimes, being a little uncertain at first is the key to finding the exact truth.

Technical Summary: From Uncertainty to Determinism: Coarse-to-Fine Visual Floorplan Localization without Ray Matching

Problem Definition

Visual Floorplan Localization (FLoc) aims to estimate the camera pose $p = (x, y, \theta)$ of an egocentric RGB observation by aligning it with a lightweight 2D floorplan map. This task is fundamentally challenged by cross-modal information asymmetry (rich visual features vs. minimalist structural maps) and repetitive indoor layouts (symmetric rooms, long corridors). These factors create a multimodal pose distribution, where a single visual observation corresponds to multiple spatially distinct, plausible locations.

Existing approaches, particularly ray-matching-based methods, attempt to solve this by predicting sparse geometric or semantic rays and matching them against pre-computed map representations. However, the authors identify three critical limitations in this paradigm:

Information Bottleneck: Compressing rich visual observations into sparse rays discards subtle appearance and structural cues.
Deterministic Failure: Front-end ray predictors often map observations to rays via one-to-one functions, failing to capture the inherent multimodal nature of the pose distribution.
Scalability Issues: These methods require heavy offline preprocessing to generate map-specific ray databases and exhaustive test-time matching, hindering real-time application.

Methodology: CF2Loc

The paper proposes CF2Loc, a rendering-free, coarse-to-fine generative localization framework that bypasses intermediate ray representations. The core insight is to treat FLoc as a direct conditional pose distribution estimation problem ( $p(p | I, M)$ ) rather than an indirect ray-matching task. The framework consists of two cascaded stages:

1. Coarse Stage: Global Pose Diffusion

To handle global multimodal uncertainty, the authors design an image-conditioned pose diffusion model.

Feature Encoding: Visual features are extracted via a frozen ViT backbone, while floorplan features are encoded via a convolutional encoder. Cross-attention fuses these modalities to create observation-conditioned map tokens.
Diffusion Process: Instead of predicting rays, the model parameterizes the continuous pose distribution in the space $\tilde{p} = (x, y, \cos\theta, \sin\theta)$ . It follows a standard Denoising Diffusion Probabilistic Model (DDPM) formulation, progressively injecting Gaussian noise into the pose representation and training a network to predict the noise given the timestep, image descriptor, and fused map tokens.
Hypothesis Generation: During inference, the model initializes pose particles from Gaussian noise and iteratively denoises them to sample a set of plausible candidate modes $\{p^k_c\}$ . Kernel Density Estimation (KDE) is used to extract the $K$ most probable modes from the particle cloud.
Augmentation: To prevent overfitting to absolute orientations, the training data undergoes joint map-pose rotation augmentation (discrete $90^\circ$ rotations).

2. Fine Stage: Local Pose Refinement

Once coarse candidates are identified, the search space is restricted to candidate-centered local floorplan crops ( $5m \times 5m$ ), where structural ambiguities are largely eliminated.

Local Refiner: A lightweight network predicts bounded sub-meter pose residuals ( $\Delta p = \Delta x, \Delta y, \Delta \theta$ ) relative to the coarse candidate.
Canonicalization: The local crop is rotated to align with the candidate's orientation, simplifying the alignment task to a unimodal residual regression.
Prediction Heads: The refiner outputs position logits, heading vectors, and a confidence score. The final pose is computed as $p_r = p_c \oplus \Delta p$ .
Training: The refiner is trained independently (with the diffusion model frozen) using a multi-task loss involving SmoothL1 for residuals, cross-entropy for spatial heatmaps, and binary cross-entropy for confidence scoring. Training candidates are generated by perturbing ground-truth poses to simulate varying levels of coarse localization error.

Key Contributions

Direct Pose Distribution Estimation: The paper formulates visual FLoc as a direct inference of the conditional pose distribution, bypassing the information-lossy intermediate ray prediction step.
Coarse-to-Fine Generative Framework (CF2Loc): A novel architecture combining a global pose diffusion model for multimodal hypothesis generation and a localized pose refiner for deterministic sub-meter correction.
Elimination of Offline Preprocessing: The method requires no offline map preprocessing (e.g., ray rendering) or test-time lookup tables, relying solely on the learned generative model and lightweight refinement.
State-of-the-Art Performance: The approach achieves superior accuracy and robustness on both synthetic (S3D) and real-world (ZInD) benchmarks, outperforming existing ray-matching and end-to-end regression baselines.

Experimental Results

The framework was evaluated on the Structured3D (S3D full) and Zillow Indoor Dataset (ZInD) benchmarks.

Performance on S3D (full): Without semantic floorplans, CF2Loc achieved a 73.7% recall at 1.0m and 72.6% at 1.0m/30°, significantly outperforming the previous best baseline (SceneAligner) which scored 53.8% and 51.6% respectively. With semantic floorplans, it reached 79.1% (1.0m) and 78.4% (1.0m/30°), surpassing the semantic baseline FoD.
Performance on ZInD: On this real-world dataset, CF2Loc improved the 0.5m recall from 11.1% (LASER) to 45.5% and the 1.0m recall to 57.3%.
Efficiency: The method achieves high accuracy with minimal inference latency. It reaches over 70% recall at 1.0m using only 5 diffusion denoising steps and converges rapidly around 10 steps. It demonstrates a superior accuracy-latency trade-off compared to ray-matching methods, reducing inference time by nearly 4 $\times$ while improving recall by over 15 percentage points.
Ablation Studies: Experiments confirmed the necessity of mode selection (avoiding naive averaging), joint rotation augmentation (for orientation robustness), and local patch canonicalization (for precise residual regression).

Significance and Claims

The paper claims that CF2Loc represents a paradigm shift from indirect ray-matching to direct pose distribution estimation. By bridging global multimodal uncertainty modeling with local unimodal determinism, the framework effectively handles the "ill-posed" nature of visual floorplan localization caused by repetitive layouts and cross-modal asymmetry.

The authors emphasize that their approach:

Eliminates the need for expensive offline map preprocessing and test-time exhaustive matching, making it more scalable and suitable for real-time applications.
Maintains robustness in ambiguous environments where deterministic regressors fail due to mode collapse.
Provides a scalable, rendering-free alternative to traditional ray-matching paradigms, applicable to both geometric and semantic floorplans.

The work positions itself not as a mere incremental improvement, but as a fundamental rethinking of the visual FLoc pipeline, leveraging generative inference to solve multimodal localization challenges without the computational overhead of ray databases.