🤖 AI

Floorplan2Guide: LLM-Guided Floorplan Parsing for BLV Indoor Navigation

The paper proposes Floorplan2Guide, an LLM-driven navigation system for blind and low-vision users that converts floor plans into knowledge graphs to generate precise instructions, demonstrating that few-shot learning and graph-based spatial reasoning significantly outperform direct visual reasoning in improving navigation accuracy.

Original authors: Aydin Ayanzadeh, Tim Oates

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Aydin Ayanzadeh, Tim Oates

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to navigate a giant, unfamiliar maze, but you can't see the walls or the paths. For people with visual impairments, moving through indoor spaces like office buildings or hospitals is often like trying to solve this maze blindfolded. Current solutions often rely on expensive, heavy infrastructure (like special beacons everywhere) or complex cameras that need to be retrained for every single new building.

The paper "Floorplan2Guide" proposes a smarter, lighter way to solve this. Think of it as giving the user a super-smart GPS that understands blueprints.

Here is how the system works, broken down into simple steps:

1. The "Translator" (Turning Blueprints into a Map)

Imagine you have a static, 2D drawing of a building (a floorplan). To a computer, this is just a picture of lines and text. To a human, it's a map.

The Old Way: Computers used to need a team of engineers to manually measure every wall and door to turn that picture into a usable map.
The New Way (Floorplan2Guide): The researchers use a "Large Language Model" (an advanced AI that is very good at reading and understanding context). They feed the floorplan image to this AI. The AI acts like a translator, reading the drawing and converting it into a Knowledge Graph.
- Analogy: Think of the Knowledge Graph as a skeleton of the building. Instead of just seeing a picture of a room, the AI understands that "Room A" is connected to "Hallway B" through "Door C," and it knows exactly how far apart they are. It turns a flat image into a 3D mental map of connections.

2. The "Guide" (Generating Instructions)

Once the AI has built this "skeleton map," a user can ask, "How do I get to the restroom?"

Instead of just looking at the picture and guessing, the AI looks at its skeleton map. It traces the path from the starting point to the destination.
It then translates that path into simple, human-like spoken instructions: "Walk forward 10 steps, turn left at the intersection, and the restroom is on your right."
The Secret Sauce (Few-Shot Learning): The researchers found that if they show the AI a few examples of good directions first (like showing a student a few sample math problems before a test), the AI gets much better at giving directions. This is called "few-shot learning," and it made the system significantly more accurate.

3. The "Checkpoint" (ArUco Markers)

How does the system know where the user actually is in the real building?

The system uses small, square QR-code-like stickers called ArUco markers placed around the building (on walls, at intersections, etc.).
The user's phone camera scans these markers.
Analogy: Think of these markers as bus stops. When the user scans a marker, the system knows, "Ah, you are at Bus Stop #4." It then checks its "skeleton map" to see which instruction comes next. If the user wanders off the path, the system knows immediately and can say, "You've gone off course, let's get you back on track."

What Did They Find?

The researchers tested this system in a real building at their university (the Math & Psychology building) and on standard test datasets.

Better than just looking: When the AI used the "skeleton map" (Knowledge Graph) to give directions, it was 15.4% more accurate than when it tried to just look at the picture and guess. The map helped it avoid "hallucinations" (making up paths that don't exist).
The Best Model: Among the different AI models they tested, Claude 3.7 Sonnet was the best navigator.
Success Rates: With the right setup (using the "skeleton map" and showing the AI a few examples first), the system was successful about 92% of the time on short routes, 77% on medium routes, and 62% on long, complex routes.

The Bottom Line

This paper introduces a system that doesn't need to rebuild the world with expensive sensors. Instead, it takes a simple picture of a floorplan, uses a smart AI to turn it into a logical map of connections, and uses that map to guide blind or low-vision users step-by-step. It's like giving the user a digital tour guide that has memorized the building's blueprint and can tell them exactly where to go, using only a smartphone and a few stickers on the wall.

Note: The paper explicitly states that this system focuses on pathfinding and navigation instructions. It does not currently handle avoiding moving obstacles (like people walking by) or dynamic changes in the environment.

Technical Summary: Floorplan2Guide

Problem Statement

Indoor navigation remains a critical challenge for Blind and Low-Vision (BLV) individuals, who face significant barriers in orientation and acquiring spatial information. While infrastructure-based solutions (e.g., BLE beacons, RFID) offer precision, they require costly installation and maintenance. Conversely, lightweight, vision-based deep learning approaches often lack generalization to new environments, requiring extensive retraining for specific layouts and failing to adapt to dynamic lighting or unseen structures. Furthermore, traditional floorplan parsing methods rely on geometric heuristics and manual preprocessing, which are labor-intensive and prone to error. The core challenge addressed is the automatic construction of a robust spatial knowledge graph from static architectural floorplan images to enable reliable pathfinding and context-aware navigation without heavy infrastructure dependencies.

Methodology

The proposed Floorplan2Guide framework utilizes Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) to transform static floorplan images into machine-readable, navigable knowledge graphs. The system operates through a structured, multi-stage pipeline:

Preprocessing & Semantic Extraction:
- High-quality floorplan images undergo Optical Character Recognition (OCR) using tools like PaddleOCR, EasyOCR, or Tesseract to extract textual data (room numbers, labels).
- This textual data is combined with the visual input to enrich the model's understanding of the layout.
Spatial Knowledge Graph Generation:
- An MLLM (e.g., GPT-4o, Claude 3.7 Sonnet, LLaMA 3.2) processes the floorplan image and extracted text to generate a structured graph $G = (V, E, A, M_\eta, W, S)$ .
- Nodes ( $V$ ): Represent distinct rooms, intersections, or places, storing geometric centroids, semantic attributes (room type), and metadata.
- Edges ( $E$ ): Encode navigability and adjacency between nodes (e.g., doors, passages).
- Weights ( $W$ ): Capture distance and accessibility information, scaled to real-world dimensions.
- The system employs a system prompt to guide the LLM in extracting node maps and validating connectivity, utilizing refinement rounds to minimize extraction errors.
Navigation Instruction Generation:
- Unlike direct visual reasoning where the LLM interprets the raw image, this approach feeds the structured knowledge graph to the LLM.
- The LLM uses the graph's explicit topological and semantic relationships to generate step-by-step, human-readable navigation instructions (e.g., "walk 15 meters, turn left").
- The system prioritizes generating optimized directions based on the graph rather than relying solely on traditional shortest-path algorithms, aiming for natural language descriptions suitable for BLV users.
Localization via ArUco Markers:
- To bridge the gap between the digital graph and the physical environment, the system employs ArUco fiducial markers placed at key locations (entrances, intersections, rooms).
- Users scan these markers via a camera-equipped device. The marker's unique ID links to a specific node in the knowledge graph, allowing the system to localize the user's position.
- The system provides feedback: if the user is on the correct path, it repeats the last instruction; if they deviate, it alerts them. This study uses markers for localization checkpoints rather than dynamic rerouting.

Key Contributions

Novel Parsing Framework: Introduction of a solution that utilizes foundation models to parse floorplans by transforming spatial information into a knowledge graph, enabling accurate semantic extraction and structured representation for navigation.
Few-Shot Learning Integration: Demonstration that few-shot learning significantly improves navigation performance (measured by Success Rate) compared to zero-shot learning, enhancing spatial reasoning consistency without requiring model fine-tuning.
Real-World Validation: Successful validation of the system in a real-world environment (UMBC Math & Psychology Building, MP-1), showing that graph-based instructions specifying step counts and distances are effective for BLV users.

Experimental Results

The system was evaluated on the CVC-FP dataset (standardized benchmark) and in a real-world setting at UMBC. Three models were tested: GPT-4o, Claude 3.7 Sonnet, and LLaMA 3.2 Vision-Instruct 11B.

Impact of Few-Shot Learning:
- On the CVC-FP dataset, all models showed improvement with few-shot prompting. Claude 3.7 Sonnet achieved the highest success rates: 84.6% (short), 76.9% (medium), and 61.5% (long) paths.
- In real-world evaluations (MP-1), Claude 3.7 Sonnet under 5-shot prompting achieved the highest accuracy: 92.31% (short), 76.92% (medium), and 61.54% (long).
- Generally, few-shot learning improved Success Rates (SR) by 6–9% over zero-shot, particularly in complex, multi-turn routes.
Graph-Based vs. Direct Visual Reasoning:
- A comparative experiment (Experiment III) evaluated the impact of explicit graph representation against direct visual reasoning (zero-shot, no intermediate graph).
- Graph-based spatial structuring improved performance across all models. For instance, Claude 3.7 Sonnet improved from 76.92% to 84.62% on short routes when using the graph.
- Overall, the graph-based approach yielded a 15.4% higher success rate than direct visual reasoning, confirming that explicit topological encoding reduces hallucination and strengthens spatial consistency.

Significance and Claims

The paper claims that Floorplan2Guide offers a more precise and accessible solution for indoor navigation by leveraging the reasoning capabilities of LLMs combined with structured spatial data.

Infrastructure Independence: The system requires only a floorplan image and a camera-equipped device to scan ArUco markers, avoiding the high costs and maintenance of infrastructure-heavy systems like BLE beacons.
Enhanced Reliability: By converting floorplans into knowledge graphs, the system mitigates the "hallucination" errors common in direct visual reasoning, providing more reliable path planning.
User-Centric Design: The generation of step-by-step instructions with specific distances and step counts makes the output more accessible to BLV users compared to abstract directions.
Scalability: The approach demonstrates potential for generalization across different environments without the need for environment-specific retraining, provided the floorplan is available.

The authors acknowledge limitations, including the lack of dynamic obstacle avoidance and the need for broader user studies with BLV participants to fully validate robustness in diverse real-world settings. However, the results suggest that LLM-guided graph parsing is a viable and promising direction for assistive indoor navigation.