From Woofs to Words: Towards Intelligent Robotic Guide Dogs with Verbal Communication

1. Problem Statement

The paper addresses the limitations of current assistive robotics for the visually impaired. While biological guide dogs are effective, they are scarce, expensive to train, and have low graduation rates (<50%). Existing robotic guide dogs often rely on unidirectional physical cues (leash tension) or limited predefined voice commands, lacking the ability to engage in natural, multi-turn dialogue.

The core challenges identified are:

• Grounding Language: Connecting open-vocabulary user requests (e.g., "I'm thirsty") to specific, feasible navigation goals in a dynamic environment.
• Spatial Awareness: Providing the visually impaired handler with sufficient environmental context to make informed collaborative decisions, rather than blindly following the robot.
• Collaborative Decision Making: Enabling a team dynamic where the human and robot negotiate routes and destinations based on constraints (e.g., distance, obstacles).

2. Methodology

The authors propose a novel dialog system that integrates Large Language Models (LLMs) for natural language understanding and Symbolic Task Planners for reasoning about navigation constraints. The system operates in two distinct phases:

A. System Architecture

• LLM Agent: Acts as the dialog manager. It receives spoken requests (via speech-to-text), clarifies ambiguous intents, and formats outputs into a structured protocol.
• Task Planner (ASP): A symbolic planner (Answer Set Programming) runs in the background. It takes the LLM's interpreted intent and computes optimal action sequences (e.g., approach, gothrough, opendoor, goto). It calculates metrics such as travel time and the number of doors to be opened.
• Navigation Controller: Executes the generated plan using a global path planner and local obstacle avoidance, integrated with a quadruped robot (Unitree Go2).

B. Key Dialog Mechanisms

The system introduces two specific verbalization strategies:

1. Plan Verbalization (Pre-Navigation):
   • Instead of simply listing destinations, the system verbalizes the costs and constraints of different plans.
   • Example: "We can go to the kitchen (1 door, ~3 mins) or the water fountain (0 doors, ~1 min). Where would you like to go?"
   • This allows the user to make an informed choice based on their physical state or preferences.
2. Scene Verbalization (During Navigation):
   • Triggered by scene changes (e.g., entering a new room) or long silences.
   • Provides real-time descriptions of the surroundings (e.g., "We are approaching an office door") to maintain the user's spatial awareness.

C. Safety and Robustness

• Safeguard Module: A post-processing layer validates LLM outputs. If the LLM fails to follow the required format or suggests invalid locations, the system defaults to a safe, generic response.
• Wizard-of-Oz Control: In human trials, an expert operator controlled the robot's physical movements to ensure safety, isolating the evaluation to the dialog strategies.

3. Key Contributions

• Novel Dialog Framework: The first system to combine LLMs with symbolic planners specifically for robotic guide dogs, enabling multi-turn negotiation of navigation tasks.
• Dual-Verbalization Strategy: The introduction of Plan Verbalization (informing users of route costs before moving) and Scene Verbalization (maintaining spatial awareness during movement) to facilitate true human-robot collaboration.
• Human-Centric Evaluation: A real-world study with legally blind participants evaluating the system's utility, safety, and preference compared to minimal or scene-only verbalization.
• Robust Simulation Analysis: A large-scale simulation using GPT-4 to test system accuracy, efficiency, and robustness against noisy speech inputs.

4. Results

Human Study (Real-World)

• Participants: 7 legally blind individuals.
• Conditions Compared: Minimum Verbalization (control), Scene Verbalization only, and Ours (Scene + Plan).
• Findings:
  • The "Ours" condition received the highest ratings for Utility (4.83/5), Helpfulness, and Ease of Communication.
  • Users preferred the "Ours" condition over real guide dogs in hypothetical scenarios (Q6: 3.67 vs. lower scores for others).
  • Safety Perception: Rated slightly lower (3.83) than other conditions, attributed to initial unfamiliarity with walking alongside a robot rather than the system's logic.
  • Conclusion: Providing both plan details and scene context significantly improves user satisfaction and perceived effectiveness.

Simulation Analysis

• Accuracy & Efficiency (Hypothesis I): The full multi-turn LLM system achieved 94.8% accuracy in resolving ambiguous requests, outperforming keyword-based baselines while requiring fewer dialog turns (more concise).
• Robustness (Hypothesis II): The system remained highly accurate (losing only 5.2% accuracy) when user inputs were perturbed with noise (simulating speech recognition errors). Keyword-based systems failed completely under noise.
• Planner-Informed Efficiency (Hypothesis III): When plan information (travel costs) was included in the dialog, simulated users consistently chose closer destinations. This reduced total task time by 16.6% (277s vs. 230s) despite a slight increase in dialog length, proving that verbalizing constraints leads to more efficient navigation.

5. Significance

This work represents a paradigm shift from "robot as a follower" to "robot as a collaborative partner." By leveraging LLMs for natural language interaction and symbolic planners for grounded reasoning, the system empowers visually impaired users to make informed decisions about their navigation.

The study demonstrates that verbalizing the "why" and "how" of navigation (plans and scenes) is critical for trust and usability in assistive robotics. While challenges regarding perceived safety remain, the framework provides a scalable path toward intelligent, autonomous guide dogs that can adapt to complex environments and diverse user needs, potentially addressing the global shortage of biological guide dogs.