LLM-Advisor: An LLM Benchmark for Cost-efficient Path Planning across Multiple Terrains

The paper introduces LLM-Advisor, a prompt-based framework that leverages large language models as non-decisive post-processing advisors to significantly improve the cost efficiency of path planning across diverse terrains without modifying underlying planners, while addressing hallucination risks and demonstrating superior performance over zero-shot LLM approaches.

The Gist

Imagine you are sending a delivery drone to drop off a package. You have a map, a starting point, and a destination. The drone's computer (the "planner") needs to figure out the best route.

Usually, the computer just looks for the shortest distance, like a crow flying in a straight line. But in the real world, the ground isn't flat. Some parts are smooth pavement (easy to fly over), some are thick mud (hard to fly over), and some are swamps (impossible to fly over).

If the drone flies over the mud, it uses way more battery. If it flies over the pavement, it saves energy. The goal isn't just to be fast; it's to be cost-efficient (saving battery).

The Problem: The "Smart" Computer vs. The "Smart" Advisor

The paper introduces a new way to solve this using Large Language Models (LLMs)—the same AI technology that powers chatbots like me.

1. The Old Way (The Robot's Brain): Traditional path-planning algorithms (like A* or RRT*) are like a robot with a very strict, local view. They look at the map grid by grid. They are great at avoiding walls, but they often miss the "big picture." They might take a slightly longer route that avoids a muddy patch, but they might also take a "short" route that accidentally cuts through a swamp because they are too focused on the immediate next step.
2. The New Problem: Researchers tried asking the AI chatbot to just "draw the best path" from scratch. It failed miserably. The AI got confused, hallucinated (made up paths that went through walls), or gave bad advice. It's like asking a brilliant philosopher to do a math problem; they understand the concepts but can't do the precise calculations.

The Solution: LLM-Advisor (The "Second Opinion")

Instead of replacing the robot's brain, the authors created LLM-Advisor. Think of it as a wise, experienced navigator sitting next to the robot.

Here is how it works:

1. The Robot does the heavy lifting: The robot's standard algorithm (A*) draws a path first. It's a valid path, but maybe not the best one.
2. The Advisor reviews the work: The LLM-Advisor looks at the robot's path and the map. It says, "Hey, I see you took a shortcut through that rocky area. That costs a lot of energy. If you took this slightly longer detour around the rocks, you'd save 20% of your battery."
3. The Robot decides: The robot doesn't blindly obey. It checks the suggestion. If the suggestion makes sense and saves energy, it takes it. If the suggestion is nonsense, it ignores it.

How They Fixed the AI's "Hallucinations"

AI chatbots are known for "hallucinating"—making things up confidently. If you ask an AI to draw a path, it might draw a line that goes through a mountain because it doesn't "see" the mountain the way a robot does.

To stop this, the authors used two clever tricks:

• The "Describe, Don't Just List" Trick: Instead of asking the AI to spit out a list of coordinates (which is where it gets confused), they asked it to describe the path in words first ("Go from the start, turn left around the big rock, then head straight to the goal"). This forces the AI to think logically about the steps before giving numbers.
• The "Show Me an Example" Trick (RAG): They showed the AI examples of good paths from similar situations before asking it for advice. It's like showing a student a solved math problem before asking them to solve a new one. This keeps the AI grounded in reality.

The Results

The team tested this on two new datasets (one made of computer-generated maps, one from real outdoor photos).

• The AI alone? Terrible at planning paths on its own.
• The Robot alone? Good, but often misses energy-saving shortcuts.
• Robot + LLM-Advisor? Excellent.
  • For paths made by the standard robot algorithm, the Advisor improved the energy efficiency in 72% of cases.
  • It worked even in "hard" scenarios with complex, winding terrain.
  • It rarely gave bad advice because of the safety checks (hallucination mitigation).

The Big Takeaway

This paper proves that you don't need to replace your robot's brain with a giant AI. Instead, you can use AI as a smart consultant. The robot handles the precise math and safety, while the AI provides the "big picture" wisdom to save energy and time. It's the best of both worlds: the precision of a calculator and the intuition of a human expert.

Technical Summary

Here is a detailed technical summary of the paper "LLM-Advisor: An LLM Benchmark for Cost-efficient Path Planning across Multiple Terrains."

1. Problem Statement

The paper addresses the challenge of cost-efficient path planning for robots navigating heterogeneous terrains (e.g., outdoor environments with varying ground types like grass, rock, water, or mud).

• Limitations of Classical Planners: Traditional search-based (e.g., A*) and sampling-based (e.g., RRT*) planners operate on discretized maps with limited connectivity (e.g., 8-connected grids). While they guarantee optimality within the discrete graph, they often fail to capture long-range cost trends or continuous terrain structures, leading to suboptimal paths with unnecessary detours and high energy consumption.
• Limitations of Direct LLM Planning: While Large Language Models (LLMs) possess strong global context understanding, they struggle with spatial reasoning and precise coordinate generation. Directly using LLMs to generate paths (zero-shot planning) often results in hallucinations (invalid paths, collisions) or failure to find valid routes.
• Goal: The authors aim to improve path cost efficiency without modifying existing planners or retraining models, specifically targeting scenarios where recharging is difficult (e.g., disaster zones, space exploration).

2. Methodology: LLM-Advisor

The core proposal is LLM-Advisor, a planner-agnostic, post-processing framework that uses LLMs as non-decisive advisors to refine paths generated by classical algorithms.

A. Workflow

1. Input: The system takes a start point, goal point, a terrain cost grid description, and a path generated by a base planner (e.g., A*, RRT*, or LLM-A*).
2. Evaluation: The LLM analyzes the provided path against the terrain description to determine if a more cost-efficient route exists.
3. Refinement: If the current path is suboptimal, the LLM suggests a new coordinate list. If the path is already optimal, it confirms this.
4. Output: The system outputs either a confirmation or a refined path list.

B. Prompt Design

The prompt is modular and includes:

• Terrain Description: A text summary of high-cost regions and obstacles (e.g., "High-cost area with cost 5 is located between coordinates...").
• Path Description: A detailed breakdown of the existing path segments, including the terrain cost of each point.
• Evaluation Directive: Instructions to compare the current path cost against potential alternatives and output in a strict format.

C. Hallucination Mitigation Strategies

To address the LLM's tendency to generate invalid coordinates or "hallucinate" paths, two strategies are introduced:

1. DescPath (Descriptive Path): Instead of asking the LLM to output raw coordinate lists (which often leads to errors), the LLM is prompted to generate a descriptive narrative of the movement (e.g., "From (x1, y1) it goes to (x2, y2)..."). This structure reduces coordinate hallucinations.
2. RAG (Retrieval-Augmented Generation): The system retrieves a similar historical path planning example (terrain description + path) from a database and includes it in the prompt as a reference. This provides context and grounding for the LLM's reasoning.

3. Key Contributions

1. LLM-Advisor Framework: A novel, plug-and-play method that leverages LLMs as advisors rather than primary planners. It improves cost efficiency without altering the underlying planner's logic or requiring retraining.
2. Hallucination Mitigation: The introduction of DescPath and RAG strategies, which significantly reduce the rate of physically invalid path suggestions.
3. New Datasets:
   • MultiTerraPath: A synthetic dataset with 2,000 maps (Easy and Hard subsets) featuring varying terrain costs and complex topologies (convex vs. irregular/concave regions).
   • RUGD v2: A real-world dataset derived from the RUGD semantic segmentation dataset, covering four outdoor scenes (Creek, Park, Trail, Village) with synthetically assigned terrain costs.
4. New Evaluation Metric (SCP): The authors propose Success weighted by Cost-aware Path Length (SCP). Unlike standard Success weighted by Path Length (SPL), which assumes uniform costs, SCP evaluates paths based on the actual traversal cost in heterogeneous environments.

4. Experimental Results

The authors evaluated LLM-Advisor (using GPT-4o) against A*, RRT*, and LLM-A* on both datasets.

• Performance Improvement:
  • A Paths:* 72.37% of paths were improved.
  • RRT Paths:* 69.47% of paths were improved.
  • LLM-A Paths:* 78.70% of paths were improved.
  • Success Rate (SR): LLM-Advisor achieved a 99.10% success rate on the MultiTerraPath Easy subset, significantly outperforming direct LLM planning (which had ~43% SR).
• Zero-Shot Limitations: State-of-the-art LLMs (GPT-4o, GPT-4-turbo, Gemini-2.5-Flash, Claude-Opus-4) performed poorly in zero-shot path planning (Improvement Ratio < 24%), confirming their inability to handle spatial reasoning tasks without assistance.
• Ablation Studies:
  • Detailed vs. Brief Descriptions: Providing detailed path descriptions significantly improved the Relative Precision (RP) of suggestions.
  • Mitigation Strategies: Combining DescPath and RAG reduced the Hallucination Rate (HR) from 3.30% to 0.70% while increasing the Improvement Ratio.
• Robustness: The method demonstrated robustness in challenging environments, including blurry images and overexposed scenes (RUGD v2), and effectively handled "Hard" subset maps with complex, connected terrain.

5. Significance and Conclusion

• Practical Applicability: LLM-Advisor offers a low-risk, cost-effective solution for real-world robotics where energy efficiency is critical. It allows legacy systems or fixed-discretization planners to benefit from the global reasoning capabilities of LLMs without the risk of replacing the planner entirely.
• Paradigm Shift: The paper argues against using LLMs as end-to-end planners for spatial tasks. Instead, it advocates for a hybrid approach where LLMs act as high-level validators and refiners, leveraging their context understanding while relying on classical algorithms for geometric validity.
• Future Work: The authors plan to fine-tune LLMs on domain-specific path planning data to further enhance accuracy and adaptability to ambiguous inputs.

In summary, LLM-Advisor successfully bridges the gap between the global reasoning of LLMs and the geometric precision of classical planners, achieving significant cost reductions in path planning while effectively mitigating the hallucination issues that typically plague LLM-based spatial reasoning.