💻 computer science

Using Language Models as Closed-Loop High-Level Planners for Robotics Applications: A Brief Overview and Benchmarks

This paper empirically investigates practical strategies for integrating Large Language Models (LLMs) and Vision Language Models (VLMs) as closed-loop high-level planners in robotics, specifically analyzing the impact of control horizon and warm-starting to provide actionable recommendations for improving planning performance and robustness.

Original authors: Hao Wang, Sathwik Karnik, Bea Lim, Somil Bansal

Published 2026-04-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hao Wang, Sathwik Karnik, Bea Lim, Somil Bansal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a very smart, but slightly scatterbrained, robot how to make a fruit salad. You give the robot a big instruction: "Make a salad." The robot has a "brain" (a large language model) that understands words and pictures, but it doesn't have hands of its own. It has to tell a separate, simpler "hand" (a low-level controller) exactly what to do, like "pick up the apple" or "put the apple on the plate."

This paper is like a guidebook for the robot's brain, testing three different ways to give it instructions to see which method works best. The authors set up a kitchen with different levels of difficulty (from stacking simple boxes to making a complex salad with specific rules) and ran hundreds of experiments.

Here is the breakdown of their findings using simple analogies:

1. The "Open-Loop" vs. "Closed-Loop" Debate

The Analogy:

Open-Loop (The "Set and Forget" GPS): You tell the robot, "Go to the store, buy milk, and come home." The robot writes down this entire plan at the start and tries to follow it perfectly without looking around. If it trips over a rug or the store is closed, it keeps trying to walk to the store anyway, failing repeatedly.
Closed-Loop (The "Check-in" GPS): You tell the robot, "Go to the store." The robot takes one step, then stops to look around. "Okay, I'm at the door. Is the store open? Yes. Good, now I'll go in." It checks its progress constantly.

The Finding:
The paper found that the "Check-in" (Closed-Loop) method is almost always better. Even in a static kitchen where nothing changes, the robot's brain makes mistakes in its initial plan. By stopping to check its work after every step, the robot can fix its own mistakes before they ruin the whole task. The "Set and Forget" method fails much more often because it doesn't realize it's going off the rails until it's too late.

2. The "Control Horizon" (How often to check in)

The Analogy:
Imagine you are driving a car with a co-pilot (the robot's brain).

Short Horizon: The co-pilot checks the map and gives you a new direction after every single step you take.
Long Horizon: The co-pilot gives you a direction for the whole trip, or maybe just for the next few blocks, and only checks again if you hit a wall.

The Finding:
Intuitively, you might think checking the map after every single step (Short Horizon) would be the safest and most perfect way to drive. However, the paper found this isn't necessarily true.

Checking too often didn't make the robot significantly smarter or more successful.
Sometimes, checking too often actually gave the robot's brain too many chances to get confused or make a new mistake.
The Takeaway: You don't need to micromanage the robot after every tiny movement. Checking in occasionally (like after a few steps) works just as well as checking constantly, provided you have a good way to correct the robot when it messes up.

3. "Warm-Starting" (Giving the robot a hint)

The Analogy:

Cold Start: The robot fails to pick up an apple. You say, "Try again!" but you don't tell it why it failed or what it was just doing. The robot has to guess from scratch.
Warm-Start: The robot fails to pick up the apple. You say, "Try again! You just tried to grab the apple, but your hand was too far to the left. Try moving your hand to the right." You give it the previous plan and the specific error as a starting point.

The Finding:
This was the most important discovery. Giving the robot the "Warm-Start" (the previous plan and the error report) made a huge difference.

Without it, the robot often got stuck in a loop of failing repeatedly.
With it, the robot could learn from its immediate past mistakes and correct them.
In some difficult scenarios, the robot simply could not succeed without this "hint." It's like trying to solve a puzzle blindfolded versus having the picture on the box to guide you.

Summary of Recommendations

Based on these experiments, the authors suggest:

Always use the "Check-in" method (Closed-Loop): Don't just give the robot a one-time plan. Let it check its work as it goes.
Always use "Warm-Starting": When the robot replans, show it what it just tried and where it failed. This is crucial for success.
Don't over-check: You don't need to force the robot to re-plan after every single tiny movement. A moderate pace of checking is fine.
The "Brain" matters most: The specific AI model used (the "brain") matters more than how often you check its work. Some models are just naturally better at planning than others.

What the paper does NOT say:
The authors are careful to note that they only tested robots in a controlled, static environment (nothing moved on its own). They did not test this in chaotic, real-world scenarios like a busy street or a hospital. They also didn't test different types of robot "hands" (only simple pick-and-place actions), though they believe their advice would likely apply there too. Their main goal was simply to figure out the best way to talk to the robot's brain right now.

1. Problem Statement

While Large Language Models (LLMs) and Vision-Language Models (VLMs) have demonstrated impressive open-world reasoning capabilities, their deployment as closed-loop high-level planners in robotics remains challenging.

The Gap: Existing literature often treats LLMs/VLMs as black-box planners without rigorously investigating how specific control strategies (like replanning frequency or initialization) affect performance.
The Challenge: In real-world robotic tasks, open-loop execution is prone to failure due to:
1. Incorrect high-level plans generated by the model.
2. Failures in low-level execution (e.g., motion planning collisions).
3. Unpredictable environmental changes.
Core Question: How can LLMs/VLMs be effectively utilized as closed-loop planners? Specifically, how do control horizon (replanning frequency) and warm-starting (using previous plans as context) impact the robustness, geometric reasoning, and logical reasoning of these systems?

2. Methodology

The authors frame the problem within Model Predictive Control (MPC), where the language model acts as the solver for an optimal control problem. The objective is defined by the user prompt, constraints are implicit or explicit in the prompt, and the model's internal knowledge serves as the dynamics.

Experimental Setup

Task Environments: Four environments of increasing complexity were designed:
1. CUBE-EASY: Basic box manipulation.
2. YCB-EASY: Manipulation of household objects (YCB set).
3. YCB-MEDIUM: Multi-goal placement (fruit on plate, objects in basket).
4. YCB-HARD: Complex reasoning (making a fruit salad with property constraints like "sour fruit").
Models Tested: Three VLMs were evaluated: GPT-4.1-mini, Gemini-2.5-flash, and Llama-4-Maverick-17B.
Planner Variants:
- Open-Loop (OL): Generates a plan once; no replanning unless a failure occurs.
- Closed-Loop (CL): Replans based on updated state.
  - Control Horizons:
    - Short: Replan every 2 steps.
    - Half: Replan every $k/2$ steps ( $k$ = task length).
    - Full: Replan only after task completion or failure (equivalent to OL in static environments).
  - Warm-Starting: Providing the model with the previous plan and execution status (success/failure) during replanning vs. Non-Warm-Starting (NWS) (starting fresh without prior context).
Metrics:
- Task Completion Rate: Success in satisfying all logical and geometric constraints.
- Goal Achieved Rate: Success in geometric placement (ignoring logical constraints).
- Logical Reasoning Metrics: Correct Final Logical Plan Rate, Positive/Negative Logical Correction Rates.

3. Key Contributions

Empirical Benchmarking: The first systematic study isolating the effects of control horizon and warm-starting on VLM-based embodied planning.
MPC Framework Application: Conceptualizing LLM planning as an optimal control problem to standardize the evaluation of replanning strategies.
Counter-Intuitive Findings: Challenging the common assumption that "more frequent replanning is always better" and demonstrating the critical necessity of warm-starting.
Public Release: The authors released the full implementation, benchmarks, and project website to facilitate reproducibility.

4. Key Results & Observations

Observation 1: Closed-Loop is Superior to Open-Loop

Geometric Reasoning: Closed-loop planners significantly outperformed open-loop planners in Goal Achieved Rate (average improvement of 21.7%). The ability to replan allowed the model to correct geometric misplacements that would otherwise cause task failure.
Logical Reasoning: Closed-loop planners did not show a statistically significant advantage in logical reasoning over open-loop planners in this specific setup. This is attributed to the fact that the "Full" control horizon (replanning only on failure) offered only one chance to correct logical errors, which was often insufficient.

Observation 2: Shorter Control Horizon is Not Always Better

Contrary to the intuition that frequent replanning (short horizon) yields better robustness, the Short control horizon (replan every 2 steps) did not consistently outperform Half or Full horizons.
Statistical Significance: In only 2 out of 12 scenarios did the Short horizon show a statistically significant improvement in Task Completion Rate.
Reasoning: While frequent replanning offers more correction opportunities, it also increases the probability of the VLM introducing negative logical corrections (making the plan worse). The performance is more heavily dependent on the capability of the specific VLM used rather than the frequency of replanning.

Observation 3: Warm-Starting is Critical

Overall Performance: Warm-starting (providing the previous plan and execution status) consistently improved performance. Warm-started variants outperformed non-warm-started variants in 21 out of 24 scenarios (average improvement of 28.2%).
Geometric Reasoning: Warm-starting improved Goal Achieved Rates by 31.7%. Explicit feedback on where and how a previous plan failed allowed the model to correct specific geometric errors.
Logical Reasoning: Warm-starting significantly reduced Negative Logical Corrections (making fewer mistakes). However, it slightly reduced Positive Logical Corrections (the model became more biased toward the previous, potentially incorrect, plan).
Failure Mode: In several scenarios (e.g., Llama on YCB-MEDIUM), non-warm-starting planners completely collapsed (0% success), whereas warm-starting enabled successful task completion. This suggests that without context, the probability of stringing together a sequence of successful actions decreases geometrically with task length.

5. Significance and Recommendations

Robustness over Frequency: The paper argues that for VLM-based robotics, warm-starting is a more critical factor for success than the frequency of replanning.
Practical Guidelines:
1. Always use Closed-Loop: Even in static environments, closed-loop planning is preferred to correct VLM inference errors.
2. Prioritize Warm-Starting: Always provide the model with the previous plan and execution status. Without this, complex tasks often fail.
3. Control Horizon: Choose a horizon that makes the system sufficiently reactive to the task, but do not assume that the shortest possible horizon (replan every step) is optimal.
4. Model Selection: The choice of VLM is the dominant factor in performance; different models exhibit vastly different reasoning capabilities.

Conclusion

This work establishes that while LLMs/VLMs are powerful for high-level planning, their reliability in robotics depends heavily on how they are integrated into the control loop. The study demonstrates that warm-starting is essential for bridging the gap between VLM reasoning and physical execution, while the benefits of frequent replanning are nuanced and model-dependent. These findings provide a foundational benchmark for developing more robust embodied AI systems.