Agentic LLM Planning via Step-Wise PDDL Simulation: An Empirical Characterisation

This paper introduces PyPDDLEngine, an open-source PDDL simulation engine that enables agentic LLM planning via step-wise interaction, demonstrating that while this approach yields a modest 3% success rate improvement over direct LLM planning on Blocksworld tasks, it incurs significantly higher costs and lacks the external verification mechanisms that drive success in other coding agent applications.

The Gist

The Big Question: Can AI "Think" Its Way Through a Puzzle?

Imagine you have a robot that needs to solve a complex puzzle, like stacking blocks to build a tower in a specific order. This is called Task Planning.

For decades, we've used "Classical Planners" (like a super-organized librarian) to solve these puzzles. They follow strict rules and math to find the perfect path.

Recently, we've started using Large Language Models (LLMs) (like advanced chatbots) to do the same thing. These chatbots are great at writing stories and coding because they've read almost everything on the internet. But can they actually plan a robot's moves, or are they just guessing based on what they've read before?

This paper asks: If we let the AI chatbot "play" the game step-by-step, checking its moves as it goes, will it get better at planning?

---

The Experiment: Two Ways to Play

The researchers set up a test using 102 different "Blocksworld" puzzles (ranging from easy to very hard). They compared four different players:

1. The Speedster (Classical Planner): A traditional, math-based robot that solves the puzzle instantly and perfectly.
2. The "One-Shot" Chatbot: An AI that tries to write down the entire solution in one go, like writing a whole essay without editing. If it gets it wrong, it starts over from scratch.
3. The "Agent" Chatbot: An AI that plays the game one move at a time. After every move, it asks the game engine, "Did that work? What does the world look like now?" If it messes up, it can say, "Oops, let's reset and try a different path."
4. The "Refined" Speedster: A classical robot that finds a solution quickly, then spends the rest of the time trying to make it shorter and better.

The Tools: PyPDDLEngine

To make the "Agent" Chatbot work, the researchers built a new tool called PyPDDLEngine.

• Analogy: Imagine a video game console that doesn't just show you the screen, but also lets you press buttons to pause, rewind, and check your inventory after every single step. This tool gave the AI a "live" view of the game so it could react in real-time.

---

The Results: What Happened?

1. The Success Rate (Did they finish the puzzle?)

• The Classical Speedster: Won 85% of the time. It's the reliable champion.
• The "One-Shot" Chatbot: Won 64% of the time.
• The "Agent" Chatbot: Won 67% of the time.

The Takeaway: The "Agent" Chatbot did slightly better (3% more wins) than the "One-Shot" version. But it cost 5.7 times more in computer processing power (tokens) to get those extra wins. It's like hiring a team of 5 people to solve a puzzle that one person could solve, just to get 3 extra points.

2. The Plan Quality (How efficient were the moves?)

Here is the weird part.

• The Classical "Refined" Speedster spent its whole time trying to make the solution shorter.
• Surprisingly, both Chatbot versions often found shorter solutions than the Speedster, even though the Speedster was trying harder to optimize.
• Why? The researchers think the Chatbots aren't actually "thinking" or "planning." They are memorizing. Since Blocksworld puzzles are famous and appear in millions of books and websites, the AI is likely just recalling the answer from its training data, like a student who memorized the textbook answers rather than learning the math.

3. The "Early Exit" Problem

The "Agent" Chatbot had a strange habit. Sometimes, it would look at the puzzle, decide, "This is impossible," and quit before the time ran out.

• The Twist: On some of these puzzles, the "One-Shot" Chatbot actually solved it!
• The Lesson: The Agent Chatbot was bad at judging its own progress. It thought it was stuck when it wasn't.

---

The Big Insight: Why Coding Agents Win, but Planning Agents Struggle

The paper makes a crucial distinction between Coding Agents and Planning Agents.

• Coding Agents (The Success Story): When an AI writes code, it runs the code. If it fails, the computer gives a clear error message: "Line 5 has a missing semicolon." This is external feedback. The computer tells the AI exactly what is wrong. The AI can fix it, run it again, and get better.
• Planning Agents (The Struggle): In this Blocksworld game, the AI moves a block. The game says, "Okay, the block is moved." It doesn't say, "You are moving away from the goal!" or "You are stuck in a loop."
  • Analogy: Imagine playing a maze game where the walls don't tell you if you're going the right way. You just have to guess if you're getting closer. Without a clear "You're wrong!" signal, the AI is just guessing in the dark.

The Conclusion

The researchers conclude that interaction alone isn't enough.

Giving an AI a "live" game to play (step-by-step) helps a little bit, but not as much as we hoped. The reason coding agents are so successful is that the computer gives them honest, objective feedback (errors). In pure planning, the AI has to judge its own progress without a referee.

The Final Metaphor:

• Classical Planners are like a GPS: They calculate the perfect route mathematically.
• LLMs are like a tourist who has read a travel guide. If the destination is famous (like Blocksworld), they can recite the route from memory.
• Agentic LLMs are like that tourist walking the streets, asking for directions. But if the locals (the game engine) only say "You walked forward" without telling you if you're getting closer to the hotel, the tourist will eventually get lost, even if they are trying their best.

The Future: For robots to truly plan in the real world, we need to build systems that give the AI clear, honest signals about whether it is making progress, not just raw data about what happened.

Technical Summary

1. Problem Statement

The paper addresses the open question of whether Large Language Models (LLMs) can serve as viable task planners for autonomous systems, either as alternatives to or complements for classical symbolic planners. While LLMs excel at generating structured sequences from natural language, their reliability in symbolic domains (like Planning Domain Definition Language, or PDDL) is debated.

Existing research often treats LLMs as "translators" (converting text to PDDL for a classical solver) or uses "direct generation" (prompting the LLM to output a full plan in one shot). A critical gap exists in agentic planning: the ability for an LLM to act as an interactive search policy that executes actions one by one, observes the resulting state transitions, and iteratively corrects its path. The authors investigate whether this step-wise, closed-loop interaction with a formal environment improves planning success rates compared to direct generation, and under what conditions.

2. Methodology

A. Tool: PyPDDLEngine

To enable step-wise interaction, the authors developed PyPDDLEngine, an open-source Python library that wraps a PDDL simulation engine.

• Functionality: It exposes the PDDL transition system as an interactive environment via the Model Context Protocol (MCP).
• Capabilities: The LLM can invoke seven specific tools:
  1. Initialize session (load domain/problem).
  2. Query current state (true predicates).
  3. Query applicable actions (valid ground actions).
  4. Execute single action (advance state).
  5. Reset to initial state (restart if trajectory is unproductive).
  6. Query action history.
  7. Validate complete plan.
• Role: Unlike classical planners that commit to a full sequence internally, PyPDDLEngine allows the LLM to navigate the state space step-by-step, acting as the search policy itself.

B. Experimental Setup

• Dataset: 102 instances from the International Planning Competition (IPC) Blocksworld benchmark, chosen for their well-defined difficulty gradient.
• Time Budget: A uniform 180-second wall-clock limit for all approaches.
• Models & Approaches Compared:
  1. Fast Downward (lama-first): Classical satisficing planner (greedy, single-shot). Baseline for speed.
  2. Fast Downward (seq-sat-lama-2011): Classical satisficing planner (iterative improvement). Baseline for plan quality.
  3. Direct LLM: Claude Haiku 4.5 prompted with the full PDDL description to generate a complete plan in one pass. Retries on failure without feedback.
  4. Agentic LLM: Claude Haiku 4.5 interacting with PyPDDLEngine via MCP. It executes actions sequentially, observes state changes, and can reset if it judges a trajectory unproductive.

C. Metrics

• Success Rate: Fraction of instances solved within the time budget.
• Plan Length: Number of actions (measured on the "co-solved" set to avoid survivorship bias).
• Failure Modes: Timeout, Early Exit (agent self-judged unsolvable), or Invalid Plan.
• Token Cost: Computational cost per solution.

3. Key Contributions

1. PyPDDLEngine: A novel open-source infrastructure that bridges LLMs and PDDL simulators via MCP, enabling genuine step-wise agent interaction rather than static plan generation.
2. Empirical Characterization: A rigorous four-way comparison showing that agentic interaction yields only a modest 3% improvement in success rate over direct generation (66.7% vs. 63.7%) but at a 5.7× higher token cost.
3. Insight on Feedback Mechanisms: The paper identifies that the lack of externally grounded feedback in PDDL step-wise simulation limits agentic gains. Unlike coding agents that receive objective compiler errors, PDDL agents rely on self-assessed state transitions, leading to unreliable progress tracking.
4. Memorization vs. Reasoning: Evidence suggests LLMs on Blocksworld rely on training-data recall rather than generalizable planning, as they produce shorter plans than iterative classical solvers but fail to improve with interaction on hard instances.

4. Key Results

• Success Rates:

  • Fast Downward: 85.3% (solved 87/102 instances).
  • Agentic LLM: 66.7% (68/102).
  • Direct LLM: 63.7% (65/102).
  • Observation: The agentic approach solved 3 more instances than the direct approach but failed on 6 instances where the direct approach succeeded due to "Early Exit" (the agent incorrectly judged the problem unsolvable).

• Plan Quality:

  • On the 49 instances solved by all four approaches, both LLM approaches produced shorter plans than the classical seq-sat-lama-2011 (which iteratively optimizes plans).
  • Implication: This suggests LLMs are recalling near-optimal patterns from training data rather than deriving solutions through search. The agentic loop provided no additional plan-quality benefit over the direct single-shot generation.

• Cost Efficiency:

  • The agentic approach consumed 5.7× more tokens per solution than the direct approach.
  • The marginal gain of 3 additional solved instances required approximately 14.4 million extra tokens.

• Hard Cases:

  • On the 15 hardest instances (where Fast Downward timed out), the LLMs solved very few. The agentic approach solved one unique instance (Instance 101) with a 186-action plan, but this is noted as a potential outlier rather than a consistent capability.

5. Significance and Discussion

• The "Grounding" Gap: The paper argues that the success of agentic coding agents (which fix code based on compiler errors) does not translate directly to planning. In coding, feedback is externally grounded (objective error messages). In PDDL simulation, feedback is self-generated (the agent sees a state change but must infer if it is "closer" to the goal). Without external signals indicating progress, the LLM cannot reliably correct its trajectory.
• Implications for Robotics: For LLMs to function as deliberative layers in robotics, the perception layer must provide externally grounded progress signals (e.g., "you are 2 meters closer to the goal") rather than just raw state observations. Broad world knowledge alone is insufficient without objective feedback loops.
• Memorization Hypothesis: The fact that LLMs produce shorter plans than iterative classical solvers, yet fail to improve with step-wise interaction, strongly suggests they are recalling solutions from their training data (Blocksworld is a common benchmark in training sets) rather than performing generalizable reasoning.
• Future Directions: The authors suggest that augmenting step-wise feedback with goal-distance heuristics (external signals) could test if agentic performance improves, and that evaluating LLMs on domains with low training-data coverage is necessary to distinguish reasoning from memorization.

In conclusion, while agentic LLM planning offers a slight success rate boost, it is currently inefficient and limited by the lack of objective feedback. The results challenge the assumption that step-wise interaction alone unlocks superior planning capabilities, highlighting the need for better environmental grounding in robotic systems.