Technical Summary: PIVOT – Bridging Planning and Execution in LLM Agents via Trajectory Refinement

1. Problem Statement

Large Language Model (LLM)-based agents frequently generate plans that appear coherent but fail during execution. This plan–execution misalignment arises from infeasible actions, incorrect state assumptions, or constraint violations that only become apparent upon interaction with the environment. Crucially, these errors compound over long horizons, triggering cascading failures that render the remainder of a plan unrecoverable.

Existing improvement methods fail to address this holistically because they lack a trajectory-level optimization view. Diagnostic methods identify failures post-hoc without optimizing the underlying plan; iterative refinement methods often revise local reasoning steps rather than the full trajectory; and evolutionary methods search candidate trajectories without directly optimizing the loss induced by execution outcomes. There is a need for a mechanism that treats the full trajectory as an optimizable object and converts execution feedback into an update signal.

2. Methodology: PIVOT Framework

The authors propose PIVOT (Plan–Inspect–eVOlve Trajectories), a self-supervised framework that formulates trajectory refinement as iterative optimization in a discrete, language-defined space. The framework operates as a closed loop comprising four stages:

A. Problem Formulation

PIVOT defines a trajectory-level objective function $L(\tau)$ combining three error sources:

1. Goal-Achievement Loss ($\ell_G$): Measures failure to achieve the desired terminal outcome.
2. Plan–Execution Divergence ($D$): Quantifies the mismatch between the planned trajectory and the actual execution trace.
3. Execution Cost ($C$): Penalizes inefficient execution (e.g., excessive tool calls).

Since exact gradients are unavailable for discrete language trajectories, PIVOT approximates the gradient $\partial L / \partial \tau$ using structured natural-language feedback (textual gradients), inspired by TextGrad.

B. The Four Stages

1. PLAN (Trajectory Generation):
   The agent generates an explicit, structured candidate trajectory $\tau$ before interacting with the environment. This stage forces the agent to articulate goals, tool selection, dependencies, and failure contingencies, serving as a "blueprint" for subsequent stages.

2. INSPECT (Backward Discrepancy Analysis):
   The agent executes the plan to produce a trace $\hat{\tau}$. The INSPECT module performs a backward analysis to identify the earliest causally responsible break ($i^\star$) where the execution diverges from the plan.

   • It computes a textual gradient $g(\tau)$ that attributes errors to specific trajectory steps.
   • In a Human-in-the-Loop (HITL) setting, this uses human feedback.
   • In a self-supervised setting, an LLM-as-a-judge provides structured feedback on outcomes and intermediate behavior.
   • The output is a repair signal identifying the failure point and actionable instructions for the next stage.

3. EVOLVE (Local Trajectory Repair):
   Guided by the textual gradient, the EVOLVE module updates the trajectory. Crucially, it operates locally: it preserves the validated prefix ($\tau_{\le i^\star}$) and rewrites only the unsupported suffix. This avoids discarding correct work.

   • The update is subject to a monotonic acceptance rule: the new trajectory $\tau'$ is accepted only if it improves the estimated objective ($\hat{L}' \prec \hat{L}$). This prevents degradation from noisy feedback.

4. VERIFY (Global Constraint Validation):
   Before finalizing, the VERIFY module performs a global check against the original task specification and user constraints. This ensures that intermediate refinements do not inadvertently violate the original task requirements.

3. Key Contributions

• Trajectory-Level Optimization: PIVOT treats the full agent trajectory as an optimizable object, converting execution feedback into a structured update signal rather than relying on local step-wise corrections.
• Textual Gradients for Discrete Spaces: The framework adapts the concept of gradients to non-differentiable language trajectories by using structured natural language feedback to guide local updates.
• Monotonic Acceptance: A preference-based rule ensures that trajectory quality is non-decreasing across iterations, providing stability without requiring model fine-tuning.
• Efficiency: The framework is designed to be computationally efficient, using in-line prompts to steer a single trajectory rather than regenerating full trajectories from scratch.

4. Experimental Results

PIVOT was evaluated on DeepPlanning (long-horizon Travel and Shopping Planning) and GAIA (open-domain tool use).

• Performance:

  • HITL Regime: PIVOT with human feedback establishes a strong upper bound, achieving up to ~94% relative improvement in constraint satisfaction on DeepPlanning compared to baselines.
  • Autonomous Regime: The fully autonomous variant (w/o HITL) significantly outperforms the strongest non-PIVOT baselines (e.g., Self-Critique, AgentDebug) in 12 of 20 model-benchmark cells. It improves over ReACT in all 20 cells.
  • Model Agnosticism: Gains are observed across various backbone models (Qwen3, Claude 4.5/4.6), indicating PIVOT complements backbone scaling rather than just compensating for weaker models.

• Efficiency:

  • PIVOT is 3–5× more token-efficient than competing refinement methods (Self-Critique, SE-Agent, AgentDebug).
  • Unlike baselines that require regenerating full trajectories based on long failure transcripts, PIVOT steers a single trajectory with bounded overhead, redirecting the agent within the current context when errors occur.

• Ablation Studies:

  • Removing the VERIFY stage caused the largest performance drop (avg. -13.3%), highlighting the critical role of final constraint validation.
  • Removing PLAN or EVOLVE also caused significant degradation, confirming the necessity of both strong initialization and discrepancy-guided repair.
  • Simply increasing the "thinking budget" (extended thinking) without structured refinement did not improve performance, suggesting the issue is the allocation of reasoning, not the quantity.

5. Significance and Claims

The paper claims that PIVOT represents a principled methodology for mitigating plan–execution gaps in autonomous agent systems. Its significance lies in:

1. Closing the Gap: It directly addresses the systematic divergence between intended actions and realized behavior through iterative, feedback-driven refinement.
2. Reliability: By enforcing monotonic improvement and global constraint validation, it produces safer and more reliable autonomous systems.
3. Efficiency: It demonstrates that high-quality trajectory optimization can be achieved with significantly lower computational costs than existing methods.
4. Interpretability: The framework provides interpretable failure attribution by localizing the earliest causal break in the trajectory.

The authors conclude that while PIVOT significantly improves reliability in constraint-driven, long-horizon settings, it remains subject to limitations such as context degradation, retrieval bottlenecks (in open-domain tasks), and the inherent reasoning capacity of the underlying model. They emphasize the need for deployment guardrails and ongoing evaluation in real-world settings.