💬 NLP

From Plan to Action: How Well Do Agents Follow the Plan?

This paper presents a systematic analysis of 16,991 agent trajectories revealing that while explicit plans generally improve task success, agents often fail to follow them due to reliance on overfitted internal workflows, and that poorly designed or misaligned plans can actually degrade performance, highlighting the need for training paradigms that teach adaptive reasoning over plan memorization.

Original authors: Shuyang Liu, Saman Dehghan, Jatin Ganhotra, Martin Hirzel, Reyhaneh Jabbarvand

Published 2026-04-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shuyang Liu, Saman Dehghan, Jatin Ganhotra, Martin Hirzel, Reyhaneh Jabbarvand

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you hire a very smart, but slightly scattered, apprentice to fix a broken machine in your factory. You give them a recipe (a plan) to follow:

Find the broken part.
Test it to see how it breaks.
Fix it.
Check that the fix actually works.

This paper asks a simple but profound question: Does the apprentice actually follow the recipe, or do they just wing it?

The researchers studied thousands of "trials" where AI agents tried to fix real software bugs. They wanted to see if giving the AI a strict step-by-step plan actually helped, or if the AI just ignored the plan and relied on its own (sometimes messy) instincts.

Here is the breakdown of their findings, using some everyday analogies:

1. The "Ghost in the Machine" (Internal Habits)

Even when the researchers removed the recipe entirely, the AI didn't just panic. It fell back on habits it learned during its training.

The Analogy: It's like asking a chef to cook a meal without a recipe. Even without the paper, they might still chop onions first, then fry the meat, because that's how they learned to cook.
The Result: The AI followed the "standard" steps (Find, Test, Fix, Check) about 60–80% of the time, even without being told to. However, without the written plan, the success rate dropped. The plan acts like a safety rail; without it, the AI wanders off the path more often.

2. The "Bad Recipe" is Worse Than No Recipe

The researchers tried giving the AI a flawed recipe. For example, they told the AI: "Find the bug, then immediately fix it, skipping the testing step."

The Analogy: Imagine a doctor telling you to skip the diagnosis and go straight to surgery.
The Result: This was disastrous. The AI performed worse with a bad plan than with no plan at all. When the AI was forced to follow a broken workflow, it got confused and failed more often. It seems the AI prefers to ignore a bad instruction and rely on its own (better) instincts, but the bad instruction still messed up its focus.

3. The "Over-Engineered" Recipe

They also tried adding extra steps to the recipe, like "Write a summary of your changes" or "Run a whole bunch of old tests first."

The Analogy: You tell the apprentice, "Before you fix the engine, you must also wash the car, check the oil, and write a poem about the tire pressure."
The Result: This slowed the AI down and made it less successful. The AI got distracted by the extra, unnecessary steps. It turns out that less is more. If the plan doesn't match the AI's natural way of thinking, adding more steps just creates noise.

4. The "Nagging Reminder" Works

One of the most interesting findings was about reminders. As the AI worked longer, it started to forget the original plan (a phenomenon known as "context window pressure").

The Analogy: Imagine a student taking a long exam. If the teacher whispers, "Remember, step 2 is testing!" every few minutes, the student stays on track.
The Result: When the researchers periodically reminded the AI of the plan, the AI followed the instructions much better and fixed more bugs. The plan needs to be "top of mind" constantly.

5. The "Hard Mode" Problem

Finally, they tested the AI on harder, cleaner problems (SWE-bench Pro) that the AI hadn't seen before.

The Analogy: The apprentice was great at fixing the specific broken toaster they practiced on (because they memorized the answer), but when given a brand new, complex microwave, they forgot the recipe entirely.
The Result: On these harder, "clean" problems, the AI's compliance with the plan dropped significantly. It suggests that for the easy problems, the AI might have just been memorizing the solution rather than truly reasoning through the plan.

The Big Takeaway

The paper concludes that we are teaching AI the wrong way.

Currently, we are trying to "hard-code" specific recipes into the AI's brain. But the AI is like a human who learns best by understanding the logic of a task, not by memorizing a rigid list of steps.

The Problem: If the recipe doesn't match the AI's internal logic, it fails.
The Solution: Instead of giving the AI a rigid script, we should train it to be adaptable. We need to teach it how to think and reason so it can create its own plan on the fly, rather than forcing it to follow a script that might be wrong for the specific situation.

In short: AI agents are smart, but they are easily distracted. A good plan helps, but a bad plan hurts. The future isn't about giving them better scripts; it's about teaching them how to write their own.

1. Problem Statement

Large Language Model (LLM)-based agents are increasingly used for software engineering tasks, such as fixing GitHub issues. These agents are typically instructed to follow a structured plan (e.g., Navigation $\to$ Reproduction $\to$ Patch $\to$ Validation) to decompose complex tasks. However, it remains unclear to what extent agents actually adhere to these instructed plans during execution.

The core problem is the lack of systematic analysis regarding plan compliance. Without understanding if an agent follows a plan, it is impossible to determine if a successful task resolution stems from correct strategic reasoning or from other factors like data contamination, overfitting to benchmark trajectories, or memorized workflows. Furthermore, the impact of plan variations (removing steps, reordering, or adding steps) on agent performance is not well understood.

2. Methodology

The authors conducted a large-scale, systematic, and fully automated evaluation of plan compliance using the SWE-agent scaffold.

Dataset & Models:
- Trajectories: 16,991 trajectories generated to resolve instances from SWE-bench Verified (500 instances) and SWE-bench Pro (266 Python instances).
- LLMs: Four diverse models were tested: GPT-5 mini, DeepSeek-V3, DeepSeek-R1, and Devstral-small.
Experimental Design:
The study evaluated agents under eight plan settings:
1. Standard Plan: Navigation (N) $\to$ Reproduction (R) $\to$ Patch (P) $\to$ Validation (V).
2. No Plan: The plan is removed entirely from the system prompt.
3. Reduced Plans: Removing specific phases (No Reproduction, No Validation).
4. Augmented Plans: Adding relevant phases (Regression Test Execution, Change Summary).
5. Reordered Plan: Patching before Reproduction (N $\to$ P $\to$ R $\to$ V).
6. Reminded Plan: The standard plan is re-injected into the context every five steps to counteract context window pressure.
Metrics (The Novel Contribution):
The authors introduced a Process-Centric Analysis using Graphectory (graph representation of trajectories) and Langutory (abstracting actions into a sequence of plan phases). They defined a Plan Compliance (PC) metric composed of three dimensions:
1. Plan Phase Compliance ($PPC$): Measures if all required phases appear in the trajectory.
2. Plan Order Compliance ($POC$): Measures if phases appear in the correct relative order (using Longest Increasing Subsequence logic).
3. Plan Phase Fidelity ($PPF$): Penalizes the inclusion of phases outside the instructed plan alphabet.
- Overall Score: $PC = (PPC \times POC \times PPF)^{1/3}$ .

3. Key Findings & Results

RQ1: Standard Plan Compliance

Variable Adherence: Agents do not strictly follow the standard plan. Compliance varies significantly by model.
- Devstral-small follows the order well but frequently adds out-of-plan phases.
- GPT-5 mini adapts based on difficulty, often skipping Reproduction for easier tasks.
- DeepSeek models often omit phases or execute them out of order.
Correlation with Success: Generally, higher plan compliance correlates with task success, though GPT-5 mini showed a negative correlation (adapting strategies led to success even when deviating from the plan).

RQ2: Behavior Without a Plan

Internalized Strategies: Even without explicit instructions, agents follow their internalized problem-solving strategies, which often overlap partially with the standard plan (e.g., Devstral-small still follows N $\to$ R $\to$ P $\to$ V).
Performance Drop: Removing the plan consistently reduces the success rate across all models. The standard plan acts as a guide that improves local reasoning, even if the agent doesn't follow it perfectly.

RQ3–RQ5: Impact of Plan Variations

Bad Plans vs. No Plan: A subpar or incomplete plan (e.g., removing a critical phase like Reproduction) often hurts performance more than having no plan at all. This suggests that forcing an agent into a flawed workflow is worse than letting it rely on its internal strategy.
Augmentation Risks: Adding "best practice" phases (like Regression Testing) that do not align with the model's internal strategy can degrade performance. Models may get distracted by these extra steps.
Reordering: Agents prioritize effective workflows over prescribed order. If a model naturally skips Reproduction, forcing it to do so later (reordered plan) does not consistently improve performance.
Periodic Reminders: Re-injecting the plan every few steps significantly improves compliance and success rates by mitigating "context window pressure" and preventing the agent from drifting off-task.

RQ6: Generalization (SWE-bench Pro)

On the more challenging and less contaminated SWE-bench Pro, plan compliance dropped by 13% on average.
Agents exhibited different behaviors (e.g., giving up on generating reproduction tests early), suggesting that the standard high-level plan is less effective for harder, less contaminated tasks.

RQ7: Nondeterminism

While LLMs are nondeterministic, repeating experiments confirmed that the observed trends in plan compliance and performance are consistent and not artifacts of randomness.

4. Key Contributions

First Large-Scale Analysis: The first systematic study quantifying how well programming agents follow instructed plans across multiple models and benchmarks.
Novel Metrics: Introduction of Plan Compliance (PC) and its sub-metrics ($PPC, POC, PPF$) to mathematically measure adherence to plans, distinct from general trajectory complexity.
Root Cause Identification: Identification of factors driving violations, including fine-tuning paradigms, context window pressure, data contamination, and overfitting.
Strategic Insight: Demonstration that plan alignment with a model's internalized strategy is crucial. A plan that conflicts with the model's learned workflow can be detrimental.

5. Significance & Implications

The paper challenges the assumption that simply providing a plan guarantees adherence or better performance.

Research Gap: Current fine-tuning paradigms often encode task-specific plans directly into models, leading to overfitting. The authors argue for a shift toward teaching models to follow instructed plans adaptively rather than memorizing specific workflows.
Practical Advice:
- Reminders Work: Periodic plan reminders are an effective, low-cost intervention to improve long-horizon task success.
- Caution with Augmentation: Adding phases to a plan should be done carefully; if the model isn't "trained" to use them, performance may drop.
- Process-Centric Evaluation: Evaluating agents solely on success rates is insufficient; analyzing how they solve problems (plan compliance) is vital for diagnosing failures and preventing data contamination artifacts.

In conclusion, while plans are beneficial, their effectiveness is tightly coupled with the model's internal reasoning strategies. Future agent design should focus on improving the model's ability to reason and act adaptively to follow dynamic instructions rather than relying on static, pre-encoded workflows.