Plans Don't Persist: Why Context Management Is Load Bearing for LLM Agents

The Big Idea: The "Fragile Note" Problem

Imagine you are a robot tasked with cleaning a messy house. Before you start, you write a to-do list on a sticky note: "1. Go to the kitchen. 2. Pick up the cup. 3. Put it in the dishwasher."

In the world of AI agents (robots powered by Large Language Models), this "sticky note" is the Plan. The paper investigates a critical question: Does the robot actually memorize this list, or does it just keep reading the note every time it takes a step?

The authors found that for standard AI models, the answer is: It just keeps reading the note.

If you take that sticky note away (or if the robot's memory window gets full and the note gets "evicted"), the robot immediately forgets what it was supposed to do. It doesn't have the plan stored in its "brain"; it only has the plan stored in the "paper" it is currently looking at.

The Experiment: The "Ghost Replay"

To prove this, the researchers used a clever trick called Replay Pairing. Imagine running a movie of the robot cleaning the house twice:

1. Movie A (The Real Deal): The robot sees the sticky note with the plan, then starts cleaning.
2. Movie B (The Ghost): The robot does the exact same actions and sees the exact same results, but the sticky note with the plan is magically removed from its view.

The researchers then compared the robot's internal "brain state" (its hidden thoughts) between Movie A and Movie B.

• If the robot memorized the plan: Its brain state should be almost identical in both movies, because it knows the plan by heart.
• If the robot just reads the note: Its brain state should change drastically the moment the note disappears.

The Result: The moment the note was removed, the robot's brain state changed immediately. The "signal" of the plan dropped by 4 to 12 times in a single step. It's as if the robot looked at the note, took a step, and then the note vanished from its mind entirely.

The "Thinking" Trap (Reasoning Models)

The researchers also tested newer, smarter AI models (called "Reasoning Models") that talk to themselves before acting. These models have a special section in their output called <thinking>, where they write out their thoughts.

When they tried the "Ghost Replay" on these smart models, the results looked weird. The robot seemed to remember the plan even without the note!

The Twist: The researchers realized this was a trick. The smart models were re-writing the plan inside their own <thinking> blocks as they went along. So, even when they removed the original note, the robot was still "reading" the plan from its own recent thoughts.

They fixed this with "Strict Stripping," which removes not just the original note, but also the robot's recent thoughts that mentioned the plan. Once they did this, the smart models also showed that they rely on having the plan visible right in front of them. They don't actually "hold" the plan in their long-term memory.

The Stress Test: What Happens When the Note is Gone?

To see how bad this is in real life, they ran a "Compression Stress Test." They forced the AI to clean a house but limited its memory window so it had to throw away old notes to make room for new ones.

• The "Naive" Approach: They let the AI throw away the plan note to save space.
  • Result: The AI's success rate crashed by 35%. It couldn't finish the job because it forgot the plan.
• The "Smart" Approach: They tried to use a detector to see when the AI was forgetting and re-insert the plan note.
  • Result: This didn't help much. The AI was already confused because it had lost the context of what it was doing, not just the plan.

The Takeaway

The paper concludes that for current AI agents, context management is the most important part of the system.

Think of the AI's memory window like a whiteboard. The plan isn't a permanent tattoo on the robot's arm; it's a marker drawing on the whiteboard. As long as the marker is on the board, the robot works. If you erase the board (even if you erase just the plan), the robot stops working.

Key Claims from the Paper:

• Plans are not persistent: They are not stored in the AI's internal "brain" state; they must be re-read from the text window every step.
• Decay is fast: The "memory" of the plan disappears almost instantly (within one step) once the text is gone.
• Reasoning models are tricky: They hide the plan in their own thinking traces, which can fool tests unless you strip those traces away too.
• Protection isn't enough: Simply protecting the plan text isn't a magic fix for memory limits; if the AI loses the context of its actions, it fails regardless of whether the plan is there.

The paper provides a new way to measure this "forgetting" and proves that for now, AI agents are fragile: they need the plan to be visible right now to keep working.