💻 computer science

PREPING: Building Agent Memory without Tasks

This paper introduces Preping, a framework that enables agents to construct procedural memory before encountering target tasks by using a proposer-guided loop to generate and selectively store high-quality synthetic practice trajectories, thereby overcoming the cold-start gap with significantly lower deployment costs than online memory construction.

Original authors: Yumin Choi, Sangwoo Park, Minki Kang, Jinheon Baek, Sung Ju Hwang

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yumin Choi, Sangwoo Park, Minki Kang, Jinheon Baek, Sung Ju Hwang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a new employee to manage your digital life. They are incredibly smart (a powerful AI), but they have never seen your specific apps, tools, or rules before.

The Problem: The "Cold Start" Gap
Usually, when you hire someone, you either:

Train them beforehand (Offline): You give them a thick manual of past successful tasks. But if you're launching a brand new app, that manual doesn't exist yet.
Train them on the job (Online): You let them start working immediately. But at first, they will make mistakes, fail, and annoy you while they figure things out. This is the "cold start" problem.

The paper asks: Can we train this employee before they ever see a real customer, using only practice sessions we create ourselves?

The Solution: PREPING (Pre-Task Reusable Playbook Making)
The authors created a system called PREPING. Think of it as a simulated training camp where the AI practices on tasks it invents for itself, but with a very strict coach to make sure the practice is actually useful.

Here is how the "training camp" works, using three characters:

1. The Proposer (The Creative Coach)

This is the part of the AI that invents practice tasks.

Without a coach: If you just tell an AI to "practice," it might do the same thing 100 times (like opening a door) or try to do impossible things (like flying). This is useless.
With PREPING: The Proposer has a memory notebook (Proposer Memory). It tracks what it has already practiced. If it just practiced "opening doors," the notebook tells it, "Okay, now try locking them" or "Try using the elevator." It also checks a map of the environment to make sure the task is actually possible (e.g., "Don't ask the AI to fly if the building has no roof").
The Goal: To create a diverse set of practice drills that cover all the tools and rules of the new environment without repeating itself.

2. The Solver (The Trainee)

This is the AI actually doing the work. It tries to complete the tasks invented by the Proposer. It writes code, clicks buttons, and calls APIs just like it would for a real user.

3. The Validator (The Strict Inspector)

This is the most important part. Not every practice session is good.

Sometimes the trainee tries to do something impossible (like withdrawing money from a card that doesn't exist).
The Validator looks at the result and says: "Stop. This was a fake task. Don't write this down."
If the task was real and the trainee succeeded, the Validator says: "Great! Here is the lesson learned. Write this down in the playbook."

The Result: A "Playbook" Ready on Day One
By the time the AI is deployed to real users, it has a Playbook (a memory of procedures) filled with:

How to combine different tools correctly.
What to do when things go wrong.
Which tools are actually available.

Why is this better than the old ways?

Vs. No Training: The AI doesn't start from zero. It doesn't make embarrassing mistakes on your first real task.
Vs. Learning on the Job: The AI doesn't need to learn from your failures. It already knows the ropes.
Cost: Learning on the job is expensive because the AI has to keep "thinking" and "updating its memory" while you wait for it to work. PREPING does all that heavy lifting before you hire it, making the actual work much cheaper and faster.

The Analogy in a Nutshell
Imagine a pilot.

Old Way: You let them fly a real plane with passengers for the first time to learn the controls. (Dangerous and slow).
PREPING Way: You put them in a flight simulator. A smart instructor (Proposer) gives them specific scenarios (cloudy day, engine trouble, new airport). A safety officer (Validator) stops them if they try to fly into a mountain. By the time they get in the real cockpit, they have a mental checklist (memory) of how to handle everything, and they are ready to fly immediately.

The Paper's Findings
The authors tested this on three different "digital worlds" (managing apps, calling functions, and using server tools). They found that:

PREPING created a much smarter AI than just reading a manual or guessing.
It performed almost as well as an AI that had been trained on thousands of real human tasks, even though it had seen zero real human tasks.
It saved money and time because the AI didn't need to "learn" while working for you.

In short, PREPING teaches an AI to be a pro before it ever meets a customer, by having it practice smartly and filtering out the bad practice.

Technical Summary: PREPING – Building Agent Memory without Tasks

1. Problem Statement

Large Language Model (LLM) agents deployed in executable environments (e.g., tool APIs, Model Context Protocol servers) require procedural memory to succeed. This memory captures environment-specific workflows, tool composition rules, and failure recovery strategies. However, existing memory construction methods face a cold-start gap:

Offline methods rely on curated human demonstrations or solved trajectories, which are costly to generate and often unavailable for new environments before deployment.
Online methods accumulate memory from user interactions after deployment. This forces the agent to start with empty memory, exposing users to early failures, latency in memory updates, and additional deployment-time costs.

The paper addresses the problem of pre-task memory construction: Can an agent build reusable procedural memory before observing any target-environment tasks, using only self-generated synthetic practice? The challenge lies in the fact that without task instructions, synthetic practice can easily become redundant, infeasible, or ungrounded, leading to memory contamination rather than improvement.

2. Methodology: PREPING

The authors propose PREPING (Pre-Task REusable Playbook MakING), a framework that treats memory construction as a control problem over what to practice and what to store. PREPING operates through a synthetic-practice loop involving three LLM-powered modules and two distinct memory states:

2.1 Dual Memory States

Proposer Memory ( $M_{prop}$ ): A construction-time control state. It tracks the history of synthetic tasks, tool usage statistics, validation outcomes, and grounded environment facts (entities, states, constraints). It guides the generation of future tasks to ensure coverage and feasibility.
Solver Memory ( $M_{sol}$ ): The deployment-facing procedural memory. It contains only validated, reusable insights distilled from successful trajectories.

2.2 The Construction Loop

At each iteration $t$ :

Proposer ( $A_{prop}$ ): Generates a synthetic task $x_t$ conditioned on environment documentation ( $D$ ) and the current Proposer Memory ( $M_{prop}$ ). The Proposer is designed to avoid redundancy and target under-explored tools while grounding tasks in observed environment facts.
Solver ( $A_{sol}$ ): Executes the task $x_t$ in the target environment ( $E$ ) to produce a trajectory $\tau_t$ .
Validator ( $A_{val}$ ): Evaluates the task-trajectory pair $(x_t, \tau_t)$ for feasibility (is the task grounded and executable?) and completion (was the objective achieved?).
Asymmetric Updates:
- Proposer Update: All experience (including failed or infeasible tasks) updates $M_{prop}$ . This allows the system to learn what not to practice and to adjust coverage.
- Solver Update: Only trajectories deemed feasible and successfully completed are admitted to $M_{sol}$ . These are distilled into compact procedural bullets (playbook style) rather than raw logs.

3. Key Contributions

Formalization of Pre-Task Memory: The paper defines a setting where agents must construct memory without access to target-task distributions, distinguishing it from standard online learning or offline fine-tuning.
Proposer-Guided Control: Unlike naive self-play or random exploration, PREPING introduces a structured control state ( $M_{prop}$ ) that actively shapes the synthetic practice distribution to maximize tool coverage and feasibility while minimizing redundancy.
Validation-Gated Admission: The framework explicitly filters out infeasible synthetic trajectories before they contaminate the solver's memory, preventing the distillation of misleading procedural guidance.
Cost-Efficient Initialization: PREPING shifts memory construction from a recurring deployment-time expense to a one-time pre-deployment investment, significantly reducing the cost per task during the cold-start phase.

4. Experimental Results

PREPING was evaluated on AppWorld (stateful app workflows), BFCL v3 (structured function calling), and MCP-Universe (realistic MCP-server tool use).

Performance Gains: PREPING substantially outperforms baselines that rely on documentation alone or free-form exploration.
- AppWorld: +17.1 points over the no-memory baseline (Base).
- BFCL v3: +19.3 points over Base.
- MCP-Universe: +5.4 points over Base.
- It achieves performance competitive with strong Task-Informed methods (e.g., ACE-Offline and ACE-Online) that utilize human-defined tasks or deployment-time user data, despite using no such data for construction.
Ablation Studies: The gains are driven by the combination of validation-gated admission and proposer-side control. Removing the validator leads to memory contamination; removing proposer memory leads to redundant or infeasible practice.
Cold-Start Mitigation: When used to initialize online learning (PREPING+ACE), the framework significantly reduces early failure rates. On AppWorld, it matches the tool coverage of ACE-Online after 58 tasks immediately upon deployment, whereas ACE-Online requires 58 tasks to reach the same coverage.
Cost Reduction: PREPING reduces deployment-time costs by 2.99× on AppWorld and 2.23× on BFCL v3 compared to online memory construction (ACE-Online), as it avoids recurring memory-update calls during user interactions.
Generalization: The method generalizes across different backbone models (GPT-5.1, GPT-OSS-120B, Qwen3-235B), indicating the benefit stems from the construction procedure rather than a specific model architecture.

5. Significance and Claims

The paper claims that agent memory need not be accumulated passively from deployment-time interactions. Instead, it can be actively prepared through controlled, environment-grounded synthetic practice before deployment.

Practical Impact: PREPING offers a viable solution for the "cold-start" problem in new environments where no historical task data exists. It allows agents to deploy with a "playbook" of reusable procedures, improving immediate performance and reducing the cost of online adaptation.
Modest Claims: The authors note that the method assumes access to sufficiently detailed API or tool documentation to ground synthetic tasks. They acknowledge that in environments with poor documentation, the feasibility of generating grounded synthetic tasks may be limited. The paper does not claim to solve memory construction for entirely undocumented environments but rather for those with structured interfaces (like MCP servers) where documentation is available but task distributions are unknown.

In summary, PREPING demonstrates that by jointly controlling the generation of synthetic tasks and the selection of trajectories for storage, agents can build effective procedural memory without prior human task data, bridging the gap between offline preparation and online adaptation.