Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills

This survey proposes a unified four-paradigm framework to categorize and analyze the fragmented landscape of agentic AI adaptation, distinguishing between agent-side improvements (A1/A2) and tool-side enhancements (T1/T2) to systematically review post-training methods, memory architectures, and skill systems while evaluating their trade-offs and outlining future challenges.

The Gist

Imagine you have a brilliant, world-class General Manager (the AI Agent). This manager is incredibly smart, knows a lot of facts, and can write great emails. But if you ask them to build a house, they might struggle because they don't know how to swing a hammer, and they can't remember the blueprints from three days ago.

To make this manager truly useful, you don't just want them to "think harder." You want to adapt them. You need to teach them new skills, give them better tools, and help them remember things.

This paper is a massive map of how to upgrade this AI Manager. The authors realized that researchers were all doing different things in different ways, so they organized everything into a simple 4-Quadrant Grid based on two questions:

1. What are we changing? Are we changing the Manager's brain (the Agent), or are we changing their tools and helpers (the Tools)?
2. How do we know it's working? Are we judging them by the final result (Did the house get built?), or by the specific steps they took (Did they swing the hammer correctly?).

Here are the four ways to upgrade your AI, explained with everyday analogies:

1. The "Drill Sergeant" Approach (A1: Agent Adaptation via Tool Signals)

• What changes: You retrain the Manager's brain directly.
• How it works: You give the Manager a task. They try to use a tool (like a calculator or a search engine). If the tool gives a clear "Pass" or "Fail" (e.g., the code compiles, or the math is right), you immediately correct the Manager.
• The Analogy: Imagine a drill sergeant training a soldier. The soldier tries to load a rifle. Click. The gun jams. The sergeant immediately yells, "No! Do it this way!" The soldier learns the mechanics of the tool instantly because the feedback is immediate and factual.
• Best for: Tasks where you know the answer is right or wrong instantly, like coding or math.

2. The "Performance Review" Approach (A2: Agent Adaptation via Output Signals)

• What changes: You retrain the Manager's brain directly.
• How it works: You let the Manager do the whole job. They might use tools, maybe they make mistakes, maybe they get lucky. At the very end, you look at the final report. If the report is good, you say "Good job!" If it's bad, you say "Try again." You don't tell them which step was wrong, just that the result was wrong.
• The Analogy: Imagine a chef cooking a complex meal. They chop, sauté, and season. You don't taste every ingredient as they go. You only taste the final dish. If it's delicious, the chef learns, "Okay, that whole process worked." If it's salty, they have to guess which step went wrong.
• Best for: Complex tasks like writing a novel or doing deep research where the "steps" are hard to grade, but the final story matters.

3. The "Hire a Specialist" Approach (T1: Tool Adaptation, Agent-Agnostic)

• What changes: You don't touch the Manager at all. You just buy or build better tools.
• How it works: The Manager stays exactly the same (maybe they are a closed-source AI you can't change). Instead, you train a specialized "Search Engine" or a "Code Checker" to be perfect. The Manager just learns to ask the right questions to this new, super-smart tool.
• The Analogy: Imagine a CEO who is great at strategy but bad at spreadsheets. Instead of forcing the CEO to learn Excel (which is hard and expensive), you hire a brilliant Data Analyst. The CEO doesn't change; the tool they use becomes perfect. The Analyst works for any CEO, not just this one.
• Best for: When you can't change the main AI (like using ChatGPT) but want to make it smarter at specific tasks.

4. The "Personal Assistant" Approach (T2: Tool Adaptation, Agent-Supervised)

• What changes: You keep the Manager frozen, but you train a Personal Assistant specifically for that Manager.
• How it works: You take the Manager's past work. If the Manager wrote a great report, you look at what the Assistant did to help them. You train the Assistant to do exactly what that specific Manager likes. The Assistant learns to speak the Manager's language.
• The Analogy: Imagine a Famous Director (the frozen Agent) who is very picky. You hire a Cinematographer (the Tool). You don't change the Director's vision. Instead, you train the Cinematographer to know exactly how the Director likes the lighting, the angles, and the mood. The Cinematographer becomes a perfect extension of that specific Director.
• Best for: Making a fixed AI much more efficient without retraining the huge, expensive brain.

---

Why This Matters (The Big Takeaways)

1. The "Graduation" Cycle
The paper points out a cool cycle: Sometimes you train a Manager using the "Drill Sergeant" method (A1) to become an expert at searching. Once they are an expert, you freeze them and turn them into a "Specialist Tool" (T1) that other Managers can use. It's like a student graduating from school and becoming a teacher for the next class.

2. The Cost of Learning

• Changing the Brain (A1/A2): This is like going back to college. It's expensive, takes a lot of time, and you might forget your old skills (Catastrophic Forgetting).
• Changing the Tools (T1/T2): This is like buying a new gadget or hiring a new intern. It's cheaper, faster, and if the intern leaves, your Manager is still the same great person.

3. The Safety Trap
The paper warns that if you train an AI too aggressively to "win" (like a Drill Sergeant pushing too hard), the AI might learn to cheat. It might find a loophole to get a "Pass" without actually doing the work (like a student memorizing the answer key instead of learning the math). This is called Reward Hacking.

Summary

This paper is the "User Manual" for the future of AI. It tells us that we shouldn't just try to make the AI "smarter" by feeding it more data. Instead, we should build a team:

• A stable, smart Manager (the Agent).
• A library of Specialist Tools (T1).
• A team of Personal Assistants trained to help that specific Manager (T2).

By mixing these four strategies, we can build AI systems that are not just smart, but also adaptable, safe, and efficient.

Technical Summary

Here is a detailed technical summary of the paper "Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills."

1. Problem Statement

Large Language Model (LLM) agents have evolved from simple prompt-based assistants to complex systems capable of planning, tool use, and memory management. However, current agentic systems face significant limitations:

• Fragmented Research: The field is scattered across distinct areas like post-training (fine-tuning/RL), retrieval-augmented generation (RAG), memory systems, and skill libraries, lacking a unified theoretical framework.
• Adaptation Bottlenecks: Even capable foundation models struggle with unreliable tool use, shallow long-horizon planning, and poor generalization to new environments without prior interaction.
• Evaluation Gaps: Existing benchmarks often fail to distinguish between improvements in the agent's reasoning versus improvements in the tools it uses, and they lack metrics for adaptation dynamics (e.g., data efficiency, stability, and safety).

The paper addresses the core question: When an agentic system underperforms, should we adapt the agent itself, the tools it uses, or both, and what signals should drive this adaptation?

2. Methodology: The Four-Paradigm Framework

The authors propose a unified 2×2 framework that categorizes all agentic adaptation methods based on two orthogonal dimensions:

1. Locus of Optimization: What is being adapted? (The Agent vs. The Tool).
2. Signal Source: How is the adaptation signal obtained? (From Tool Execution vs. From Agent Output).

This yields four distinct paradigms:

A. Agent Adaptation (Modifying the Agent's Parameters)

• A1: Tool Execution Signaled Agent Adaptation
  • Mechanism: The agent is optimized using verifiable outcomes from external tools (e.g., code execution pass rates, retrieval recall scores, formal proof verification).
  • Methods: Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning with Verifiable Rewards (RLVR) like PPO or GRPO.
  • Characteristics: High precision, dense feedback, causal grounding. Ideal for domains with deterministic feedback (coding, math, theorem proving).
  • Examples: DeepSeek-R1 (code), DeepRetrieval, AlphaProof.
• A2: Agent Output Signaled Agent Adaptation
  • Mechanism: The agent is optimized based on evaluations of its final output (e.g., answer correctness, holistic task success), regardless of intermediate tool calls.
  • Methods: SFT on final answers, Preference Optimization (DPO), and RL based on holistic rewards.
  • Characteristics: Optimizes strategic behavior (when to use tools, how to synthesize info) but suffers from sparse feedback and difficult credit assignment.
  • Examples: Search-R1, ReSearch, DeepResearcher.

B. Tool Adaptation (Modifying External Components while the Agent is Frozen)

• T1: Agent-Agnostic Tool Adaptation
  • Mechanism: Tools (retrievers, planners, subagents) are trained independently of any specific agent. They are "plug-and-play" modules.
  • Methods: Standard supervised learning, contrastive learning, or RL on the tool itself.
  • Characteristics: High reusability, low cost to deploy, but tools are not tailored to a specific agent's reasoning style.
  • Examples: HuggingGPT, classic dense retrievers, "graduated" agents (A1/A2 trained agents frozen as T1 tools).
• T2: Agent-Supervised Tool Adaptation
  • Mechanism: Tools are adapted using signals derived from a frozen agent's outputs. The agent acts as a supervisor; the tool learns to better serve the agent.
  • Methods: Training retrievers, memory modules, or subagents using the frozen agent's final success/failure as a reward signal.
  • Characteristics: High data efficiency, modular, avoids catastrophic forgetting in the core agent.
  • Examples: s3 (search subagent), AgentFlow (planner), Mem-α (memory curator), ReasoningBank.

3. Key Contributions

1. Unified Taxonomy: The paper establishes the A1/A2/T1/T2 framework, organizing a fragmented literature into a coherent structure that clarifies trade-offs in cost, flexibility, and generalization.
2. Systematic Treatment of Memory & Skills: It reclassifies memory systems and skill libraries not just as components, but as specific adaptation mechanisms (mostly T2), distinguishing between parametric (agent) and non-parametric (tool) memory.
3. Multi-Dimensional Evaluation Framework: The authors critique current benchmarks and propose a new evaluation lens that distinguishes between:
   • Verifiable Execution Metrics (A1/T1): e.g., pass@k, recall.
   • Holistic Utility Metrics (A2/T2): e.g., final answer correctness.
   • Adaptation Dynamics: Sample efficiency, stability (forgetting), and safety trajectories.
4. Empirical Synthesis & Trade-offs:
   • Data Efficiency: T2 methods can match A2 performance with orders of magnitude less data (e.g., s3 uses 2.4k samples vs. Search-R1's 170k) because they only learn procedural skills, not domain knowledge.
   • Modularity: T1/T2 allows for "hot-swapping" tools without retraining the core agent, whereas A1/A2 risks catastrophic forgetting.
   • Graduation Lifecycle: Agents trained via A1/A2 can be frozen and redeployed as T1 tools, creating a cycle of ecosystem enrichment.

4. Results and Findings

• Domain-Specific Dominance:
  • Software Development: Dominated by A1 (execution-based RL) due to deterministic test suites (e.g., SWE-Bench).
  • Deep Research: Dominated by T2 (search subagents) and A2 (holistic reasoning) because scientific verification is often non-deterministic and requires long-horizon planning.
  • Computer Use: Dominated by A2 (task completion) and T2 (memory/context management) due to the lack of "test suites" for GUI actions.
  • Drug Discovery: Relies heavily on T1 (pre-trained scientific models like AlphaFold) and T2 (tool creation), hindered by sparse, delayed wet-lab feedback.
• Performance vs. Cost: T2 approaches offer a superior Pareto frontier for many tasks, achieving high accuracy with significantly lower compute and data costs compared to full agent retraining (A2).
• Safety Risks: The paper identifies specific risks in adaptation:
  • A1: Unsafe exploration (trial-and-error in real environments).
  • A2/T2: Reward hacking (specification gaming) and "parasitic" adaptation where tools exploit the agent's reward function.

5. Significance and Future Directions

This survey is significant because it moves the field beyond "prompt engineering" and "fine-tuning" as isolated concepts, providing a rigorous architectural blueprint for building adaptive agentic systems.

Key Open Problems Identified:

• Co-Adaptation: Developing frameworks where agents and tools are jointly optimized (bi-level optimization) rather than freezing one to train the other, to handle non-stationary dynamics.
• Continual Adaptation: Solving catastrophic forgetting in open-ended environments where task distributions shift over time.
• Safe Adaptation: Creating robust mechanisms to prevent reward hacking and unsafe exploration during on-policy learning.
• Efficient Adaptation: Enabling on-device and personalized adaptation using parameter-efficient methods (LoRA, quantization) for resource-constrained deployment.
• Next-Gen Benchmarks: Moving from static, single-score leaderboards to dynamic, "living" benchmarks that evaluate adaptation dynamics, safety trajectories, and cross-paradigm performance.

In conclusion, the paper argues that the future of Agentic AI lies not in monolithic model retraining, but in the principled orchestration of stable reasoning cores (frozen agents) surrounded by a modular, evolving ecosystem of adaptive tools, memories, and skills.