💬 NLP

Toward Efficient Agents: Memory, Tool learning, and Planning

This paper investigates the often-overlooked efficiency of agentic systems by reviewing recent advancements in memory, tool learning, and planning, characterizing the trade-off between effectiveness and cost through a Pareto frontier perspective, and summarizing evaluation protocols, metrics, challenges, and future directions.

Original authors: Xiaofang Yang, Lijun Li, Heng Zhou, Tong Zhu, Xiaoye Qu, Yuchen Fan, Qianshan Wei, Rui Ye, Li Kang, Yiran Qin, Daizong Liu, Qi Li, Ning Ding, Siheng Chen, Jing Shao

Published 2026-07-07

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Xiaofang Yang, Lijun Li, Heng Zhou, Tong Zhu, Xiaoye Qu, Yuchen Fan, Qianshan Wei, Rui Ye, Li Kang, Yiran Qin, Daizong Liu, Qi Li, Ning Ding, Siheng Chen, Jing Shao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, hyper-intelligent assistant (an AI Agent) who can solve almost any problem. But there's a catch: this assistant is incredibly expensive to run. It eats up money, time, and computer power with every single thought it has. If you ask it a simple question, it might spend hours "thinking" and make hundreds of phone calls to other services just to find the answer.

This paper, "Toward Efficient Agents," is like a manual for turning that expensive, over-enthusiastic assistant into a lean, mean, cost-effective machine. The authors argue that while making agents smarter is great, making them efficient is what will actually let us use them in the real world without going broke.

Here is the breakdown of their ideas using simple analogies:

The Core Problem: The "Over-Thinker"

Think of a standard AI agent like a person who refuses to write anything down. Every time they face a new problem, they try to remember everything that ever happened to them, re-read their entire life story, and call every possible expert in the world before making a move.

The Result: They eventually get the answer, but they spent a fortune on paper (tokens), took days to do it (latency), and called 500 people (tool use) when they only needed to call one.
The Goal: The paper defines an "Efficient Agent" not as a smaller brain, but as a smart manager. It's an agent that gets the job done just as well, but uses fewer resources.

The authors break down efficiency into three main areas: Memory, Tools, and Planning.

1. Efficient Memory: The "Smart Filing Cabinet"

Imagine your agent's memory is a giant warehouse.

The Old Way: Every time you talk to the agent, you dump the entire conversation into the warehouse. Next time, the agent has to search through every single box to find one relevant fact. This is slow and messy.
The Efficient Way: The paper suggests three upgrades:
- Compression (Summarizing): Instead of keeping the full transcript of a 3-hour meeting, the agent writes a one-page summary. It keeps the important points but throws away the "umms" and "ahhs."
- The "Filing System" (Management): Instead of just piling papers on a desk, the agent uses a smart system. It throws away old, irrelevant notes (forgetting) and highlights the most important ones. Some agents even use a "rule-based" system (like a librarian who automatically files books by color) to save time, while others use the AI itself to decide what to keep (more accurate but slower).
- The "Secret Code" (Latent Memory): Sometimes, instead of writing things down in English, the agent stores memories as a secret code inside its own brain. This is super fast to access but hard for humans to read. It's like having a mental shortcut that saves you from having to read a whole book.

The Takeaway: Don't hoard everything. Summarize, organize, and sometimes store things in a way that's fast for the machine, even if it's weird for humans.

2. Efficient Tool Use: The "Smart Shopper"

Agents use "tools" like search engines, calculators, or code writers.

The Old Way: The agent is like a shopper who walks into a massive department store and asks every single clerk for help, even for things they could have figured out themselves. They buy 50 items when they only needed 2.
The Efficient Way:
- Selection: Before entering the store, the agent checks a list to see exactly which aisle it needs. It doesn't wander aimlessly.
- Parallel Shopping: If the agent needs to check the weather in 10 different cities, the old way is to call the weather service 10 times, one by one. The efficient way is to call all 10 at once (parallel calling).
- Knowing When to Stop: The agent learns to say, "I know the answer; I don't need to ask a tool." It stops wasting money on unnecessary calls.

The Takeaway: Don't call for help unless you really need it, and if you do, call everyone you need at the same time.

3. Efficient Planning: The "Strategic General"

Planning is how the agent decides what to do next.

The Old Way: The agent is like a general who tries to plan every single move of a war before making a single step. They simulate thousands of battles in their head, wasting time, before finally deciding to march forward.
The Efficient Way:
- Fast vs. Slow Thinking: Sometimes, the agent should just "gut check" and act fast (System 1 thinking). Other times, it should stop and think deeply (System 2). Efficient agents know the difference and don't overthink simple tasks.
- Breaking it Down: Instead of trying to solve a huge problem in one go, the agent breaks it into small, manageable chunks (like a recipe).
- Learning from Mistakes: If the agent tries a path and fails, it learns from that failure so it doesn't try the same wrong path again. It "distills" its experience into a skill so it doesn't have to re-learn it next time.

The Takeaway: Don't over-plan. Know when to act fast, break big problems into small ones, and learn from your mistakes so you don't repeat them.

The Golden Rule: Effectiveness First

The paper makes a crucial point: Efficiency doesn't matter if the agent fails.
If you make an agent so cheap and fast that it gives you the wrong answer, you haven't saved anything. You've just wasted time.

The Analogy: It's like buying a super-cheap, fast car that breaks down every mile. It's "efficient" at moving fast, but it's useless because it never gets you to your destination.
The Solution: The authors say we must measure efficiency alongside success. We want the agent to get the job done and do it cheaply.

How Do We Measure This?

The paper reviews how we test these agents. Currently, it's a bit of a mess. Some people count how many words the agent wrote (tokens), others count how long it took (time), and others count how many tools it used.
The authors are calling for a standardized scoreboard. Just like a video game has a score for "points" and a score for "time," we need a standard way to measure an agent's "success rate" and its "cost" so we can compare different methods fairly.

Summary

This paper is a roadmap for building AI agents that are smart enough to solve problems but frugal enough to be affordable. It teaches us to:

Organize memory so the agent doesn't drown in its own history.
Use tools wisely so it doesn't waste money on unnecessary calls.
Plan strategically so it doesn't overthink simple tasks.

The ultimate goal is to move from "expensive, experimental AI" to "practical, everyday AI" that can actually be used in the real world without breaking the bank.

Technical Summary: Toward Efficient Agents: A Survey of Memory, Tool Use, and Planning

1. Problem Statement

The rapid evolution from Large Language Models (LLMs) to agentic systems has introduced a critical bottleneck: efficiency. While LLM-based agents demonstrate superior capabilities in autonomous action, complex reasoning, and tool interaction, their recursive, multi-step nature incurs exponential resource consumption compared to standard linear query-response models. Agents must manage long interaction histories, perform iterative tool usage, and execute complex planning loops, leading to prohibitive latency, context window saturation, and excessive token consumption.

Existing surveys have addressed individual components (memory, tool use, planning) in isolation or focused on model-level efficiency (e.g., compression). However, there is a lack of system-level analysis addressing how to reduce resource consumption (tokens, latency, steps, API calls) while maintaining task effectiveness. This paper defines an Efficient Agent not merely as a smaller model, but as a system optimized to maximize task success rates while minimizing resource consumption across memory, tool usage, and planning modules.

2. Methodology and Taxonomy

The paper conducts a comprehensive survey of recent approaches, categorizing efficiency strategies into three core components of the agentic workflow: Memory, Tool Use, and Planning. The authors adopt an effectiveness-first perspective, arguing that an agent is only meaningfully efficient if it preserves acceptable task quality. Efficiency is characterized in two complementary ways:

Comparing effectiveness under a fixed cost budget.
Comparing cost at a comparable level of effectiveness.

2.1 Efficient Memory

Memory is identified as a key mechanism for improving the efficiency-effectiveness trade-off by avoiding redundant computation and costly retries. The survey organizes memory techniques by their lifecycle:

Construction:
- Latent and Parametric Memory: Compresses experience into hidden states, KV caches, or model parameters (e.g., Activation Beacon, MemoryLLM, Titans) to avoid serializing full histories. This trades interpretability for reduced token overhead.
- Textual Memory: Represents experience as explicit text. Strategies include Prompt-resident (rewriting/compressing context), Item-based (discrete units like events or insights, e.g., MemoryBank, Expel), Graph-based (structured entities and relations, e.g., Zep, AriGraph), and Hierarchical (multi-tier storage, e.g., MemGPT, ReadAgent).
Management: Controls the memory lifecycle via Rule-based (e.g., Ebbinghaus forgetting, FIFO), LLM-based (semantic decisions on updates/deletion), or Hybrid approaches (combining cheap rules with semantic LLM checks).
Access: Determines retrieval strategies, including Rule-enhanced (recency/frequency scoring), Graph-based traversal, LLM/Tool-based active search, and Hierarchical coarse-to-fine retrieval.
Procedural Reuse (Skills): Treats skills as procedural memory, amortizing the cost of planning and tool invocation by reusing distilled procedures (e.g., Skill-Pro, Trace2Skill).
Multi-Agent Memory: Addresses shared, local, and mixed memory designs to balance coordination efficiency with role specialization.

2.2 Efficient Tool Use

Efficient tool use focuses on reducing the number of calls, optimizing the invocation trajectory, and minimizing interaction costs.

Tool Selection: Strategies to narrow the action space before generation, including External Retrievers (semantic matching), Multi-Label Classification (treating selection as a classification task), and Vocabulary-based Retrieval (treating tools as special tokens, e.g., ToolkenGPT).
Tool Calling: Optimizes the execution phase via In-Place Parameter Filling (filling params during generation), Parallel Calling (executing independent calls simultaneously, e.g., LLMCompiler), and Cost-Aware Calling (formulating calls as knapsack problems or using budget constraints).
Tool-Integrated Reasoning (TIR): Optimizes the reasoning trajectory itself. This includes Selective Invocation (learning when not to call tools), Cost-Aware Policy Optimization (using RL to penalize redundant calls), and Structured Rewards to balance correctness with efficiency.

2.3 Efficient Planning

Planning is reframed as a budget-aware control problem, deciding how much reasoning, search, or coordination is necessary.

Single-Agent Planning:
- Inference-Time Strategies: Includes Adaptive Control (fast/slow thinking, budget-aware allocation), Structured Search (MCTS, A* search with cost pruning), and Task Decomposition (breaking tasks into sub-goals to avoid monolithic reasoning).
- Learning-Based Evolution: Uses Policy Optimization (RL, DPO) to internalize planning logic and Memory/Skill Acquisition to amortize planning costs over future tasks.
Multi-Agent Collaborative Planning: Focuses on reducing coordination overhead. Strategies include Topological Efficiency (structuring communication graphs to reduce $O(N^2)$ complexity), Protocol Optimization (compressing message content), and Distillation (transferring collaborative reasoning into single-agent models).

3. Key Contributions

System-Level Efficiency Framework: The paper moves beyond component-centric views to define agent efficiency as a system-level optimization problem involving the interplay of memory, tools, and planning.
Comprehensive Taxonomy: It provides a structured classification of hundreds of recent works, organizing them by mechanism (e.g., latent vs. textual memory, parallel vs. sequential tool calling) and lifecycle stage (construction, management, access).
Benchmark and Metric Synthesis: The survey aggregates existing benchmarks (e.g., GAIA, SWE-Bench, ToolBench) and efficiency metrics (tokens, latency, API cost, interaction steps), highlighting the current lack of unified evaluation standards.
Identification of Trade-offs: The authors explicitly discuss the trade-offs between efficiency and effectiveness, such as the risk of information loss in compression or the latency overhead of complex retrieval mechanisms.

4. Results and Findings

The survey does not present new experimental results but synthesizes findings from the literature:

Convergence of Principles: Despite diverse implementations, many methods converge on high-level principles such as bounding context via compression, designing RL rewards to minimize tool invocations, and employing controlled search mechanisms.
Lifecycle Costs: Efficiency gains in one module (e.g., memory compression) often shift costs to another (e.g., retrieval latency or update overhead), necessitating a holistic view.
Effectiveness-First Necessity: Methods that reduce costs but degrade task success rates are deemed not meaningfully efficient. The Pareto frontier between cost and effectiveness is the primary evaluation target.
Emerging Trends: There is a shift from static tool lists to dynamic retrieval, from sequential execution to parallel orchestration, and from unbounded reasoning to budget-constrained deliberation.

5. Significance and Future Directions

The paper claims significance in providing a foundational resource for researchers aiming to deploy agentic systems in real-world scenarios where cost and latency are critical constraints. It highlights that efficiency is not a standalone objective but a property of the entire agentic loop.

Key Challenges and Future Directions identified:

Unified Efficiency Evaluation: The field lacks standardized metrics and reporting protocols for efficiency, making cross-method comparison difficult. The authors call for a unified framework accounting for tokens, latency, memory, and tool overhead.
Agentic Latent Reasoning: The potential of performing intermediate computations in continuous hidden representations (latent space) rather than token-by-token decoding remains underexplored for agents, offering a path to reduce token overhead.
Deployment-Aware Design: Future work must distinguish between theoretical multi-agent designs and practical deployment realities (e.g., single-model role-play vs. true multi-model orchestration) to accurately assess overhead.
MLLM-Based Agents: Efficiency strategies for text-only agents need adaptation for Multimodal LLM (MLLM) agents, which face compounded challenges in visual history retention and grounding latency.

In conclusion, the survey consolidates the design space for efficient agents, emphasizing that future progress depends on jointly optimizing memory, tool use, and planning while maintaining rigorous, standardized evaluation of both effectiveness and cost.