Scaling Teams or Scaling Time? Memory Enabled Lifelong Learning in LLM Multi-Agent Systems

1. Problem Statement

Large Language Model (LLM) Multi-Agent Systems (MAS) are typically scaled in two isolated dimensions:

1. Horizontal Scaling: Increasing the number of agents in a team to handle complex tasks through parallelism and diverse expertise.
2. Temporal Scaling (Lifelong Learning): Improving system performance over time by accumulating and reusing experience across sequential tasks.

The Gap: Existing research treats these dimensions separately. It is unclear how they interact under realistic cost constraints. Specifically, it is unknown whether simply adding more agents yields better long-term performance, or if the "quality" of learning (memory design) can allow smaller teams to outperform larger ones. Furthermore, current memory designs often fail to support effective cross-task learning in multi-agent settings, leading to information fragmentation, coordination overhead, and high token costs due to context inflation.

Core Question: How does memory design influence the trade-off between scaling team size and scaling learning over time in LLM multi-agent systems?

---

2. Methodology: LLMA-Mem

The authors propose LLMA-Mem, a lifelong memory framework designed to support flexible memory topologies and enable cross-task knowledge transfer.

A. Memory Architecture

LLMA-Mem decomposes memory into three interdependent modules:

1. Episodic Memory ($E$): Stores raw, task-level execution records (context, actions, outcomes, reflections). It serves as the substrate for future abstraction.
   • Structure: $\langle \text{agent\_id}, \text{timestamp}, \text{task\_desc}, \text{team\_comp}, \text{actions}, \text{outcome}, \text{context}, \text{lessons}, \text{links} \rangle$.
2. Procedural Memory ($P$): Abstracts reusable strategies, skills, and workflows from successful episodic experiences. This enables efficient transfer to new tasks.
   • Structure: $\langle \text{owner}, \text{creation\_time}, \text{title}, \text{knowledge\_content}, \text{success\_count}, \text{failure\_count}, \text{source\_episodes} \rangle$.
   • Reliability: Estimated via empirical success rate ($\rho = \frac{s}{s+f}$).
3. Transactive Memory ($T$): Models "who knows what" and "which team configurations work best." It tracks agent specializations, proficiency, collaboration history, and team-level success rates to optimize task allocation and team formation.

B. Memory Topologies

The framework supports three distinct topologies to manage knowledge distribution:

• Local: Each agent maintains private memory stores. Knowledge transfer occurs only via explicit communication.
• Shared: All agents access a centralized memory pool.
• Hybrid: Episodic memory remains local (preserving role-specific context), while procedural memory and team statistics are shared.

C. Memory Lifecycle

The system operates in a continuous loop:

1. Retrieval: Uses a hierarchical strategy (Procedural $\to$ Episodic) with a scoring function combining semantic relevance and memory importance (success rate or task score).
2. Update: After each task, new episodes are appended, and success/failure counters for procedures and agent profiles are updated.
3. Consolidation: Periodically (every $N$ tasks), the system clusters semantically similar successful episodes, extracts generalized strategies, and creates new procedural memories, removing redundancy.

---

3. Key Contributions

1. Joint Scaling Perspective: Introduces a conceptual framework viewing MAS scaling as an interacting space of Team Size vs. Lifelong Learning Ability, rather than isolated axes.
2. LLMA-Mem Framework: Proposes a novel memory architecture separating episodic, procedural, and transactive memory to facilitate cross-task transfer and team coordination.
3. Discovery of Non-Monotonic Scaling: Empirically demonstrates that larger teams do not always yield better long-term performance. Under specific memory conditions, smaller teams can outperform larger ones due to reduced coordination overhead and better experience reuse.
4. Cost-Efficiency: Shows that effective memory consolidation reduces token consumption significantly compared to baselines that rely on raw context replay.

---

4. Experimental Results

Setup

• Benchmark: MultiAgentBench (Collaborative settings in Coding, Research, and Database environments).
• Models: Tested on Claude-Sonnet-4.5, DeepSeek-V3.2, Qwen3-next-80B, and Qwen3-32B-Instruct.
• Baselines: No-Memory, MARBLE (standard multi-agent memory), and A-Mem (evolutionary memory).

Performance Metrics

• Task Score (TS): Quality of final output.
• Communication Score (CS): Quality of inter-agent planning.
• AAS (Average of AS): Cumulative average performance over the task sequence (measuring lifelong learning).
• CMA (Cumulative Moving Average): Tracks performance gain relative to the no-memory baseline over time.

Key Findings

1. Superior Long-Horizon Performance: LLMA-Mem consistently outperformed baselines in AAS across most model-environment pairs. For example, it improved AAS by +5.92 for DeepSeek-v3.2 in Research and +3.19 for Qwen3-32B-Instruct in Database.
2. Stability: Unlike baselines (MARBLE, A-Mem) which often showed performance decay or negative cumulative gains over long sequences, LLMA-Mem maintained positive, stable improvement trajectories.
3. Cost Reduction: LLMA-Mem reduced average token usage by 9.4% to 71.7% compared to baselines. This is achieved by consolidating long context traces into compact procedural memories, avoiding the "context inflation" seen in other frameworks.
4. Non-Monotonic Team Scaling:
   • Increasing team size (1 to 7 agents) generally increased token costs linearly or super-linearly.
   • Performance gains were not monotonic. For instance, a 3-agent team sometimes outperformed a 5-agent team, and a 5-agent team outperformed a 7-agent team in specific settings (e.g., Qwen3-32B-Instruct).
   • Insight: Larger teams introduce coordination overhead and information fragmentation that can negate the benefits of parallelism, especially if memory does not effectively aggregate experience.
5. Ablation Studies:
   • Topology: Local Topology performed best for lifelong learning in coding tasks, as it allowed agents to build role-specific procedural knowledge without cross-role interference.
   • Consolidation Interval: A moderate interval ($N=5$) was optimal. Too frequent ($N=2$) led to unstable procedures; too sparse ($N=10, 20$) delayed knowledge reuse.

---

5. Significance and Conclusion

This paper fundamentally shifts the paradigm for scaling LLM multi-agent systems. It argues that memory design is the primary lever for effective scaling, not just adding more agents.

• Efficiency over Brute Force: The study proves that "scaling time" (better memory and learning) can be more effective and cost-efficient than "scaling teams" (adding agents).
• Practical Guidance: It provides a roadmap for practitioners: prioritize memory architectures that support procedural abstraction and transactive memory (knowing who knows what) over simply increasing team size.
• Future Direction: The findings suggest that the optimal configuration for a multi-agent system depends on the specific task and the memory topology, challenging the assumption that "more agents = better results."

The code is open-sourced, enabling further research into the intersection of multi-agent coordination and lifelong learning.