🤖 AI

ConMem: Structured Memory-Guided Adaptation in Training-Free Multi-Agent Systems

ConMem is a training-free, relation-aware framework that enhances multi-agent system adaptation by distilling historical interactions into structured memory cards organized in a graph, enabling efficient cross-experience coordination that significantly improves performance and inference efficiency without additional training.

Original authors: Zhixun Tan, Qiang Chen, Tairan Huang, Xiu Su, Yi Chen

Published 2026-06-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhixun Tan, Qiang Chen, Tairan Huang, Xiu Su, Yi Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a team of expert AI assistants (a "Multi-Agent System") trying to solve complex problems, like writing code, planning a trip, or answering tricky trivia. These assistants are smart, but they have a major flaw: they have short-term memory. If they make a mistake or find a clever trick today, they might forget it tomorrow.

To fix this, researchers usually try to teach the AI new things (which takes a lot of time and money) or just dump a huge pile of past conversations into its brain. But dumping a huge pile of past conversations is messy. It's like trying to find a specific needle in a haystack that is on fire, full of smoke, and tangled with old, useless rags. The AI gets confused by the noise.

Enter ConMem: The "Flashcard" System for AI Teams.

The paper introduces ConMem, a clever, free way to help these AI teams learn from their past without needing to retrain them. Think of it as giving the AI team a set of structured, signed flashcards instead of a messy diary.

Here is how it works, using simple analogies:

1. The Problem: The Messy Backpack

Imagine an AI team goes on a long hike (a complex task). Along the way, they encounter obstacles, find shortcuts, and sometimes trip over rocks.

Old methods try to save the entire hike in a giant, heavy backpack. When they start a new hike, they have to dig through the whole backpack to find one useful tip. It's slow, and they often get distracted by irrelevant details (like the color of a flower they saw 10 miles ago).
The ConMem approach says: "Stop carrying the whole backpack." Instead, after the hike, we take the most important moments and turn them into Flashcards.

2. The Flashcards (Signed Strategy Cards)

ConMem takes the messy history and distills it into tiny, clean cards.

The Content: Each card has four parts: Where we were (State), What we planned (Plan), What we did (Execution), and Did it work? (Evaluation).
The "Sign": This is the magic part. Every card has a Plus (+) or Minus (-) sign.
- Plus Cards: "Hey, this trick worked! Do it again." (e.g., "Check the inventory before shipping.")
- Minus Cards: "Hey, this failed! Don't do this." (e.g., "Don't trust that specific shipping carrier; they are unreliable.")
Why it helps: Instead of reading a 50-page story about a failed delivery, the AI just sees a red card that says: "Avoid Carrier X." It's instant and clear.

3. The Map (The Relation Graph)

If you just have a pile of flashcards, you might pick two that contradict each other (e.g., one says "Go Left" and another says "Go Right").
ConMem doesn't just store cards; it draws a map connecting them.

Some cards Support each other (like a bridge).
Some cards Conflict (like a roadblock).
Some cards Satisfy a condition (like a key fitting a lock).

When the AI team faces a new task, ConMem doesn't just grab random cards. It looks at the map, finds the right path, and makes sure the cards it picks don't fight with each other. It's like a tour guide who checks the map to ensure the group doesn't get sent in two opposite directions at once.

4. The Budget (The "Briefcase" Limit)

AI models have a limit on how much text they can read at once (like a briefcase that can only hold so many papers).
ConMem is smart about space. It acts like a strict editor:

It grabs the most relevant cards.
It checks the map to resolve conflicts.
It prunes (cuts out) anything that is redundant or doesn't fit.
It packs only the most essential, conflict-free instructions into the briefcase.

The paper claims this system cuts out more than 50% of the extra noise and reduces the time spent planning by 80%, all while making the AI smarter.

The Result: A Smarter Team, No Extra Training

The best part? You don't need to teach the AI anything new. The AI's brain (the model weights) stays exactly the same. ConMem just acts as a super-organized librarian that hands the AI the perfect, pre-filtered notes right before it starts working.

In summary:

Old Way: Give the AI a messy, endless diary of past events.
ConMem Way: Give the AI a set of signed flashcards (Good tips vs. Bad warnings) connected by a map that ensures they make sense together, all packed into a small briefcase that fits perfectly.

This allows the AI team to adapt quickly, avoid past mistakes, and reuse successful strategies without needing expensive retraining or getting confused by noise.

Technical Summary: ConMem – Structured Memory-Guided Adaptation in Training-Free Multi-Agent Systems

1. Problem Statement

Large Language Model (LLM)-based Multi-Agent Systems (MAS) excel at complex continual tasks but face significant challenges in Multi-Agent Adaptation (MAA). Existing approaches to MAA generally fall into three categories, each with distinct limitations:

Memory-driven methods: Retain raw interaction histories. While they preserve context, the trajectories are often long, noisy, and entangled with local tool results or intermediate disagreements, leading to distorted decision signals and low retrieval precision.
Structured-memory/Skill-driven methods: Abstract interactions into reusable procedures or skills. While they reduce redundancy, they often lack relation-aware coordination at inference time, leaving conflicts, dependencies, and redundancies unresolved when composed.
Training-driven methods: Use reinforcement learning or fine-tuning to manage memory. While effective, these approaches require additional training, high-quality supervision, and modifications to agent roles or orchestration, increasing computational costs and implementation complexity.

The central challenge addressed by this work is: How can LLM-based MAS achieve efficient, training-free adaptation by transforming fragmented experiences into structured, coordinated, and reusable collective memory without modifying host model weights?

2. Methodology: ConMem Framework

ConMem is a relation-aware, training-free framework that enables efficient adaptation through cross-experience coordination. It operates on a "frozen-host" setting, controlling only the memory prefix injected into the prompt without altering the underlying LLM or agent orchestration.

2.1 Core Unit: The Signed Strategy Card

Instead of raw trajectories, ConMem distills historical interactions into atomic strategy cards. A card $c$ is a structured tuple containing:

Task Description ( $z$ ): The context of the interaction.
Trigger Semantics ( $\chi$ ): Conditions under which the card is relevant.
Four Structured Slots ( $\kappa$ ): Encoding State, Plan, Execution, and Evaluation.
Sign: A binary indicator distinguishing positive cards (reusable strategies) from negative cards (failure-derived warnings).

This abstraction allows the system to treat memory as a compact, inspectable, and reusable procedural signal rather than a raw log.

2.2 The Pipeline

ConMem operates via two distinct paths: a Write Path (memory construction) and a Read Path (inference-time adaptation).

A. Write Path: Building the Card Bank

After a task is scored, the system extracts candidate cards from the trajectory:

Reflection & Aggregation: Informative failures are reflected into negative cards; successes become positive cards.
Admission: Cards are admitted only if they are coherent (internal consistency of state/plan/execution) and meet quality thresholds (reliability, novelty, recency, utility).
Merging: Admitted cards are stored in a persistent bank ( $B$ $B$ ) and linked into a Typed Relation Graph ( $G$ ).
- Graph Edges: Edges carry types such as supports, satisfies, constrains, and conflicts, encoding dependencies and incompatibilities between cards.

B. Read Path: Needs-Aware Retrieval & Coordination

At runtime, for a new task $t$ , ConMem executes a four-stage process to generate a bounded memory prefix $\tilde{m}_t$ :

Needs-Aware Retrieval: Cards are ranked based on a scoring function combining semantic similarity, trigger matching, task needs alignment (estimated from the current context), and card credibility. A diversity filter (MMR) suppresses near-duplicates.
Typed-Graph Expansion: Retrieved "seed" cards are expanded by traversing the relation graph. The system follows supports and satisfies edges to recover dependencies while avoiding conflicts. This ensures the retrieved context is a coherent, dependency-complete set rather than a flat list.
Coordinated Budgeted Composition: The expanded set is reconciled to resolve local conflicts (retaining higher-quality representatives) and remove redundancies.
Serialization: The final set is serialized into a prompt prefix that strictly adheres to a token budget ( $B$ ). If the budget is exceeded, the system attempts compact serialization or skips lower-priority cards.

2.3 Theoretical Foundation

The framework is grounded in a Budgeted Context-Control view. The objective is not maximal recall but maximizing utility under a fixed token budget. The authors posit that if the card extraction is sufficient (i.e., the card contains all necessary information for the downstream outcome), then conditioning on the card is Bayes-optimal and strictly more efficient than conditioning on raw trajectories due to token compression.

3. Key Contributions

Training-Free Adaptation Formulation: The authors propose a context-controlled approach for continual MAS adaptation, framing the problem as budgeted control over reusable procedural signals, thereby avoiding the need for additional training or host modification.
Atomic Procedural Memory Units: Introduction of typed and signed cards that encode state, plan, execution, and evaluation, providing compact, reusable, and inspectable knowledge.
Relation-Aware Coordination: Development of a mechanism that resolves memory conflicts and dependencies under a context budget using a typed graph, enabling lightweight adaptation without modifying model weights.
Empirical Validation: Extensive experiments demonstrating consistent gains across diverse benchmarks and MAS architectures.

4. Experimental Results

The authors evaluated ConMem on four benchmarks (TriviaQA, PopQA, KodCode, PDDL) across three MAS hosts (AutoGen, CAMEL, MacNet) using the Qwen/Qwen3-4B-Instruct-2507 backbone.

Performance Gains: ConMem achieved consistent improvements over "No-memory" baselines, with average gains ranging from 10.9 to 12.9 absolute points.
Comparison with SOTA: ConMem outperformed strong memory baselines (including Generative Agents, Voyager, G-Memory) and matched or slightly exceeded training-based methods like LatentMem and ReMe (dynamic), despite being training-free.
- On the AutoGen host, ConMem achieved an average score of 58.33, surpassing LatentMem (57.38) and ReMe (57.46).
- It ranked first in 8 out of 15 host-metric comparisons and second in the remaining 7.
Efficiency: The coordination mechanism aggressively pruned the candidate set, removing >50% of expanded candidates in general settings and >80% in planning tasks (PDDL), significantly reducing planning overhead.
Ablation Studies: Removing any component (Graph Expansion, Coordination, Failure Reflection, or Admission) resulted in performance drops, confirming that the gains stem from the integrated pipeline rather than a single module. Notably, removing Coordination caused the largest drop in QA tasks, highlighting the necessity of conflict resolution before prompt injection.

5. Significance and Claims

The paper claims that ConMem demonstrates that long-term adaptation in frozen-host settings can be achieved through prompt-level context control rather than model retraining.

Structural Advantage: Unlike methods that retrieve flat lists of history, ConMem's relation-aware coordination resolves contradictions and dependencies before the information reaches the host LLM. This is particularly critical for tasks with high inter-card dependencies (e.g., code generation and planning), where the performance gap between ConMem and flat-retrieval baselines is largest.
Lightweight Adaptation: By transforming historical experience into compact, signed cards, ConMem enables stable performance improvements across diverse frameworks without the computational cost of training or the complexity of modifying agent roles.
Failure Utilization: The framework uniquely leverages negative cards (failure-derived warnings) as actionable signals to avoid past mistakes, treating failures as first-class citizens in the memory system.

In conclusion, ConMem posits that the future of efficient multi-agent adaptation lies not in recalling more history, but in curating and coordinating reusable strategies under strict context budgets.