🤖 machine learning

\textsc{MasFACT}: Continual Multi-Agent Topology Learning via Geometry-Aware Posterior Transfer

The paper introduces \textsc{MasFACT}, a geometry-aware posterior transfer framework that mitigates topology forgetting in continual multi-agent systems by leveraging Fused Gromov-Wasserstein optimal transport and PAC-Bayes-guided adaptation to preserve and reuse historical collaboration knowledge across evolving tasks.

Original authors: Xuefei Wang, Jialu Wang, Fengbo Zhang, Yihan Hu, Di Zhang, Yutong Ye, Yikun Ban, Jun Han, Ruijie Wang

Published 2026-05-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xuefei Wang, Jialu Wang, Fengbo Zhang, Yihan Hu, Di Zhang, Yutong Ye, Yikun Ban, Jun Han, Ruijie Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Teamwork Team" Problem

Imagine you have a team of AI assistants (agents) working together to solve problems. To work well, they need a specific way of talking to each other. Maybe Agent A talks to Agent B, who then talks to Agent C. This "conversation map" is called a topology.

The Old Way: When a new problem comes up, the team tries to invent a brand new conversation map from scratch. They forget how they solved the last problem.
The New Problem (Topology Forgetting): As the team faces a stream of different tasks (like math, then coding, then writing), they keep changing their conversation map. In doing so, they accidentally "forget" the perfect map they had for the previous tasks. It's like a group of friends trying to solve a puzzle, then a game of chess, then a cooking challenge, but every time they switch games, they forget how they were sitting and talking to each other during the previous game.

The authors call this "Topology Forgetting." The team gets better at the current task but gets worse at the old tasks because they keep overwriting their old, successful teamwork styles.

The Solution: MasFACT (The "Memory Bank" for Teamwork)

The authors propose a new system called MasFACT. Think of it as a smart librarian for the team's conversation maps. Instead of throwing away old maps, MasFACT saves them, organizes them, and helps the team pick the right one for the new job.

Here is how it works in three simple steps:

1. The "Recipe Book" (Factorized Historical Topology)

Imagine the team has solved many problems before. MasFACT doesn't just save the whole messy kitchen; it extracts the core recipes.

It separates the structure (who talks to whom) from the people (which specific agent is doing what).
It creates a "Recipe Book" (a Prior Bank) of successful teamwork patterns. For example, it saves a pattern like: "When you need to check facts, the Researcher talks to the Critic, who talks to the Writer."

2. The "Smart Matchmaker" (Geometry-Aware Retrieval)

When a new task arrives (e.g., "Write a poem"), the system doesn't just guess. It looks at the new task and asks: "Which old recipe is most similar to this?"

It uses a special math tool (called Fused Gromov-Wasserstein) to compare the new task with the old recipes.
The Analogy: Imagine you are moving to a new city. You don't just look for a house that looks exactly like your old one. You look for a house where the neighborhood layout (structure) and the types of shops nearby (semantics) feel familiar. MasFACT finds the "neighborhood" from the past that fits the new city best, even if the street names are different.

3. The "Gentle Edit" (Conservative Posterior Adaptation)

Once the team picks the best old recipe, they don't just copy-paste it. They make tiny, careful adjustments to fit the new task.

The Analogy: Think of it like wearing a favorite old jacket. You don't throw it away for a new one. Instead, you take the old jacket and maybe roll up the sleeves or add a patch to make it fit the new weather.
The system uses a safety rule (called PAC-Bayes) to ensure they don't change the jacket too much. This prevents them from accidentally ruining the parts of the jacket that were working perfectly.

Why This Matters (The Results)

The authors tested this system on a "stream" of different tasks (math, coding, trivia, etc.).

Without MasFACT: The team gets good at the current task but forgets how to do the previous ones. Their performance on old tasks drops significantly.
With MasFACT: The team learns the new task and remembers how to do the old ones. In fact, sometimes learning a new task actually helps them get better at the old ones (a phenomenon called "backward transfer").

Summary Analogy

Imagine a band playing different genres of music.

The Problem: Every time they switch from Jazz to Rock, they forget the chord progressions and rhythms they used for Jazz. They have to relearn everything from scratch.
The MasFACT Solution: They keep a "Greatest Hits" book of their best arrangements. When they switch to Rock, they look at the book, find a Jazz arrangement that has a similar rhythm, and gently tweak it to sound like Rock. They keep the core "vibe" of the old songs while adapting to the new style.

In short: MasFACT teaches AI teams how to learn new skills without forgetting their old ones, by treating past teamwork patterns as valuable, reusable assets rather than disposable drafts.

Technical Summary: MasFACT

Problem Definition

The paper addresses the challenge of Continual Multi-Agent System (MAS) Topology Learning. While Large Language Model (LLM)-based MAS have proven effective for complex reasoning, their performance relies heavily on the underlying communication topology (how agents coordinate). Existing methods typically optimize topologies for isolated, static tasks. In real-world scenarios where tasks evolve in streams, these methods suffer from topology forgetting: as the system adapts to new tasks, the topology generator drifts away from the communication structures that were effective for previous tasks.

This failure mode differs from standard parameter forgetting. It arises from cross-task misalignment in two dimensions:

Agent-level functional semantics: Roles and tools may shift meaning across tasks.
Relational communication structures: The optimal graph connections change, causing the generator to overwrite previously useful collaboration patterns.

Current approaches (search-based or generative) either overwrite past structures or suffer from distribution drift without explicit structural memory, leading to a trade-off where short-term adaptation degrades long-horizon knowledge.

Methodology: MasFACT

The authors propose MasFACT (Factorized Aligned Consensus Topology transfer), a geometry-aware posterior transfer framework designed to preserve and reuse historical collaboration knowledge. The method operates through a three-stage pipeline:

1. Factorized Historical Topology Construction

MasFACT maintains a Prior Bank ( $M$ ) of "prior atoms" derived from high-utility historical topologies. Each atom $H_m$ is factorized into three components to enable transfer across tasks with different agent identities and graph cardinalities:

Consensus Topology ( $S_m$ ): A directed graph encoding reusable communication structures.
Prototype Attributes ( $X_m$ ): Agent attributes capturing transferable functional semantics (roles, tools).
Node Measure ( $\nu_m$ ): A measure over prototype nodes for optimal transport alignment.
Metadata: Each atom also stores historical utility ( $\bar{U}_m$ ) and structural dispersion ( $\sigma_m$ ) to guide retrieval.

2. Geometry-Aware Prior Retrieval

When a new task $T_t$ arrives, MasFACT retrieves the most transferable prior atom. Instead of simple index matching, it formulates retrieval as an Optimal Transport problem using Fused Gromov-Wasserstein (FGW) distance.

Alignment: It aligns the current task's scaffold graph (agents and tentative structure) with historical atoms by jointly minimizing the cost of matching agent attributes (semantics) and graph relations (structure).
Retrieval Criterion: The system selects the atom $H_{m_t}$ that balances geometric transferability (FGW cost), structural concentration (dispersion), and historical utility.
Projection: The selected consensus topology is projected into the current agent space via the optimal coupling matrix, creating a continuous structural prior ( $C_t$ ) that serves as a reference point.

3. Conservative Posterior Adaptation

To adapt to the new task without forgetting, MasFACT learns a stochastic posterior ( $q_t$ ) centered around the aligned prior ( $C_t$ ), rather than searching the entire topology space.

Residual Learning: The posterior is defined as a perturbation of the prior: $q_t(B) \propto \exp(\langle C_t + R_t, B \rangle)$ , where $R_t$ is a sparse, learnable residual score matrix. This ensures adaptation is limited to necessary local edits.
PAC-Bayes Constraint: The training objective is guided by a PAC-Bayes transfer bound. The loss function minimizes empirical risk while penalizing the complexity of the posterior relative to the prior (KL divergence) and enforcing sparsity on the residual edits. This theoretically bounds the risk of topology forgetting by controlling structural drift.

Key Contributions

Problem Formulation: The authors identify and formalize topology forgetting as a distinct failure mode in continual MAS learning, caused by misalignment in both topology space and agent semantics.
Novel Framework (MasFACT): They introduce a geometry-aware framework that treats historical topologies as transferable structural priors. By combining FGW optimal transport for cross-task alignment with PAC-Bayes-guided conservative posterior adaptation, MasFACT balances plasticity (adapting to new tasks) and stability (retaining old topologies).
Evaluation Protocol: The paper proposes a hierarchical continual MAS evaluation protocol spanning task-level (different reasoning families), domain-level (different datasets within a family), and class-level (fine-grained sub-skills) evolution across four reasoning domains and 17 datasets.

Experimental Results

Experiments were conducted across 12 continual scenarios using Llama-3.1-8B as the backbone. MasFACT was compared against single-agent methods, fixed topologies, and state-of-the-art adaptive topology generators (e.g., GDesigner, MasRouter, ARGDesigner), including replay-based variants.

Performance: MasFACT consistently outperformed all baselines. Across all settings, it improved Average Accuracy (AA) by approximately 9.3% and reduced Average Forgetting (AF) by 69.4% compared to strong baselines.
Forgetting Mitigation: Unlike other methods that often exhibit positive forgetting (performance degradation on old tasks), MasFACT frequently achieved negative forgetting (positive backward transfer), indicating that learned collaboration patterns improved performance on previous tasks.
Plug-and-Play: MasFACT served as an effective continual layer for diverse topology generators (pruning-based, graph-generation, and routing-based), improving their performance without modifying their core architectures.
Ablation: Removing the historical prior, FGW alignment, or PAC-Bayes constraints significantly degraded performance, confirming that the joint alignment of structure and semantics, combined with conservative adaptation, is critical.

Significance and Claims

The paper claims that topology forgetting is a structural phenomenon distinct from parameter forgetting, requiring a shift from parameter-centric continual learning to structure-centric transfer.

Structural Knowledge as Priors: The work demonstrates that effective collaboration topologies are reusable structural knowledge that can be factorized, aligned, and transferred, rather than being re-discovered from scratch for every task.
Geometry-Aware Transfer: By utilizing FGW, the method successfully handles the misalignment of agent roles and graph structures across tasks, a limitation of standard Euclidean or index-based retrieval.
Stability-Plasticity Balance: The PAC-Bayes formulation provides a theoretical guarantee that allows the system to adapt to new tasks while strictly bounding the deviation from proven historical structures.

The authors conclude that MasFACT offers a robust, architecture-agnostic solution for scaling reliable and adaptive LLM-based MAS in dynamic, evolving environments, moving beyond static, one-shot topology design.