Technical Summary: MetaGen: Self-Evolving Roles and Topologies for Multi-Agent LLM Reasoning

1. Problem Statement

Current Multi-Agent Systems (MAS) utilizing Large Language Models (LLMs) typically rely on rigid, pre-defined architectures. These systems suffer from three primary limitations:

1. Task Mismatch: Fixed role libraries (e.g., planner, solver, verifier) and static interaction topologies often fail to adapt to varying task granularities, tool preferences, and error modes, particularly under distribution shifts.
2. Structural Closure: Once an interaction topology (e.g., chain, star, or fully connected graph) is determined, it remains "execution-frozen." It cannot be revised mid-run to incorporate new evidence or resolve contradictions.
3. High Cost: Tailoring prompts and interaction structures to specific tasks requires significant manual engineering, and rigid systems often incur inflated inference costs due to redundant or mismatched agent interactions.

While recent work has explored automating topology design, most approaches still assume a pre-defined role library and freeze the graph structure once execution begins. There is a gap in frameworks that can simultaneously generate necessary roles and evolve collaboration structures entirely at inference time without updating base model weights.

2. Methodology: MetaGen

MetaGen is a training-free framework that treats both role specifications and collaboration topologies as editable, first-class entities during inference. It operates without modifying the underlying LLM parameters, relying instead on lightweight feedback signals to drive adaptation.

Core Components

• Architect Agent: Synthesizes and revises query-conditioned role specifications to form a controllable, dynamic role pool.
• Generative Role Space:
  • Candidate Synthesis: The Architect generates a raw set of candidate roles conditioned on the input query.
  • Constraint-Based Filtering: Candidates undergo strict validation against schema requirements (e.g., placeholders) and safety constraints (e.g., restricted keywords).
  • Embedding-Based Diversity Gating: To prevent semantic redundancy, roles are projected into a vector space. A Marginal Utility Score balances external novelty (against a historical library) and internal distinctiveness (against other candidates) to select a diverse set of roles.
• Task-Adaptive Graph Construction:
  • Hybrid Initialization: The system anchors reasoning to a minimal functional backbone (e.g., a dispatch-producer-evaluator chain for code generation). It then expands this skeleton using a hybrid pool of accumulated generalist roles and query-synthesized roles.
  • Score-Based Selection: A linear priority scoring mechanism selects roles and edges based on feature vectors combining lexical cues, capabilities, and semantic relevance.
• Inference-Time Evolution Loop:
  • Intra-task Evolution: During execution, the system collects lightweight feedback (runtime logs, test outcomes, self-consistency checks). If performance is suboptimal, it performs two types of edits:
    1. Role Prompt Rewrite: Modifies system/user templates for low-utility roles to better align with instance requirements.
    2. Prior-Filtered Edge Exploration: Selectively deactivates or swaps edges to encourage simpler, more informative communication while maintaining DAG constraints.
  • Inter-task Evolution: After an instance completes, a scalar reward (balancing success and token cost) updates shallow priors (weights for role/edge scoring). This allows the system to learn from past instances without fine-tuning the backbone LLM.
  • Verified Role Solidification: Roles that contribute to successful executions are serialized and stored in a persistent "Role Cache" for reuse in future instances, improving cold-start behavior.

3. Key Contributions

1. MetaGen Framework: A training-free approach that adapts both role specifications and communication topology during inference, eliminating the need for fixed presets or model weight updates.
2. Controllable Dynamic Role Pool: Introduces query-conditioned role generation and revision with lightweight validity constraints, ensuring roles are executable, non-redundant, and semantically unique.
3. Inference-Time Evolution Loop: Develops a mechanism to update prompts and structural decisions under explicit constraints (e.g., DAG preservation, cost bounds), maintaining auditability and reproducibility.
4. Empirical Validation: Demonstrates consistent improvements in the accuracy–cost trade-off over competitive multi-agent baselines across diverse benchmarks.

4. Experimental Results

Experiments were conducted on five benchmarks: GSM8K (math), HumanEval (code), MMLU (knowledge), AQuA (algebra), and MNLI (natural language inference), using DeepSeek-V3 as the backbone.

• Performance: MetaGen achieved the highest average accuracy (95.1%), outperforming the strongest baseline (G-Designer) by 1.8%. Notable gains were observed on AQuA (+5.1% over ARG-Designer) and MNLI (+9.4% over CoT few-shot).
• Cost Efficiency: MetaGen reduced inference token usage by approximately 85–90% compared to other multi-agent systems (e.g., G-Designer, GPTSwarm) and required zero training tokens for role or topology design.
• Adaptability: In non-stationary stream evaluations (shifting between MMLU, MNLI, and HumanEval), MetaGen showed superior cold-start recovery, improving accuracy by 10 points over fixed-topology baselines immediately after distribution shifts while reducing token costs.
• Robustness: The system demonstrated graceful degradation under noisy conditions (corrupted nodes/edges), maintaining performance through redundant reasoning routes rather than collapsing.
• Ablation Studies: Removing any core component (Role Generation, Learned Policy, Intra-task Evolution, or Cross-instance Memory) resulted in measurable performance drops, confirming the complementary value of dynamic roles, within-instance refinement, and cross-instance accumulation.

5. Significance and Claims

The paper posits that MetaGen addresses the core limitations of rigid Multi-Agent Systems by enabling self-evolving roles and topologies entirely at inference time. Its significance lies in:

• Scalability and Adaptability: Offering a path to scalable MAS that can tailor collaboration strategies to instance-specific needs without the computational overhead of retraining or the engineering overhead of manual prompt design.
• Engineering Efficiency: Reducing the "manual engineering" burden by automating the discovery of effective roles and interaction patterns, achieving near-saturated performance with significantly less hand-written specification.
• Auditability: Providing a structured, inspectable log of generated roles, structural edits, and feedback triggers, supporting reproducibility beyond ad-hoc orchestration.

The authors conclude that optimizing text-level roles and discrete collaboration structures at inference time is a practical and effective strategy for enhancing multi-agent reasoning capabilities without modifying backbone model weights.