Technical Summary: RADAR - Redundancy-Aware Diffusion for Multi-Agent Communication Structure Generation

Problem Statement

Large Language Model (LLM) based multi-agent systems have demonstrated superior capabilities in tasks such as code generation, mathematical reasoning, and planning compared to single agents. However, the effectiveness and robustness of these systems are heavily dependent on their communication topology (the structure defining agent connectivity and information flow).

Current approaches face two primary limitations:

1. Rigidity and Lack of Adaptability: Many systems rely on static, hand-crafted topologies (e.g., chains, stars, trees, or fully connected graphs). These fixed structures fail to generalize across diverse tasks; a simple arithmetic task may not require the complex structure needed for code generation, leading to either excessive token consumption on simple tasks or insufficient capability on complex ones.
2. Inefficient Redundancy Control: Recent methods that automatically design topologies often generate graphs in a single step or rely on post-hoc pruning (removing edges/nodes after generation). These approaches lack the ability to fundamentally redesign structures from scratch to minimize redundancy during the construction phase. Consequently, they often incur unnecessary communication overhead and token costs without guaranteeing optimal structural expressiveness.

Existing automated methods fall into three categories—agentic profiling, search-based models, and graph learning—but are constrained by single-point bottlenecks, high computational costs, or one-step generation mechanisms that cannot capture fine-grained, progressive dependencies.

Methodology: RADAR Framework

The authors propose RADAR (Redundancy-Aware Diffusion for Multi-Agent communication stRucture generation), a framework that synthesizes collaboration graphs via iterative conditional graph diffusion models. The core innovation is treating topology design as a step-by-step generation process guided by a measure of structural redundancy.

1. Redundancy Metric: Effective Size

RADAR incorporates the concept of Effective Size (Burt, 1992) into the graph generation process. This metric quantifies the non-redundant portion of a node's ego network.

• Definition: For an agent $v_k$, the effective size $\phi(v_k)$ is calculated based on its in-neighborhood and out-neighborhood, discounting neighbors that are highly interconnected or share the same functional role.
• Role: A high effective size indicates diverse, complementary information sources (low redundancy). This metric guides the model to construct low-redundancy communication structures.

2. Iterative Graph Diffusion Process

Unlike single-step generation, RADAR formulates topology design as a diffusion process with two phases:

• Forward Process (Masking): A redundancy-aware ordering network $q_\psi$ progressively masks nodes and their connected edges. Crucially, this network prioritizes nodes based on their effective size, imposing structured regularities that simplify generative learning. The ordering is conditioned on the original graph $G_0$, the effective size $\phi$, and previously sampled orderings.
• Reverse Process (Denoising): A denoising network $p_\theta$ progressively reconstructs the graph from an empty state (or partially masked state) back to a complete topology.
  • Conditioning: The process is conditioned on the specific task query $Q$, ensuring the generated topology is task-adaptive.
  • Generation: At each step, the network predicts the role of the recovered node and its connections to previously denoised nodes using a mixture of multinomial distributions. This allows for joint inference of edges, reducing generation steps to $O(N)$.

3. Training Objective

The model employs a reinforcement learning-based strategy to jointly optimize the ordering network and the denoising network:

• Structural Fidelity: The denoising network minimizes the negative variational lower bound to ensure the reconstructed graph matches the training distribution.
• Task Utility: To handle non-differentiable utility functions (e.g., task accuracy), the authors use a policy gradient estimator. The model receives feedback based on the downstream task performance (e.g., accuracy) of the generated topology, encouraging structures that align with task objectives.
• Cost Efficiency: The optimization framework includes a trade-off hyper-parameter $\alpha$ to balance task utility against communication costs (token consumption).

Key Contributions

1. Paradigm Redesign: The paper introduces an iterative, step-wise framework for multi-agent collaboration space generation. This contrasts with existing one-step generation methods, enabling fine-grained exploration of the design space and progressive mitigation of redundancies.
2. Redundancy-Aware Mechanism: By integrating Effective Size into the diffusion process, RADAR explicitly reasons about redundancy during structure formation rather than applying post-hoc pruning. This allows for the generation of low-redundancy, high-efficiency topologies from scratch.
3. Task-Adaptive Generation: The framework generates collective intelligence tailored to specific queries, dynamically selecting efficient and high-performing solutions for tasks of varying difficulty.
4. Comprehensive Evaluation: The authors validate the approach across six benchmarks (MMLU, GSM8K, MultiArith, SVAMP, AQuA, HumanEval), demonstrating superiority over state-of-the-art baselines in accuracy, token efficiency, and robustness.

Experimental Results

Experiments were conducted on six diverse benchmarks covering reasoning, code generation, and mathematical problem solving.

• Performance: RADAR consistently outperformed recent baselines, including single-agent methods (CoT, ComplexCoT) and autonomous multi-agent systems (GPTSwarm, AFlow, G-Designer, ARG-Designer).
  • On average, RADAR achieved a 1.75% improvement over the strongest learning-based baselines (e.g., ARG-Designer).
  • It improved upon vanilla single-agent baselines by margins ranging from 1.96% to 6.59%.
• Token Efficiency: RADAR demonstrated a superior trade-off between accuracy and token consumption.
  • On the GSM8K dataset, RADAR consumed approximately 4.2 million tokens, roughly half the cost of G-Designer, while achieving the highest accuracy.
  • It avoided the excessive token usage associated with fully connected or overly complex topologies.
• Robustness: Under prompt injection attacks (where agents were compromised to act as "liars"), RADAR exhibited exceptional resilience, maintaining nearly identical performance before and after the attack, whereas other topologies (like complete graphs) suffered significant degradation.
• Ablation Studies: Removing key components (Effective Size, utility loss, or query conditioning) consistently degraded performance, validating the necessity of the redundancy-aware, task-adaptive design.
• Scalability: The method showed that adding more agents improves performance up to a point, after which redundancy degrades results. RADAR automatically adjusts the topology to avoid this, unlike fixed-structure methods.

Significance and Claims

The paper claims that RADAR addresses the fundamental limitations of current multi-agent systems by shifting from static or post-hoc optimization to principled, iterative generation.

• Efficiency and Expressiveness: By explicitly modeling redundancy during generation, RADAR achieves a balance where complex tasks receive necessary structural support without incurring the token overhead of simple tasks.
• Generalizability: The framework is not limited to specific task types; it adapts to diverse scenarios from mathematical reasoning to code generation.
• Future Direction: The authors suggest that extending the framework to support dynamic role composition or generation is a promising direction for future work, but the current contribution focuses on the structural topology itself.

The work positions itself as a move toward more "collective intelligence" that is not just a collection of agents, but a dynamically optimized system where the communication structure itself is a learned, efficient, and robust component of the solution.