💻 computer science

Symphony-Coord: Adaptive Routing for Multi-Agent LLM Systems

Symphony-Coord is a decentralized coordination framework for multi-agent LLM systems that addresses the limitations of static routing by transforming agent selection into an adaptive online multi-armed bandit problem, utilizing a two-stage beacon protocol and LinUCB selector to dynamically optimize task routing and fault recovery based on real-time feedback.

Original authors: Zhaoyang Guan, Huixi Cao, Ming Zhong, Yin Wang, Guanyu Liu, Eric Yang, Lynn Ai, Yongxin Ni, Bill Shi

Published 2026-06-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhaoyang Guan, Huixi Cao, Ming Zhong, Yin Wang, Guanyu Liu, Eric Yang, Lynn Ai, Yongxin Ni, Bill Shi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive team of experts, but they are all different. Some are great at math, others at coding, some are fast but make mistakes, and others are slow but very accurate. You also have a flood of complex jobs coming in every second.

The problem with most current systems is that they act like a rigid manager who assigns jobs based on a fixed rulebook: "Math jobs go to Person A, coding jobs go to Person B." This works fine until Person A gets tired, the math job changes slightly, or Person B is actually better at this specific type of math than Person A. The system gets stuck, inefficient, and breaks easily.

Symphony-Coord is a new way to run this team. Instead of a rigid manager, it uses a smart, adaptive "traffic controller" that learns in real-time who is best for the job right now.

Here is how it works, using simple analogies:

1. The Problem: The Rigid Orchestra

In traditional systems, every musician (agent) is assigned a fixed instrument (role) before the concert starts. If the conductor needs a violin solo, they call the violinist. But what if the violinist is sick? Or what if the piece of music actually needs a cello for that specific note? The system doesn't know how to adapt, leading to bad performances or long delays while they figure out who to call next.

2. The Solution: The Smart Traffic Controller

Symphony-Coord treats the team like a dynamic pool of talent. When a new task arrives, it doesn't just look at a name tag; it asks, "Who is available, who is fast, and who has done similar things successfully before?"

It solves this using a two-step process, which the authors call a "Dynamic Beacon Protocol."

Step 1: The "Lightning Round" (Top-L Filtering)

Imagine you have 100 experts in a room. Calling all 100 to ask, "Can you do this?" would take forever and waste time.

What Symphony-Coord does: It quickly shines a "beacon" (a quick check) on everyone. It looks at their resume (capabilities) and their current mood (are they busy? are they crashing?).
The Result: It instantly narrows the 100 people down to the top 3 or 5 best candidates. This saves a huge amount of time and money (computing power).

Step 2: The "Gambler's Choice" (LinUCB Selection)

Now, you have a shortlist of 3 great candidates. How do you pick the one?

The Old Way: Pick the one who has won the most in the past. (This is risky because they might be on a slump today).
The Symphony-Coord Way: It uses a strategy called LinUCB, which is like a smart gambler balancing two things:
1. Exploitation: "This person has a great track record; let's pick them."
2. Exploration: "I haven't tried this person in a while; maybe they are actually the best for this specific tricky question. Let's give them a shot to learn."
The Learning: After the job is done, the system gets feedback. Did the answer work? Was it fast? If yes, it gives that agent a "high five" (increases their score). If no, it learns to be more careful next time.

3. Why It's Better: The "Living" System

The paper claims this system is superior because it doesn't rely on static labels.

Adaptability: If an agent starts making mistakes (like a tired worker), the system notices the feedback and stops sending them hard jobs immediately.
Resilience: If a top expert suddenly goes offline, the system doesn't crash. It quickly learns to route the work to the next best available person.
Efficiency: By filtering out the "maybe" candidates early, it saves resources, allowing the system to handle more tasks without getting bogged down.

4. The Proof: The "Medical Quiz" Test

The authors tested this on difficult tasks like medical questions and math problems.

They compared Symphony-Coord against single experts and other multi-agent teams.
The Result: Symphony-Coord consistently got higher scores. It was better at figuring out that a specific "Math Agent" was actually the best for a specific "Medical Logic" question, even if that agent wasn't officially labeled as a "Medical Expert."
They also showed that as the team grew larger (from 5 to 100 agents), Symphony-Coord stayed fast and accurate, while other systems got slow and confused.

Summary

Think of Symphony-Coord as a conductor who doesn't just read a sheet music script. Instead, they listen to the orchestra in real-time, quickly spot the best players for the current note, and constantly adjust the lineup based on who is playing perfectly and who is out of tune. It turns a rigid, breakable system into a flexible, self-healing team that gets smarter with every task it completes.

Technical Summary: Symphony-Coord

Problem Statement

Multi-agent large language model (LLM) systems aim to solve complex, multi-step tasks by decomposing work and coordinating specialized behaviors. However, existing coordination mechanisms typically rely on statically assigned roles and centralized controllers. As agent pools grow and task distributions evolve, these design choices introduce significant limitations:

Inefficient Routing: Static role labels are often too general to distinguish subtle differences in specific model capabilities, leading to suboptimal agent selection.
Poor Adaptability: Fixed task-to-role mappings, established during the design phase, struggle to switch to more suitable agents when task distributions shift.
Fragile Fault Recovery: When a highly capable agent degrades or becomes unavailable, static allocation lacks a rapid replacement mechanism, causing continuous performance decline.

While some frameworks attempt to soften role rigidity via prompt-based "role playing" (e.g., CAMEL), these approaches often lack a principled decision mechanism to learn from experience which agent is most suitable for a subtask under current context and system states.

Methodology: Symphony-Coord

The authors propose Symphony-Coord, a task-local coordination framework with decentralized execution. It reframes agent selection not as a static matching problem but as an online contextual bandit problem. The system treats each agent as an "arm" and selects executors based on observable context, updating routing policies via feedback.

The framework employs a two-stage dynamic Beacon routing protocol:

Stage 1: Lightweight Candidate Screening

To manage communication and computation overhead, the system does not query the entire agent pool for every subtask. Instead, it computes a lightweight composite score ( $s_{j,t}$ ) for each available agent $j$ , incorporating:

Match Score: Cosine similarity (or lexical fallback) between the agent's capability text and the subtask requirements.
Prior Success: A smoothed estimate of historical success.
Reliability: Short-horizon operational stability (e.g., recent latency, error status, availability).

The system selects the Top-L agents to form a candidate set $C_t$ , significantly pruning the decision space.

Stage 2: Adaptive LinUCB Selection

Within the shortlisted candidate set $C_t$ , Symphony-Coord applies a LinUCB-style online selector. For each candidate, a context vector $x_{j,t}$ is constructed, fusing:

Task requirement features.
Agent static capability representation.
Dynamic runtime signals (load, latency, reputation, availability).

The selector maximizes an Upper Confidence Bound (UCB) score:
$UCB_{j,t} = x_{j,t}^\top \hat{\theta}_{t-1} + \beta_{t-1} \sqrt{x_{j,t}^\top A_{t-1}^{-1} x_{j,t}}$
This balances exploitation (selecting agents with high estimated rewards) and exploration (testing uncertain candidates to reduce uncertainty).

Feedback and Update Mechanism

The system utilizes a delayed post-execution feedback loop. Rewards are shaped based on:

Base validity (non-error completion).
A "winner bonus" if an execution agrees with the voted final output (agreement-based proxy).
Optional correctness terms (when gold labels are available).
Efficiency penalties (latency).

Updates to the LinUCB statistics ( $A_t, b_t, \hat{\theta}_t$ ) occur after the task-level vote is resolved. This introduces a "deferred-update" effect, where parameters are updated with a slight delay, but the routing decision itself relies only on current context and state.

Key Contributions

Problem Formulation: Formalizes subtask–agent routing as an online contextual bandit problem, integrating task requirements and agent states into context features.
Two-Stage Dynamic Beacon Routing: Proposes a practical pipeline combining Top-L candidate filtering (to control overhead) with LinUCB-based adaptive selection (to handle non-stationarity).
Online Learning Closed Loop: Designs a feedback-to-update mechanism that continuously improves routing decisions end-to-end, aligning with sequential decision-making views.
Theoretical Scope: Proves a high-probability sublinear regret bound for the selector under standard candidate-conditional linear bandit assumptions. The analysis explicitly separates the effects of Top-L filtering and delayed post-vote feedback.
Empirical Validation: Validates the framework through simulation, real-world LLM benchmarks (GSM8K, BBH, MedicalQA), and extensive ablation studies.

Experimental Results

Performance Gains: Symphony-Coord consistently outperforms single-agent baselines (gains of 8.5–22.0% on GSM8K, 16.5–23.5% on BBH, and 27.0–33.0% on MedicalQA) and achieves the best average accuracy against multi-agent baselines (MetaGPT, AFLOW, GPTSwarm) across various backbones.
Heterogeneity: The system benefits significantly from heterogeneous agent pools. In MedicalQA experiments, skill heterogeneity in 5-agent teams yielded the highest accuracy, suggesting capability specialization is a key source of complementarity.
Scalability: The framework maintains stability as the agent pool grows from $N=5$ to $N=100$ . Unlike centralized baselines (AutoGen, MetaGPT) which suffer performance drops and increased latency with larger pools, Symphony-Coord keeps routing overhead below 100ms by restricting online selection to a fixed Top-L=3 shortlist.
Cost-Efficiency: Symphony-Coord achieves a favorable Pareto frontier, offering higher accuracy than single-agent baselines with moderate budgets and comparable/higher accuracy than multi-agent baselines with lower token consumption and latency.
Robustness: In semi-real recovery experiments involving agent degradation, Symphony-Coord successfully reallocated tasks to healthy agents, whereas static and frozen-UCB variants failed to recover.

Significance and Claims

The paper claims that Symphony-Coord demonstrates that scalable multi-agent LLM coordination does not need to rely on static role assignment. Instead, effective coordination can be realized through adaptive routing driven by runtime context and feedback.

Key distinctions from prior work include:

Emergent Specialization: Rather than manually assigning planner/solver/checker identities, routing specialization emerges gradually as the system adapts to task features and feedback.
Task-Local Coordination: The system operates with a decentralized execution model where workers are independent, but a lightweight, bounded routing plane manages candidate selection and feedback updates for each task instance.
Practical Integration: The novelty lies not in a new bandit algorithm, but in the integration of capability-aware filtering, contextual selection, and feedback-driven routing specialization specifically for heterogeneous LLM-agent pools.

The authors note that the framework is designed to be robust under task distribution shifts and agent failures, addressing open challenges in LLM-based multi-agent systems regarding communication overhead, coordination complexity, and robustness. Future work is suggested to extend the framework to larger open agent pools, richer tool-use workflows, and longer-horizon task streams.