💬 NLP

Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution

This paper introduces a cooperative game framework for multi-agent LLM attribution that identifies bottleneck agents via efficient Leave-One-Out methods and enables cost-effective system optimization through strategic model replacement, as demonstrated by improved performance and ethical alignment in medical applications.

Original authors: Mingyu Lu, Yushan Huang, Chris Lin, Su-In Lee

Published 2026-05-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mingyu Lu, Yushan Huang, Chris Lin, Su-In Lee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a team of expert chefs working together in a busy kitchen to create a complex five-course meal. Sometimes the meal is perfect; other times, it's a disaster. The big question is: Who actually made the difference? Was it the head chef who organized the menu? The sous-chef who chopped the vegetables? Or maybe the new intern who accidentally added too much salt?

This paper, titled "Agents that Matter," tackles this exact problem for teams of AI agents (like the chefs). As these AI teams get bigger and more complex, it becomes hard to know which specific AI is doing the heavy lifting and which one is just getting in the way.

Here is a simple breakdown of what the researchers did and found, using everyday analogies.

1. The Problem: The "Black Box" Kitchen

When AI agents work together, they pass information back and forth. If the final result is bad, it's hard to tell if it was because:

One specific AI was bad at its job.
The way they were talking to each other was broken.
The "head chef" (orchestrator) gave bad instructions.

Previous methods tried to figure this out by simulating what would happen if you removed an AI, but they were either too expensive (like hiring a whole new kitchen staff to test a recipe) or too inaccurate (like asking the remaining chefs to guess what would happen if a colleague left).

2. The Solution: A New Way to "Fire" an AI

The authors created a unified framework to measure exactly how much each agent contributes. They treat the AI team like a cooperative game. To see who matters, they use three main tools:

The "Leave-One-Out" (LOO) Method: This is the simplest approach. You take the whole team, remove one person, and see how the meal turns out. If the meal gets much worse, that person was crucial. If the meal stays the same, they weren't doing much.
- The Paper's Finding: This simple method works just as well as incredibly complex math formulas (like Shapley values) but is 3 to 7 times cheaper and faster. It's like realizing you don't need a supercomputer to know who the star player is; you just need to watch the game with and without them.
The "Removal Protocol" (How you remove them): This is where it gets interesting. The paper shows that how you remove an agent changes the answer you get.
- Ablation (The "Amputation"): You literally delete the agent from the system. This reveals structural bottlenecks. For example, if you delete the "Head Chef" (Orchestrator) in a hierarchical kitchen, the whole system might collapse because no one is giving orders.
- Introspective Removal (The "Daydream"): You ask the remaining agents to imagine the missing agent isn't there and guess the outcome. The paper found this is unreliable. It's like asking the remaining chefs to guess how the soup would taste without the salt; they often get it wrong because they can't truly simulate the missing ingredient.
- Model Replacement (The "Substitute"): Instead of deleting an agent, you swap their brain for a cheaper, simpler one (like swapping a Michelin-star chef for a talented home cook). This tells you if that specific role needs a super-smart AI or if a cheaper one would do the job.

3. The Big Discoveries

A. The "Head Chef" isn't always the most important
In kitchens where everyone talks to everyone (decentralized), the "Head Chef" isn't the star. But in kitchens where one person gives orders to everyone else (centralized), the Head Chef is the most critical link. If you remove them, the whole system fails. The paper shows that how you organize the team changes who is important.

B. You can save money by swapping brains, not firing people
The researchers found a clever way to cut costs. By using the "Model Replacement" method, they identified agents who were using expensive, powerful AI models but didn't actually need them.

The Result: They swapped these expensive models for cheaper, open-source ones.
The Gain: In some tests, this improved the task success rate by 17% while cutting costs by 35%. It's like realizing your fancy espresso machine is overkill for a simple cup of tea, so you switch to a French press and get better coffee for less money.

C. Being "Smart" and Being "Ethical" are different jobs
The team tested this on a medical AI system (a team of AI doctors). They discovered a surprising split:

Some agents were great at getting the diagnosis right (accuracy) but terrible at following ethical rules.
Other agents were great at ethics but didn't help much with the diagnosis.
The Fix: By swapping out the "unethical" agents for better ones, they improved the system's ethical behavior without hurting the diagnostic accuracy. It's like realizing your best salesperson is rude, so you swap them for a polite one, and surprisingly, sales actually go up because customers like them more.

4. Why This Matters

This paper gives us a "rulebook" for managing AI teams. It tells us:

Don't overcomplicate it: Simple "remove one and test" methods work best.
Know your topology: Who is important depends on how your team is organized.
Swap, don't just delete: You can often save massive amounts of money by using cheaper AI models for specific roles without breaking the system.
Check your values: You can tune your AI team to be more ethical without sacrificing performance, but you have to know which "agent" is responsible for which behavior.

In short, the paper provides a practical, cost-effective way to audit, debug, and optimize teams of AI agents, ensuring they work together efficiently, ethically, and affordably.

Technical Summary: Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution

Problem Statement

As Large Language Models (LLMs) are increasingly deployed as collaborative Multi-Agent Systems (MAS) for complex tasks like software engineering and medical diagnosis, system-level failures and performance improvements become difficult to trace to specific agents. An agent's contribution is not solely a function of its underlying model capability but is heavily dependent on its assigned role, interactions with other agents, and the system's communication topology. Existing approaches for credit assignment lack a rigorous, unified framework. They often rely on computationally expensive re-evaluations of agent coalitions or divergent removal procedures (e.g., in-context simulation, independent judges) without systematically investigating how these design choices interact across different collaboration topologies. Consequently, practitioners lack clear guidance on selecting attribution methods that balance accuracy, computational cost, and interpretability.

Methodology

The authors formalize agent attribution as a protocol-conditioned cooperative game, extending the framework of Covert et al. (2021). This unified framework defines an attribution query via three parameters:

Removal Protocols ( $b$ ): The method by which an agent is "removed" from the system.
Coalition Distributions ( $w_i(S)$ ): The weighting of different agent subsets (coalitions) used to estimate contributions.
Target Metrics ( $G$ ): The scalar score (e.g., task success, token usage) derived from the execution trace.

Core Components

Formal Definition: A MAS is defined as a tuple $(A, \Gamma, \nu)$ , where $\nu$ is the utility function conditioned on a protocol $b$ . The attribution score $\phi^b_i$ is a weighted average of marginal contributions $\Delta^b_i(S)$ .
Removal Protocols:
- Agent Ablation: Removes the agent entirely (null protocol). This captures structural importance but can cause system failures if critical roles (e.g., orchestrators) are removed.
- Introspective Removal: Uses an LLM judge to simulate the absence of an agent via in-context prompting. This avoids system crashes but relies on the fidelity of the simulation.
- Model Replacement (Novel): A topology-preserving protocol where an agent's backbone model is substituted with a different model (e.g., a weaker open-source model) while keeping the role instantiated. This measures the marginal utility of the specific model capacity assigned to a role without breaking the system graph.
Coalition Distributions: The paper compares Leave-One-Out (LOO), Shapley values, Owen values (for group structures), and Myerson values (for graph constraints).

Experimental Setup

The framework was instantiated across three benchmarks (PlanCraft, WorkBench, BrowseComp-Plus) and four communication topologies (Independent, Centralized, Decentralized, Hybrid). The study evaluated attribution methods using closed-source models (Gemini-2.5-Flash, Claude 3.5 Haiku, GPT-5-mini) and open-source substitutes (Qwen3.5-122B-A10B) for the model replacement protocol.

Key Contributions

Unified Framework: The paper establishes a formal, protocol-conditioned game-theoretic framework for MAS attribution, clarifying that attribution scores are relative to the specific query (protocol, distribution, metric) rather than universal importance scores.
Efficiency of LOO: The authors demonstrate that Leave-One-Out (LOO) attribution, despite its simplicity, identifies bottleneck agents as effectively as combinatorial methods (Shapley, Owen, Myerson) but at a fraction of the computational cost (3.2× to 7.5× reduction in token usage).
Fidelity of Removal Protocols: The study reveals that removal protocols induce distinct attribution games. Agent ablation highlights structural bottlenecks (e.g., orchestrators in centralized systems), whereas introspective removal (using LLM judges) fails to faithfully approximate ablation, showing poor correlation with true ablation values, especially in complex topologies.
Model Replacement for Optimization: The introduction of model replacement allows for capability-allocation analysis. By substituting low-contribution agents with cheaper open-source models, the system can reduce costs without sacrificing performance.
Decoupling of Metrics: In medical MAS, the authors show that contributions to diagnostic accuracy and ethical alignment are often decoupled. Agents that improve accuracy may degrade ethical scores, and vice versa.

Key Results

Cost vs. Performance: In the PlanCraft benchmark, using LOO-based model replacement to substitute the bottom-ranked agents reduced billable token usage by up to 35% while improving task success rates by up to 17%. In contrast, agent ablation in hierarchical topologies often degraded performance.
Protocol Divergence: In centralized and hybrid topologies, agent ablation assigned high attribution to the orchestrator (scores ~0.3), whereas model replacement assigned low or negative attribution to the same role, indicating the orchestrator's specific model capacity was not the marginal bottleneck.
Introspective Limitations: Introspective removal showed limited agreement with agent ablation (e.g., $R^2$ as low as -9.94 in hybrid settings), confirming that LLM judges cannot reliably simulate true agent ablation.
Medical Case Study: In MDTeamGPT, replacing agents identified as having negative ethical attribution (Specialist, Triage, CoT Reviewer) with open-source models increased ethical alignment scores by 4% while maintaining diagnostic accuracy and reducing token usage by up to 78.8% for ethics tasks.
Code Generation Case Study: In MetaGPT, replacing the Product Manager agent (which had negative attribution) with a cheaper model improved pass@1 from 62% to 79% while cutting costs by 38%.

Significance and Claims

The paper claims to provide a principled approach for cost-effective MAS attribution and intervention. Its significance lies in:

Demystifying Attribution: It clarifies that there is no single "correct" attribution score; the choice of protocol and distribution dictates the answer.
Practical Optimization: It offers a concrete, low-cost method (LOO + Model Replacement) for optimizing MAS performance and cost, moving beyond theoretical credit assignment to actionable system engineering.
Safety and Ethics: It demonstrates the utility of attribution frameworks in auditing complex systems for non-performance metrics (like ethics), revealing that optimizing for one metric (accuracy) may inadvertently harm another (ethics), and that targeted intervention can resolve this trade-off.

The authors remain modest regarding limitations, noting that their analysis is currently constrained to moderate-sized systems (4–6 agents) due to the computational cost of combinatorial evaluation, and that extending the framework to dynamic agent recruitment or instance-level attribution remains future work.