💻 computer science

What Do Agents Communicate? Characterizing Information Exchange in Multi-Agent Systems

This paper identifies that the lack of reasoning and verification in multi-agent communication causes error propagation, leading the authors to propose a Category-Aware Recovery Augmentation technique that recovers up to 86.2% of failed cases by enforcing the exchange of critical information.

Original authors: Yong Jin Chun, Iftekhar Ahmed

Published 2026-05-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yong Jin Chun, Iftekhar Ahmed

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a team of three expert chefs trying to cook a perfect meal together. They are all using the same high-end cookbook (a Large Language Model), but they are working in a kitchen where they can only talk to each other through a walkie-talkie.

This paper, titled "What Do Agents Communicate?", investigates exactly what these chefs say to each other over the walkie-talkie and why sometimes the meal turns out burnt, even though the chefs are talented.

Here is the breakdown of their findings in simple terms:

The Problem: The "Telephone Game" of Errors

In the past, researchers tried to get these AI chefs to work better by just adding more chefs or changing the kitchen layout. But they noticed a problem: Error Propagation.

Think of it like a game of "Telephone." If Chef A makes a small mistake in the first step (like adding salt instead of sugar), and Chef B doesn't catch it, Chef B might build on that mistake. By the time Chef C finishes the dish, the whole meal is ruined. The paper found that these AI teams often fail not because they lack intelligence, but because they pass along bad information without checking it.

The Investigation: What are they actually saying?

The authors listened in on thousands of conversations between these AI teams. They realized that the chefs were mostly just shouting out the final answer or repeating the steps they took, but they were rarely doing two crucial things:

Explaining their logic (Why did you do that?).
Double-checking the work (Wait, does that math actually add up?).

They categorized the conversation into five types of "ingredients":

The Answer: The final result.
The Reasoning: The step-by-step thinking.
The Verification: Checking if the thinking is correct.
The Reference: Acknowledging what the other chef said.
The "No Change": Saying, "I'm sticking with my original answer."

The Experiment: The "Mute Button" Test

To figure out which ingredients were essential, the researchers played a game of "hide and seek." They took the recorded conversations and systematically deleted one type of ingredient at a time (like muting the "Verification" part of the walkie-talkie) to see what happened to the final meal.

What they found:

Removing the "Reasoning" was disastrous. When the chefs stopped explaining how they got their answer, the team's performance tanked. It's like trying to fix a car engine without looking at the manual; you might get lucky, but you'll likely break something.
Removing the "Verification" was also bad. When the chefs stopped checking each other's work, errors slipped through.
Surprisingly, removing the "Final Answer" sometimes helped. It turns out that shouting the answer too early can confuse the other chefs. Sometimes, it's better to focus on the process first.

The Solution: CARA (The "Safety Checklist")

Based on these findings, the authors built a new tool called CARA (Category-Aware Recovery Augmentation).

Think of CARA as a strict kitchen manager who stands between the chefs. Before a chef can speak into the walkie-talkie, the manager checks their message against a checklist:

"Did you explain your reasoning?"
"Did you verify your math?"
"Did you mention what the previous chef said?"

If the chef tries to send a message without these ingredients, the manager hits the "Stop" button and says, "Go back and rewrite this. You're missing the reasoning!" The chef has to try again (up to three times) until the message is complete.

The Result

When they used this "Safety Checklist" on the AI teams that had previously failed:

It fixed up to 86.2% of the failed cases.
It didn't require hiring smarter chefs or buying new equipment; it just required them to communicate better.

The Big Takeaway

The paper concludes that in a team of AI agents, quality of conversation matters more than the number of people talking. Just because agents are talking doesn't mean they are collaborating effectively. If they skip the "thinking" and "checking" parts of the conversation, the whole team will likely fail. By forcing them to include these critical pieces of information, we can turn a group of confused chefs into a well-oiled machine.

Technical Summary: What Do Agents Communicate? Characterizing Information Exchange in Multi-Agent Systems

Problem Statement

While Large Language Models (LLMs) have enabled collaborative Multi-Agent (MA) systems that improve performance through diverse reasoning and iterative refinement, these systems remain vulnerable to error propagation. In such systems, incorrect intermediate information can be reinforced and amplified through successive interaction rounds, often degrading downstream reasoning. Existing mitigation strategies are largely reactive, modifying agent behaviors or system structures only after failures have occurred, and largely overlook the specific role of inter-agent communication content during task execution. There is a critical gap in understanding what specific types of information agents exchange and how the presence or absence of these information categories shapes collective performance.

Methodology

The authors conduct a systematic, large-scale empirical analysis of inter-agent communication across five representative MA systems, six benchmarks, three task domains (Math, Question Answering, Code), and two LLM backbones (Qwen2.5-32B-Instruct and Qwen2.5-Coder-32B). The methodology proceeds in three stages:

1. Information Categorization (RQ1)

The authors collected 41,330 communication traces from 4,133 samples. Through a combination of manual annotation by two expert evaluators and LLM-based scaling (using GPT-4o as a judge), they identified and defined five distinct categories of information exchanged in agent responses:

C1 (Answer): The agent's final answer or decision.
C2 (Reasoning): A sequence of logical steps deriving the answer.
C3 (Verification): An assessment of the reasoning process or answer validity.
C4 (Reference): A direct reference to another agent's reasoning or answer to confirm, challenge, or build upon it.
C5 (Unchanged): An explicit statement that the answer remains unchanged from the prior turn.

2. Occlusion Analysis (RQ2)

To determine the causal impact of these categories, the authors applied a leave-one-out (LOO) occlusion framework. They systematically masked specific information categories (replacing text spans with [MASK] tokens) within agent responses before they were passed to subsequent agents. A length-matched control setting was used to ensure performance changes were due to semantic removal rather than prompt length variations. They measured the change in task accuracy ( $\Delta C_x$ ) for each category across different MA architectures (Sequential, Debate, Collective Refinement) and tasks.

3. Recovery Augmentation (RQ3)

Based on the occlusion findings, the authors proposed Category-Aware Recovery Augmentation (CARA). This is a targeted intervention strategy that enforces the presence of critical information categories during communication. CARA operates in two layers:

Prompt Augmentation: Extending system prompts to explicitly require specific categories (Reasoning, Verification, and Reference).
Response Verification: Automatically checking generated responses for these required categories. If critical information is missing, the agent is re-invoked (up to three times) to revise the response until the required structural components are present.

Key Results

Prevalence of Information Exchange

Reasoning (C2) and Answer (C1) are the most prevalent categories, appearing in over 90% of responses across most systems.
Verification (C3) and Reference (C4) are significantly less common in non-debate systems (often under 5%), whereas Debate systems exhibit higher prevalence of these categories (up to 50%).
Unchanged (C5) is the least prevalent category, suggesting agents rarely explicitly state when no correction is needed.

Impact of Information Removal (Occlusion)

Reasoning (C2): Removing reasoning consistently degrades performance (up to -18.63%), highlighting its critical role in collaborative solution derivation.
Verification (C3): The impact is task-dependent. Removing verification improves performance in Code tasks (+23.27%) but degrades it in Math tasks (-18.08%), suggesting verification acts as noise in some contexts but is vital in others.
Reference (C4) & Unchanged (C5): Removing these categories often leads to significant performance drops (e.g., -38.69% for Reference in Collective Refinement systems), indicating their importance in maintaining coherent inter-agent communication and decision consistency.
System Dependency: The utility of information categories is tightly coupled with system design. For instance, removing Reasoning hurts Role-based Sequential systems but can improve Uniform Sequential systems, suggesting role assignment guides the utility of reasoning.

Effectiveness of CARA

CARA successfully recovers failed cases by enforcing the presence of critical information.
The method recovers up to 86.2% of failed cases (specifically in Math tasks with Role-based Sequential systems).
Recovery rates are generally higher for True→False (TF) cases (where the baseline succeeded but occlusion failed) than False→False (FF) cases, suggesting that while structural enforcement helps, it cannot fully recover performance if the underlying candidate solutions are fundamentally incorrect.
Instruction-tuned models (Qwen2.5-Inst) showed higher recovery rates than code-specialized models, likely due to better adherence to the enforced prompt instructions.

Significance and Claims

The paper claims to be the first systematic categorization of information exchanged between agents in MA systems, moving beyond quantitative metrics (like token count or rounds) to analyze the content of communication.

The authors assert that:

Information Quality Matters: Effective MA collaboration depends not just on system topology or the number of agents, but on the quality and presence of specific information types (reasoning, verification, reference) in inter-agent exchanges.
Recovery via Structure: It is possible to recover failed multi-agent tasks without modifying the underlying model or architecture, simply by enforcing the presence of essential communication categories.
Limitations of Current Systems: Current MA systems often propagate answers and reasoning without sufficient verification or reference, leading to error propagation. The dominance of "Answer" and "Reasoning" at the expense of "Verification" and "Reference" indicates a need for more balanced communication protocols.

The work suggests that future improvements in MA systems should focus on communication-level interventions that ensure the exchange of critical information, rather than solely optimizing agent behaviors or system structures.