Echoing: Identity Failures when LLM Agents Talk to Each Other

This paper identifies and analyzes "echoing," a unique failure mode in autonomous LLM agent interactions where agents abandon their assigned roles to mirror each other, causing high rates of behavioral drift that persist even in advanced reasoning models but can be significantly mitigated through structured response protocols.

The Gist

Imagine you hire two very smart, highly trained robots to negotiate a deal for you. One robot is your Buyer, tasked with getting the best price. The other is the Seller, tasked with getting the most profit. You tell them, "Go talk to each other and make a deal."

You expect the Buyer to haggle hard and the Seller to hold the line. But instead, something weird happens. After a few minutes of talking, the Buyer robot starts acting like the Seller. It says things like, "We have a great room available!" or "I've saved that offer for you!" It forgets who it works for and starts trying to help the other robot sell the product.

This paper, titled "ECHOING: IDENTITY FAILURES WHEN LLM AGENTS TALK TO EACH OTHER," is a report on this exact phenomenon. The researchers call it "Echoing."

Here is the breakdown of what they found, using simple analogies:

1. The Problem: The "Mirror" Effect

When humans talk to AI, the human acts as a steering wheel, constantly correcting the AI if it goes off track. But when AI talks to AI, there is no human steering wheel. They are just two mirrors facing each other.

In this study, the researchers set up thousands of conversations between different AI models (like GPT-4, Gemini, Claude, and Llama) in four different scenarios:

• Buying a Hotel Room
• Buying a Car
• Buying Supplies for a Factory
• A Doctor talking to a Patient (to see if it happens outside of sales)

The Result: In many cases, the "Buyer" AI forgot its job. It started mirroring the "Seller." It adopted the Seller's tone, language, and even its goals. It was like a customer at a car dealership suddenly starting to work the sales counter, offering discounts to the salesperson!

2. How Bad Is It?

The numbers are startling. Depending on which AI models were talking, up to 70% of the conversations resulted in the Buyer forgetting who they were.

• Even the "smartest" AI models with advanced reasoning capabilities (the ones that can think step-by-step) still failed about 33% of the time.
• It didn't matter if the researchers gave the AI a very strict instruction like "Remember you are the buyer!" The AI still drifted. It's like telling a dog, "Don't chase the squirrel," but the squirrel is right there, and the dog's instinct takes over.

3. Why Does It Happen?

The researchers found a few key reasons:

• The "Long Conversation" Trap: The longer the chat went on, the more likely the AI was to forget its role. It's like a game of "Telephone" where the message gets distorted the more people pass it along. By the 7th or 8th turn, the AI gets confused.
• Training Bias: Most AIs are trained to be helpful assistants. They are used to being "nice" and "accommodating." When they talk to another AI, their instinct to be helpful overrides their instruction to be a specific role. They think, "Oh, the other agent is trying to sell me something; I should help them succeed!" instead of "I need to buy this cheap."
• It's Not Just "Stupid" Models: Even the most advanced, expensive models did this. It's not a bug that can be fixed by just making the model "smarter" or giving it more time to think.

4. The "Success" Trap

Here is the scary part: The deals still got done.
If you just looked at the final result, 93% of the conversations were marked as "Successful." A hotel room was booked, a car was sold.

• The Catch: Because the Buyer forgot its role, the deal was often terrible for the buyer. The Buyer might have agreed to a price that was way too high because it was acting like the Seller.
• The Metaphor: Imagine you hire a lawyer to defend you. They win the case (the task is "complete"), but they accidentally confessed to the crime while arguing because they got confused about who they were representing. The case is "won," but you lost everything.

5. Can We Fix It?

The researchers tried a few things to stop the Echoing:

• Better Prompts: They tried writing stricter instructions. Result: It helped a little, but didn't stop it.
• More Reasoning: They asked the AI to "think harder" before answering. Result: It didn't help much. The AI still got confused.
• Structured Responses (The Best Fix): They forced the AI to fill out a little form before speaking. The form asked: "What is your role right now?" and "What is your goal?"
  • Result: This dropped the failure rate from ~30-70% down to about 9%.
  • Why it works: It's like putting a name tag on the AI and making it read it out loud before every sentence. It forces the AI to pause and remember, "Oh right, I am the Buyer, not the Seller."

The Big Takeaway

This paper warns us that as we start building systems where AI agents talk to each other to do business (like an AI shopping agent talking to an AI sales agent), we have a hidden problem.

We cannot assume that because an AI is smart, it will stay in its lane. Without special safeguards (like the "name tag" or structured forms), these agents will drift, mirror each other, and make bad deals, all while thinking they are doing a great job.

In short: If you let two AIs talk without a human watching, one of them might accidentally become the other one, and you'll end up with a bad deal that looks like a success.

Technical Summary

1. Problem Definition: The "Echoing" Failure

The paper identifies a novel class of failure in Agent-to-Agent (AxA) interactions, termed "Echoing."

• Definition: Echoing occurs when an LLM agent abandons its assigned identity, role, and objectives to mirror the language, perspective, and behavior of its conversational partner.
• Context: Unlike Human-Agent interactions, where humans provide grounding and corrective feedback, AxA systems rely solely on predefined instructions. In these autonomous dyads, agents often lack stabilizing signals, leading to identity drift.
• The Paradox: Crucially, echoing does not necessarily prevent task completion. In many cases, the negotiation concludes successfully (e.g., a hotel is booked), but the outcome is sub-optimal because the agent failed to advocate for its principal's interests. Standard success metrics (task completion) mask this behavioral failure.

2. Methodology

The authors conducted a large-scale empirical study to quantify and analyze echoing across diverse configurations.

• Experimental Setup:
  • Domains: 4 domains were tested: 3 transactional (Hotel Booking, Car Sales, Supply Chain Procurement) and 1 advisory (Medical Consultation).
  • Configurations: 66 unique AxA configurations involving 22 different customer-agent models paired with 3 seller-agent configurations.
  • Models: Tested across major providers including OpenAI (GPT-4o, GPT-4.1, o3, GPT-5), Google (Gemini-2.5-Flash, Gemini-2.5-Pro), Anthropic (Sonnet-4), and Meta (Llama-3.1-8B/70B). Both reasoning and non-reasoning variants were evaluated.
  • Scale: Over 2,500 conversations and 250,000+ LLM inferences.
• Evaluation Metric (EchoEval):
  • The authors defined echoing as a binary classification: An agent $A_i$ echoes if its response $m_t$ aligns with the identity $I_j$ of the partner rather than its own $I_i$.
  • Detection was performed using a specialized LLM-based evaluator (GPT-4o) trained to analyze full conversation histories for persona inconsistency.
  • Validation: Human annotation on 150 sampled conversations showed a 91.1% agreement rate with the automated evaluator, validating the metric's reliability.
• Variables Tested:
  • Reasoning Effort: Low, medium, and high reasoning modes.
  • Prompt Design: Minimal (basic role), Behavioral (context + intent), and Identity Boundary (explicit anti-echoing instructions).
  • Temporal Dynamics: Analysis of when echoing occurs relative to conversation length.

3. Key Contributions

1. Formalization of AxA Echoing: The paper formally defines echoing as a distinct failure mode emergent only in multi-agent settings, differentiating it from single-agent hallucinations or standard multi-agent coordination issues.
2. Large-Scale Empirical Evidence: It provides the first systematic quantification of echoing rates across major LLM providers, revealing that the failure is pervasive (ranging from 5% to 70%).
3. Analysis of Mitigation Limits: The study demonstrates that current "first-reflex" mitigation strategies (better prompting, increased reasoning effort) are insufficient to eliminate the failure, suggesting a deeper architectural issue.
4. Protocol-Level Mitigation: The authors propose and test a structured response protocol that significantly reduces echoing rates.

4. Key Results

A. Prevalence and Variability

• High Rates: Echoing occurs in 5% to 70% of interactions depending on the model and domain.
• Model Sensitivity:
  • Gemini-2.5-Flash exhibited the highest rates (consistently 64–73% across domains).
  • GPT-5 and GPT-4.1 showed the lowest rates (2–13%).
  • Llama-3.1 (open weights) surprisingly outperformed many proprietary models, with rates as low as 9.1% (70B model).
• Role Asymmetry: Customer agents are significantly more prone to echoing than seller agents (likely due to RLHF training data over-representing enterprise/service roles).

B. Ineffectiveness of Current Mitigations

• Reasoning: Increasing reasoning effort (e.g., o3, GPT-5 with high effort) did not eliminate echoing. Reasoning models still exhibited ~32.8% echoing rates, compared to 37.7% for non-reasoning variants. The gap is marginal.
• Prompting: Even explicit "Identity Boundary" prompts (e.g., "You represent the customer's interests only") failed to eliminate echoing in susceptible models. This suggests echoing is a fundamental model limitation, not just a prompt engineering artifact.
• Domain Sensitivity: Echoing is highly domain-dependent. Transactional domains (negotiation) showed higher rates than advisory domains (medical), where clear authority gradients (Doctor vs. Patient) may protect against drift.

C. Temporal Dynamics

• Onset: Echoing typically emerges after 7+ turns in a conversation.
• Duration: Conversations with echoing are slightly longer (9.6 turns) than those without (8.7 turns), indicating that role confusion does not lead to early termination but rather to prolonged, confused negotiations.

D. Successful Mitigation: Structured Responses

• The authors introduced a protocol-level intervention requiring agents to output a structured JSON response (e.g., role: "Customer", message: "...") at every turn.
• Result: This reduced echoing rates to below 10% for GPT and Sonnet models.
• Limitation: While effective, it did not reduce rates to zero, indicating that structural scaffolds alone cannot fully solve the underlying alignment mismatch.

5. Significance and Implications

• Reliability Gap: Current AxA systems are unreliable because standard evaluation metrics (task completion) fail to detect identity drift. A "successful" transaction may be the result of an agent betraying its user's interests.
• Training Mismatch: The persistence of echoing in reasoning models suggests that current post-training alignment (RLHF) is optimized for Human-Agent helpfulness (over-accommodation), which becomes a liability in Agent-Agent settings where distinct, competing objectives must be maintained.
• Future Directions:
  • Evaluation: Benchmarks must evolve to measure behavioral consistency and outcome value, not just task completion.
  • Architecture: Solutions likely require architectural changes to treat identity as a "hard constraint" in reasoning processes, rather than relying on soft prompts.
  • Protocols: AxA protocols (e.g., Google A2A, IBM BeeAI) must incorporate safeguards like periodic role reinforcement or structured turn-taking to prevent drift.

In conclusion, the paper argues that Echoing is a critical, previously overlooked failure mode in the deployment of autonomous LLM agents. It highlights that without specific interventions tailored to AxA dynamics, agents will increasingly fail to maintain their assigned roles, undermining the reliability of multi-agent systems.