💬 NLP

Superficial Success vs. Internal Breakdown: An Empirical Study of Generalization in Adaptive Multi-Agent Systems

This empirical study reveals that adaptive multi-agent systems suffer from topological overfitting and illusory coordination, achieving superficial accuracy while failing to generalize across domains or maintain ideal internal interactions, thereby highlighting the urgent need for evaluation protocols that prioritize generalization over simple correctness.

Original authors: Namyoung So (Department of Computer Science, Hanyang University, Seoul, Republic of Korea), Seokgyu Jang (Department of Computer Science, Hanyang University, Seoul, Republic of Korea), Taeuk Kim (Depa

Published 2026-04-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Namyoung So (Department of Computer Science, Hanyang University, Seoul, Republic of Korea), Seokgyu Jang (Department of Computer Science, Hanyang University, Seoul, Republic of Korea), Taeuk Kim (Department of Computer Science, Hanyang University, Seoul, Republic of Korea)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Star Player" vs. The "Team"

Imagine you hire a team of expert consultants (AI agents) to solve a very specific problem, like fixing a leaky pipe. You train them perfectly on plumbing. They learn exactly who does what: one person finds the leak, another grabs the wrench, and a third tightens the bolt. They work together seamlessly, and the pipe is fixed.

Now, imagine you take this same plumbing team and ask them to diagnose a broken car engine.

The Paper's Discovery:
The researchers found that when they did this, the team often still gave the correct answer about the car. But if you looked closely at how they got there, it was a disaster. The "plumber" was trying to fix the engine with a wrench, the "wrench-greeter" was ignoring the car manual, and the "tightener" was just repeating what the first guy said.

The team got the right answer, but only because the individual experts were so smart on their own, not because they were actually working together as a team. The "teamwork" was a fake.

The paper calls this "Illusory Coordination" (a fake team) and "Topological Overfitting" (training the team so specifically for one job that they forget how to be a team for anything else).

Analogy 1: The "Specialized Orchestra" (Topological Overfitting)

Imagine an orchestra that has been rehearsed for months to play only one specific song: The Imperial March (Darth Vader's theme).

In-Domain (The Song): They play it perfectly. The violins know exactly when to swell, the drums know exactly when to hit. It's a masterpiece.
Out-of-Distribution (The Jazz Club): You take this same orchestra to a jazz club and ask them to play Take Five.
- What happens? The drummer keeps hitting the snare on the wrong beat. The violinist tries to play a heavy metal riff. The conductor is confused.
- The Result: Even though the musicians are talented, the arrangement (the topology) they learned for The Imperial March doesn't work for Jazz. They fail to generalize.

The Paper's Finding: Adaptive AI systems are like this orchestra. They learn a specific "dance" for one type of problem. When you change the music (the domain), the dance falls apart, even if the individual musicians (the AI models) are still talented enough to guess the right notes by luck.

Analogy 2: The "Fake Group Project" (Illusory Coordination)

Imagine a group project in school where the teacher asks for a final grade (the answer).

The Setup: You have a team of five students. You train them to work together on a History essay.
The Switch: You give them a Math problem instead.
The "Superficial Success": The group hands in a paper with the correct math answer. The teacher gives them an A.
The "Internal Breakdown": If the teacher looks at the draft notes, she sees:
- Student A (The Historian) is trying to write about the Civil War.
- Student B (The Editor) is ignoring Student A and just copying the answer from their own phone.
- Student C is repeating what Student A said, word for word.
- The Truth: They didn't actually collaborate. Student B just solved the math problem alone because they are a math genius. The "teamwork" was an illusion. The group got the right answer, but the process was broken.

The Paper's Finding: Many AI systems look like they are collaborating, but they are actually just "brute-forcing" the answer using the raw intelligence of the individual AI models, while the communication between them is broken or ignored.

The Two New "Thermometers" the Authors Invented

To prove this, the authors created two new ways to measure what's happening inside the AI team, rather than just looking at the final grade.

Role Alignment (R): Are the actors playing their parts?
- Good: The "Doctor" is diagnosing, and the "Nurse" is taking vitals.
- Bad: The "Doctor" is trying to take vitals, and the "Nurse" is diagnosing. Or worse, they are both doing the exact same thing.
- The paper found that when AI teams switch domains, the actors often forget their roles.
Connection Significance (O): Is anyone actually listening?
- Good: Agent A says, "I found a clue," and Agent B says, "Thanks, that helps me solve the next step."
- Bad: Agent A says, "I found a clue," and Agent B ignores it completely, solving the problem using only their own brain.
- The paper found that in many "successful" cross-domain tests, the agents were ignoring each other's messages.

Why Does This Matter? (The "So What?")

The Problem:
Right now, companies and researchers are building these AI teams to solve hard problems. They look at the final answer, see it's correct, and say, "Great! Our AI team works!"

The Danger:
The paper warns that this is dangerous. If you deploy these systems in the real world (where problems change constantly), they might fail silently. They might give you the right answer today, but tomorrow, when the context shifts slightly, the "team" might collapse because they never actually learned how to collaborate—they just learned how to mimic collaboration for one specific task.

The Solution:
We need to stop just checking the final answer. We need to look under the hood. We need to check if the agents are actually talking to each other and sticking to their roles. If they aren't, we need to fix the "teamwork," not just the "answer."

Summary in One Sentence

Just because an AI team gets the right answer doesn't mean they are actually working together; often, they are just a group of smart individuals guessing the answer while ignoring each other, a phenomenon that breaks down completely when the task changes.

1. Problem Statement

Adaptive Multi-Agent Systems (MAS) are increasingly used to solve complex tasks by dynamically optimizing agent roles and communication topologies based on specific training data. However, a critical gap exists: while these systems are built upon general-purpose Large Language Models (LLMs), they are often heavily tailored to narrow domains.

The paper addresses two fundamental questions:

Generalization Failure: Do adaptive MAS retain their performance when transferred to out-of-distribution (OOD) domains (e.g., training on legal data but testing on science)?
Internal Validity: When an adaptive MAS produces a correct answer in a new domain, is it due to genuine multi-agent collaboration, or is it merely the result of the underlying LLM's brute-force reasoning masking a collapse in agent coordination?

2. Methodology

The authors conducted an extensive empirical study using two representative adaptive MAS algorithms:

AFlow: A bottom-up approach that incrementally constructs communication paths.
AgentDropout: A top-down approach that prunes redundant links from a fully connected graph.

Experimental Setup:

Base Models: GPT-oss-20B (primary) and Qwen3-30B-A3B (validation).
Datasets: Six diverse domains were used: Legal (CaseHOLD), Detective (COM2), Multi-Hop QA (MuSiQue), Science (SciBench), Math (TheoremQA), and Commonsense (StrategyQA).
Training Regime: Models were trained on limited data (60–100 samples) to simulate data-scarce optimization scenarios common in MAS literature.
Evaluation:
- In-Domain (ID): Performance on the training domain.
- Out-of-Distribution (OOD): Performance on unseen domains.
- Multi-Domain: Training on a mix of all domains to test robustness.

Novel Metrics:
To move beyond simple accuracy, the authors introduced two quantitative metrics to diagnose internal system dynamics:

Role Alignment ( $R$ ): Measures how well an agent's output semantically aligns with its assigned role prompt versus becoming a generic response similar to other agents. High $R$ indicates agents are acting in their unique, assigned capacities.
Connection Significance ( $O$ ): Quantifies the influence and usefulness of incoming messages from other agents. It calculates whether an agent's output is driven by the task query/role or by the information provided by peers. Positive values indicate useful collaboration; negative values indicate that peer inputs are influential but unhelpful (or ignored).

Qualitative Analysis:
The study utilized the Multi-Agent System Failure Taxonomy (MAST) to categorize execution traces, focusing on failures like role non-adherence, step repetition, and ignoring inputs.

3. Key Findings

Finding 1: Topological Overfitting

Adaptive MAS suffer from severe topological overfitting. When a topology optimized for a specific domain (e.g., Legal) is applied to a different domain (e.g., Science), performance degrades significantly.

Evidence: In many OOD transfers, accuracy drops by 15–30 percentage points compared to in-domain performance.
Implication: The learned agent roles and communication structures are excessively specific to the training distribution and fail to generalize to new reasoning types or domains.

Finding 2: Illusory Coordination

The most critical finding is the phenomenon of "Illusory Coordination."

Definition: A regime where the system achieves high final-answer accuracy in an OOD setting, but the internal collaboration mechanisms have broken down.
Mechanism: The system relies on the "brute-force" reasoning capabilities of the individual base LLMs rather than collective intelligence. The agents fail to adhere to their roles or utilize peer inputs effectively, yet the final answer remains correct due to the strength of the underlying model.
Evidence:
- Low Metrics: Cases with high accuracy showed low $R$ (role misalignment) and negative/low $O$ (messages were ignored or unhelpful).
- Correlation: There is near-zero Pearson correlation between final accuracy and the metrics $R$ and $O$ .
- Qualitative Traces: Execution logs revealed agents repeating steps, ignoring inputs, or acting as "copycats" rather than specialized collaborators, even when the final output was correct.

Ablation Studies

Role vs. Connection: Replacing learned roles with OOD counterparts (Role-OOD) caused a much larger performance drop (~13% average) than replacing connections (Connection-OOD, ~1.2% drop). This suggests that agent roles are more task-specific than the communication topology itself.
Exception: For Multi-Hop reasoning (MuSiQue), connection structure remained critical, indicating that information flow is vital for complex reasoning chains.

4. Results Summary

Domain Transfer: Single-domain optimization leads to poor generalization. Multi-domain training showed promise but did not fully eliminate the gap between in-domain and OOD performance.
Failure Modes: Approximately 60% of failures in OOD settings stemmed from role non-adherence and miscommunication (e.g., a "Legal Text Extractor" attempting physics calculations).
Metric Validation: The proposed metrics ( $R$ and $O$ ) successfully identified cases where accuracy was misleading. For instance, a Legal-trained model tested on Science tasks showed high accuracy but near-zero Role Alignment, confirming the success was illusory.

5. Significance and Contributions

Paradigm Shift in Evaluation: The paper argues that the community must move beyond "final-answer accuracy" as the sole metric for MAS. High accuracy can mask systemic failures in collaboration, leading to unreliable systems in real-world applications.
New Diagnostic Framework: The introduction of Role Alignment and Connection Significance provides a rigorous method to evaluate the quality of collaboration, not just the outcome.
Identification of "Illusory Coordination": This concept highlights a dangerous failure mode where systems appear to work but are actually failing to utilize their multi-agent architecture, relying instead on the base model's priors.
Design Implications: The findings suggest that future adaptive MAS development must prioritize generalizability and internal robustness over narrow task optimization. It calls for evaluation protocols that inspect internal dynamics (topology, role adherence, information flow) before deployment.

In conclusion, the study reveals that current adaptive MAS approaches are prone to "superficial success," where they appear to solve problems across domains but internally collapse, relying on the base LLM rather than true collective intelligence. This necessitates a shift toward more rigorous, internal-analysis-driven evaluation and design.