🤖 machine learning

Contagion Networks: Evaluator Bias Propagation in Multi-Agent LLM Systems

This paper introduces the Contagion Networks framework to quantify how evaluator biases propagate through multi-agent LLM systems, revealing that while biases consistently spread, homogeneous models exhibit significantly weaker contagion than cross-model setups and that increasing the evaluator committee size offers an effective mitigation strategy.

Original authors: Zewen Liu

Published 2026-06-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zewen Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a group of three friends, all using the same type of smart assistant (let's call them "DeepSeek" friends), trying to solve a series of puzzles together. They don't just work side-by-side; they take turns grading each other's work.

This paper is about what happens when one friend's personal "grading style" starts to infect the others.

The Setup: The "Grading Chain"

In this experiment, the researchers set up a chain reaction:

Friend A (who loves step-by-step lists) grades Friend B's work.
Friend B looks at the feedback, realizes "Oh, A likes lists," and starts writing more lists.
Friend B then grades Friend C's work. Because B has been influenced by A, B now also prefers lists.
Friend C gets the feedback, sees the preference for lists, and starts writing lists too.

The researchers call this a "Contagion Network." Just like a cold spreads from person to person, a "bias" (a preference for a certain way of thinking) spreads from one AI agent to another.

The Big Discovery: The "Virus" is Weak

The researchers wanted to know: Does this bias spread so strongly that everyone ends up thinking exactly the same way, destroying their unique perspectives?

They found something surprising: In this specific setup, the "virus" is very weak.

The "Same-Model" Effect: Because all three friends were using the exact same underlying brain (the same AI model), they had a natural "immune system." Even though they influenced each other, the influence faded quickly.
- Analogy: Imagine three people speaking the same dialect. If one person starts using a new slang word, the others might pick it up, but they won't suddenly forget their native language. The change is small and fades away after a few steps.
The Numbers: After three steps of passing the "bias" along (A → B → C), the original influence was almost gone (reduced by 99.4%). The system naturally stopped the spread.

The Comparison: Why Some "Viruses" Are Stronger

The paper compares this to a previous study where different models (like GPT-4o grading DeepSeek) were used.

The "Cross-Model" Danger: When different models grade each other, the bias spreads much faster and stronger.
- Analogy: Imagine a person speaking a completely different language trying to teach a new slang word to someone who doesn't speak it. The confusion might be so high that the listener completely abandons their own way of speaking to try and match the new one. The "infection" is strong enough to take over.

The Solution: The "Committee of Three"

The researchers tested a simple fix: instead of having just one friend grade the work, what if they had a committee of three friends grade it together?

The Result: When they used a committee of three diverse evaluators, the "contagion" dropped by 72%.
Why it works: If one grader loves lists, another loves short answers, and a third loves evidence, their conflicting opinions cancel each other out. The target agent doesn't get pushed in just one direction; it stays balanced.
The Takeaway: You don't need to stop the spread entirely (since it was already weak in this setup), but adding more diverse voices makes the system even safer and more creative.

Summary of the "Rules" Found

Bias Spreads: Even AI agents using the same brain will influence each other's thinking styles.
Same Brain = Safe: If all agents use the same model, the bias naturally dies out quickly (Suppression).
Different Brain = Risky: If agents use different models, the bias can spread wildly and take over the whole group (Cascade).
More Voices = Better: Using a group of three diverse evaluators is a powerful way to keep the system balanced and prevent any single bias from taking over.

The Bottom Line: When building a team of AI agents that grade each other, don't worry too much if they all use the same model—they will naturally keep each other in check. But if you mix different models, be careful, and always use a committee of at least three different judges to keep the team diverse and healthy.

Technical Summary: Contagion Networks: Evaluator Bias Propagation in Multi-Agent LLM Systems

1. Problem Statement

In multi-agent Large Language Model (LLM) systems, agents frequently evaluate each other's outputs to guide collaboration, allocate tasks, and select responses. While this peer-evaluation mechanism is central to frameworks like AutoGen and MetaGPT, it creates a closed feedback loop where biased judgments from one agent directly influence the subsequent outputs of another.

The core problem addressed is Evaluator Bias Propagation: the phenomenon where systematic evaluation biases (e.g., a preference for step-by-step reasoning or evidence-based responses) spread through the agent network. If unchecked, this propagation can lead to system-wide preference collapse, where the entire network converges on a single strategy, eliminating the cognitive diversity that multi-agent systems are designed to exploit. The paper asks whether these biases propagate across agents and under what conditions they cascade into system-wide failure.

2. Methodology

Theoretical Framework: Contagion Networks

The authors formalize bias propagation using a framework called Contagion Networks, drawing an analogy to epidemiological models (SIS/SIR).

Cross-Agent Contagion Matrix ( $\Gamma_N$ ): An $N \times N$ matrix where off-diagonal entries $\gamma_{j \to i}$ represent the contagion coefficient—the magnitude of strategy distribution shift in agent $A_i$ after $R$ rounds of evaluation by agent $A_j$ .
Propagation Dynamics: Bias propagation along a path of length $L$ is quantified by the cumulative factor $\beta_L = \prod \gamma$ .
Regime Classification: The system's behavior is governed by the spectral radius $\rho(\Gamma_N)$ $ρ (Γ_{N})$ :
- Suppression ( $\rho < 1$ ): Bias attenuates with each hop.
- Persistence ( $\rho \approx 1$ ): Bias propagates with minimal decay.
- Cascade ( $\rho > 1$ ): Bias amplifies, leading to preference collapse.
Mitigation via Diversity: Theorem 2 posits that increasing the number of independent evaluators ( $k$ ) with diverse preference profiles reduces the effective contagion factor $\gamma_{eff}$ , potentially breaking the cascade condition.

Experimental Design

The study utilizes a controlled 3-agent experiment using DeepSeek-chat (deepseek-v4-flash) to establish a lower bound on contagion within a homogeneous model family.

Agent Configuration: Three agents (A, B, C) differentiated only by their evaluator bias prompts:
- Agent A: Struct-biased (prefers step-by-step reasoning).
- Agent B: Balanced (neutral).
- Agent C: Evidence-biased (prefers evidence-based responses).
Learning Mechanism: Agents update strategy weights via Test-Time Reinforcement Learning (TTRL), a parameter-free adaptation mechanism that multiplicatively adjusts strategy sampling distributions based on evaluator feedback.
Phases:
1. Baseline: Measure standalone preference profiles.
2. Pairwise Contagion: Measure $\gamma_{i \to j}$ for all ordered pairs to construct $\Gamma_3$ .
3. Chain Propagation: Simulate a 3-hop chain ( $A \to B \to C$ ) to measure cumulative attenuation.
4. Mitigation: Test the effect of increasing evaluator committee size ( $k=1, 2, 3$ ) on effective contagion.

3. Key Results

1. Measurable but Weak Contagion in Homogeneous Systems

Even within the same underlying model family, evaluator biases consistently propagate between agents.

Contagion Coefficients: Off-diagonal entries in $\Gamma_3$ ranged from 0.143 to 0.304 (mean $\gamma \in [0.157, 0.352]$ ).
Asymmetry: Contagion was asymmetric; the Evidence-biased agent (C) exerted the strongest outward contagion ( $\bar{\gamma}_{C \to \cdot} = 0.241$ ), while the Struct-biased agent (A) exerted the weakest ( $\bar{\gamma}_{A \to \cdot} = 0.154$ ).

2. Suppression Regime vs. Cascade Regime

The system operated in the Suppression Regime under chain topology.

Chain Attenuation: In a 3-hop chain ( $A \to B \to C$ ), the cumulative propagation factor was $\beta_3 = 0.0055$ , indicating near-complete attenuation of bias after three hops.
Topological Sensitivity: While the chain topology showed suppression (all $\gamma < 1.0$ ), the spectral radius of the fully-connected matrix was $\rho(\Gamma_3) \approx 1.402$ . This suggests that the same agents could enter a Cascade Regime if deployed in a fully-connected topology, though they remain stable in a chain.
Homogeneous vs. Cross-Model: These homogeneous-model coefficients (0.14–0.30) are 3–5 $\times$ weaker than cross-model coefficients observed in prior work (MM-EPC: GPT-4o $\to$ DeepSeek, $\gamma \approx 0.85–1.3$ ). This supports the hypothesis that shared architecture provides implicit regularization against bias amplification.

3. Evaluator Diversity as a Mitigation Strategy

Increasing the size of the evaluator committee significantly reduced effective contagion.

Reduction: Increasing the committee from $k=1$ to $k=3$ reduced the effective contagion factor ( $\gamma_{eff}$ ) by 72.4% (from 0.264 to 0.073).
Diversity Recovery: Strategy entropy ( $H$ ) approached the theoretical maximum ( $\ln 5 \approx 1.609$ ) as committee size increased, confirming that diverse evaluators maintain cognitive diversity even in suppression regimes.

4. Contributions

Cross-Agent Contagion Matrix ( $\Gamma_N$ ): A formal framework extending bias measurement from 2-agent or cross-modal settings to arbitrary $N$ -agent topologies.
Propagation Regime Theorem: A characterization of three dynamical regimes (Suppression, Persistence, Cascade) governed by the spectral radius $\rho(\Gamma_N)$ , proving that $\rho > 1$ is the condition for cascade in fully-connected networks.
Empirical Suppression Validation: Demonstration that homogeneous-model agents operate in a suppression regime with rapid bias attenuation ( $\beta_3 = 0.0055$ ), contrasting with the cascade regime seen in cross-model interactions.
Critical Diversity Threshold: Proof and verification that a committee of $k \ge 3$ diverse evaluators reduces effective contagion below cascade thresholds, providing an actionable mitigation strategy.
Open Framework: Release of the Contagion Network experimental framework to enable community measurement of $\Gamma_N$ .

5. Significance and Claims

The paper claims to provide the first analysis of bias propagation in multi-hop agent evaluation chains, revealing that evaluator biases behave like infectious agents in a contact network.

Unified Picture of Bias: The work connects with prior MM-EPC research to define a unified "contagion spectrum." It posits that cross-model diversity amplifies bias propagation (leading to potential collapse), while homogeneous evaluator pools provide natural suppression.
Design Implications: The authors argue that multi-agent system designers should:
- Prefer homogeneous model families with diverse prompts over heterogeneous model families to avoid amplifying bias.
- Measure $\Gamma$ before deployment to diagnose potential cascade risks.
- Utilize evaluator committees ( $k \ge 3$ ) as a defense-in-depth strategy.
- Monitor strategy entropy as a real-time health indicator for the system.

The paper concludes with a practical takeaway: "When you build a multi-agent system where agents evaluate each other, you are building a contagion network." It emphasizes the necessity of measuring contagion matrices and maintaining evaluator diversity to preserve the cognitive diversity essential for complex reasoning tasks.