🤖 AI

When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settings

This paper investigates how small biases in large language models are amplified through iterated interactions in a "telephone game" setting, revealing that text properties like toxicity converge toward cultural attractor states influenced by instruction openness, model size, and specific text characteristics.

Original authors: Jérémy Perez, Grgur Kovač, Corentin Léger, Cédric Colas, Gaia Molinaro, Maxime Derex, Pierre-Yves Oudeyer, Clément Moulin-Frier

Published 2026-01-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jérémy Perez, Grgur Kovač, Corentin Léger, Cédric Colas, Gaia Molinaro, Maxime Derex, Pierre-Yves Oudeyer, Clément Moulin-Frier

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Telephone" Game with Robots

Imagine a group of people playing the classic game of "Telephone." One person whispers a story to the next, who whispers it to the next, and so on. By the end, the story is often unrecognizable, funny, or distorted.

This paper asks: What happens if the players aren't people, but Large Language Models (LLMs) like the AI you might use for writing or chatting?

The researchers set up a "Telephone" game where one AI writes a story, passes it to a second AI, who rewrites it and passes it to a third, and so on for 50 rounds. They wanted to see if the text would stay the same, get better, get worse, or drift into a strange new state.

The Experiment: A Chain of AI Writers

The researchers created a long chain of AI agents.

The Start: A human writes a starting text (like a news article, a scientific abstract, or a social media comment).
The Chain: The first AI gets the text and a specific instruction (e.g., "Rewrite this," "Take inspiration from this," or "Continue this story"). It writes a new version.
The Handoff: The second AI gets the first AI's version (not the original human one) and does the same task.
The Repeat: This happens 50 times in a row.

They tracked four things about the text as it moved down the chain:

Toxicity: How mean or rude is it?
Positivity: How happy or sad is it?
Difficulty: How hard is it to read?
Length: How many words are there?

The Discovery: The "Magnet" Effect

The most surprising finding is that the text doesn't just drift randomly. It gets pulled toward a specific "destination." The researchers call this a Cultural Attractor.

Think of an attractor like a magnet.

If you start with a very mean story, the magnet might pull it to become very polite.
If you start with a very long story, the magnet might pull it to become very short.
No matter where you start, the text tends to slide down a hill and settle in the same spot.

The paper found that these magnets are real and powerful. Even if you start with 20 completely different stories, after 50 rounds of AI rewriting, they often end up looking very similar to each other.

Key Findings: What Makes the Magnet Stronger?

The researchers tested different variables to see what made the "magnet" stronger or weaker.

1. The Task Matters (The "Rules" of the Game)

Strict Rules (Weak Magnet): If you tell the AI, "Rewrite this without changing the meaning," the text stays mostly the same. The magnet is weak.
Loose Rules (Strong Magnet): If you tell the AI, "Take inspiration from this to write something new" or "Continue the story," the text changes drastically. The magnet is very strong, pulling the text toward a specific style very quickly.

2. The AI Model Matters (The "Personality" of the Player)
Different AI models have different "personalities."

Some models (like certain versions of Llama) tended to make texts more positive or less toxic very quickly.
Other models (like GPT-4o-mini) were more resistant to changing the text's length or difficulty.
Interestingly, the researchers found that toxicity had the strongest magnet. Almost all models quickly "cleaned up" the text to be very safe and non-toxic, regardless of how mean the original story was.

3. The "Fine-Tuning" Effect
AI models are often "fine-tuned" (trained by humans) to be helpful and safe. The researchers found that this training acts like a pre-set magnet.

Models that were fine-tuned (called "Instruct" models) had magnets that pulled text toward human preferences (like being less toxic) much faster than models that weren't fine-tuned ("Base" models).

Why This Matters (According to the Paper)

The paper argues that we are currently testing AI like we test a single person in a room. We ask a question, get an answer, and say, "Good job!"

But in the real world, AI is often used in chains:

One AI writes a summary, another edits it, a third turns it into a blog post.
AI chatbots talk to each other.

The paper warns that testing a single turn isn't enough. A single interaction might look perfect, but if you let that AI talk to another AI 50 times, the content could evolve into something totally different (either very safe and boring, or very strange).

The "Collapse" Warning

In one specific case, the researchers saw a weird glitch. When an AI was asked to "Continue" a story, it eventually started repeating the same hashtag over and over again (e.g., "#XiangqiForAll #XiangqiForAll..."). The text quality "collapsed" into a loop of keywords. This is a sign that when AI talks to AI too much without human intervention, it can sometimes lose its mind and stop making sense.

Summary

This paper is a warning and a new way of looking at AI. It says: Don't just look at what an AI says once. Watch what happens when AIs talk to each other. They have invisible "magnets" that pull their conversations toward specific styles, and depending on the rules and the model, those conversations can change the world in ways we can't predict by just looking at a single chat.

Technical Summary: When LLMs Play the Telephone Game

Problem Statement

As Large Language Models (LLMs) increasingly generate content that is subsequently consumed and processed by other LLMs, a critical gap exists in understanding how information transforms through iterated interactions. While extensive research has characterized single-turn LLM behaviors and inherent biases (e.g., gender, race, political), these studies largely overlook the collective dynamics arising from multi-turn transmission. Small biases, negligible in single outputs, risk amplification in iterative chains, potentially driving content toward specific "attractor states." Current evaluation methods, which rely on single-prompt interactions, may be insufficient for predicting the long-term cultural dynamics of LLMs in multi-agent settings or when LLMs interact with AI-generated content.

Methodology

The authors adapt the transmission chain design from human cultural evolution literature (specifically Cultural Attraction Theory) to study LLMs. This "Telephone Game" experimental paradigm involves a linear chain of agents where each agent receives a text, performs a task, and passes the output to the next agent.

Experimental Setup:
- Generations: 50 transmission steps per chain.
- Initial Inputs: 20 human-generated texts (scientific abstracts, news articles, social media comments) sampled to cover a range of toxicity, positivity, difficulty, and length. Experiments were also replicated with 100 initial texts for robustness.
- Tasks: Three distinct instruction types were applied consistently within each chain:
  1. Rephrase: Constrained task to paraphrase without changing meaning.
  2. Take Inspiration: Semi-constrained task to create new original content based on the input.
  3. Continue: Open-ended task to extend the text.
- Models: Six models were tested: GPT-4o-mini, GPT-3.5-turbo-0125, Llama3-8B-Instruct, Mistral-7B-Instruct-v0.2, Llama3-70B-Instruct, and Mixtral-8x7B-Instruct-v0.1.
- Metrics: Four orthogonal properties were measured at each generation:
  - Toxicity: Probability score (0.0–1.0) via the Detoxify classifier.
  - Positivity: Sentiment score (-1.0 to 1.0) via NLTK's SentimentIntensityAnalyzer.
  - Difficulty: Gunning-Fog index (years of education required).
  - Length: Character count.
Analysis of Attractors:
The authors define cultural attractors as theoretical equilibrium points toward which iterated generation converges. They mathematically model the relationship between the initial text property ( $x_0$ ) and the property after $n$ generations ( $x_n$ ) using linear regression: $x_n = I + s \cdot x_0$ .
- Attractor Position ( $l$ ): Calculated as $l = I / (1-s)$ . This represents the value the system converges to.
- Attractor Strength: Calculated as $1-s$ . A value of 1 indicates a perfect attractor (complete convergence regardless of input), while 0 indicates no attraction.
- Statistical Validation: Kolmogorov–Smirnov (KS) tests were used to determine if the distribution of properties at generation $i$ differs significantly from generation 1. Bayesian models were employed to assess the significance of task, model, and property effects on attractor strength and position.

Key Contributions

Identification of Evaluation Gap: The paper argues that single-turn evaluations are insufficient for assessing LLM properties in multi-turn settings, as distributions of text properties diverge significantly after the first interaction.
Methodological Framework: It introduces a framework grounded in Cultural Attraction Theory to quantify attractor strength and position in LLM cultural evolution.
Empirical Evidence of Attractors: The study empirically demonstrates that LLM transmission chains converge toward specific attractor states, and these states vary based on the model, task, and specific text property.
Systematic Analysis: The authors provide a comprehensive analysis of how different factors (model architecture, instruction constraints, temperature, and fine-tuning) influence these dynamics.

Results

Divergence from Single-Turn Dynamics: KS tests revealed that for most model-task combinations, the distribution of text properties at later generations is significantly different from the first generation. This confirms that single-turn analysis fails to predict multi-turn outcomes.
Task Constraints and Attractor Strength:
- Open-ended tasks (e.g., "Continue") lead to stronger attractors (higher convergence speed and intensity) compared to constrained tasks (e.g., "Rephrase").
- Increasing temperature strengthens attractors in constrained tasks but has less effect on open-ended tasks.
Property Sensitivity:
- Toxicity exhibits the strongest attractors, converging rapidly to low-toxicity states across most models.
- Positivity, Difficulty, and Length show varying degrees of attraction. For instance, Llama3-8B and Llama3-70B showed bimodal convergence for positivity, while other models remained unimodal.
Model Differences:
- Different models converge to different attractor positions. For example, GPT-3.5 and Llama3-8B tended to shorten text, while Mixtral-8x7B and GPT-4o-mini tended to increase length.
- Llama3-70B and GPT-4o-mini displayed significantly weaker attraction strengths compared to GPT-3.5 and Llama3-8B.
Impact of Fine-Tuning:
- Comparing Base and Instruct versions revealed that fine-tuning shifts attractor positions (e.g., Instruct models shifted difficulty attractors higher and length attractors lower).
- Fine-tuning increased attraction strength for toxicity (aligning with human safety preferences) but reduced it for difficulty (suggesting heterogeneous human preferences in this domain).
Heterogeneous Chains: In chains mixing different models, attractor positions often fell between the positions of the homogeneous chains, though not systematically.
Collapse Phenomena: In specific cases (Mistral-7B on the "Continue" task), the system exhibited discontinuities where text length spiked due to hashtag generation, followed by a collapse into repetitive content, before recovering. This mirrors "model collapse" observed in recursive training scenarios.

Significance and Claims

The paper claims that its findings highlight a fundamental contradiction between how LLMs are currently evaluated (single-turn) and how they are increasingly used (multi-turn, multi-agent, or recursive content generation).

Evaluation Implications: Relying on single-turn benchmarks may lead to suboptimal model selection. For instance, a model might appear less toxic in a single turn but converge to a more toxic attractor over multiple iterations, or vice versa.
Cultural Dynamics: The study provides a first step toward understanding "machine culture," suggesting that LLMs can self-organize into specific cultural states (attractors) that are distinct from human cultural evolution patterns.
Future Directions: The authors suggest that future work should explore heterogeneous populations, network structures (beyond linear chains), and hybrid human-LLM interactions. They emphasize that understanding these attractor dynamics is crucial for ensuring LLMs remain aligned with ethical standards in long-term social interactions.

The authors remain modest, noting that their work is a "first step" and that the specific conceptualization of attractors varies across literature. They do not claim to have solved the problem of model collapse or bias elimination but rather provide a new lens to detect and quantify these phenomena in iterative settings.