🔬 physics

Generative artificial intelligence reduces social welfare through model collapse

This paper argues that while generative AI offers short-term individual benefits, its widespread adoption leads to model collapse and habit formation that ultimately degrade collective social welfare, particularly for critical tasks, unless interventions are implemented.

Original authors: Fabian Baumann, Erol Akçay, Joshua B. Plotkin

Published 2026-04-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Fabian Baumann, Erol Akçay, Joshua B. Plotkin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Copy-Paste" Trap

Imagine a world where everyone starts using a magical, super-smart robot assistant to write their emails, paint their pictures, and write their stories. At first, this is amazing. The robots work fast, they are cheap, and they save everyone time.

But here is the catch: The robots are learning by reading what other robots wrote.

If a robot writes a story, and then another robot reads that story to learn how to write, and then a third robot reads the second one's story... eventually, the robots stop learning from real humans. They start learning from a hall of mirrors. The stories get repetitive, the facts get fuzzy, and the creativity dries up. The paper calls this "Model Collapse."

The authors argue that while using AI feels like a smart move for you individually, if everyone does it, we all end up with worse tools and a worse society.

The Two Types of Work: "Busy-Work" vs. "Poetry"

The authors divide all the things we do into two buckets to explain how AI affects us differently.

1. The "Busy-Work" Bucket (Low Stakes)

What it is: Things like transcribing a meeting, sorting a spreadsheet, or writing a generic email.
The Analogy: Imagine a factory that makes paperclips. If a machine can make paperclips faster and cheaper than a human, that's great. If the machine makes a mistake, it doesn't matter much because paperclips aren't "art."
The AI Effect: AI is actually good here. It does the boring stuff humans hate doing. Even if the AI gets slightly worse over time, it's still better than doing nothing.
Result: Social welfare goes up. We get more done with less effort.

2. The "Poetry" Bucket (High Stakes)

What it is: Things that require deep human insight, creativity, and truth. Like writing a novel, diagnosing a rare disease, or reporting the news.
The Analogy: Imagine a chef who learns to cook only by tasting food cooked by other chefs who learned from their recipes. Eventually, the food loses its flavor. It becomes a bland, repetitive loop. The "tail" of unique, weird, and wonderful flavors disappears.
The AI Effect: This is where the disaster happens.
- The Trap: Humans are smart. If AI is 90% as good as a human but costs half the price, we will use the AI.
- The Collapse: As we use the AI more, it trains on its own output. Its quality drops to 80%, then 70%.
- The Dilemma: Even though the AI is getting worse, it's still "good enough" for us to keep using it because it's cheap. We stop doing the human work ourselves.
Result: Social welfare goes down. We end up with a society full of cheap, mediocre content, and we lose the ability to create high-quality work ourselves.

The "Habit" Problem: The Spillover Effect

The paper adds a twist that makes things even worse: Habits.

Imagine you get used to using your AI assistant to write your grocery lists (Busy-Work). You get so used to it that you start using it for your love letters and your business proposals (Poetry) too, just because "that's how I do things now."

The Metaphor: It's like getting used to driving a car with automatic transmission. You get so comfortable that you forget how to drive a manual car. If the automatic car breaks down or gets bad gas, you can't switch back to the manual car because you've lost the skill.
The Consequence: Because we get in the habit of using AI for the easy stuff, we accidentally start using it for the hard stuff, too. This "spills over" and drags down the quality of our most important work. Even if AI is great for the grocery list, using it for the love letter ruins the love letter.

The Social Dilemma: Why We Can't Stop

This is the core of the paper's argument. It's a Social Dilemma.

Individual View: "If I don't use AI, I'll be slower and more expensive than my competitors. I must use it to survive."
Group View: "If everyone uses AI, the quality of our collective knowledge drops, and we all suffer."

It's like a game of Musical Chairs, but the chairs are getting smaller every time a new robot sits down.

If you stand up (stop using AI), you lose your seat.
If everyone sits down (uses AI), the music stops, and the chairs disappear (model collapse).
The result? Everyone is worse off than if they had just kept dancing (doing the work themselves).

The Bottom Line

The paper concludes that individually rational choices lead to collectively disastrous outcomes.

For boring tasks: AI is a helpful tool that makes life easier.
For important tasks: AI is a trap. It lures us in with short-term savings, but it degrades the quality of our culture, science, and art in the long run.
The Solution: We need to be careful. We can't just let AI run wild. We need to:
- Keep humans in the loop for important work.
- Stop AI from training on its own garbage (limit the feedback loop).
- Recognize that just because AI is "good enough" doesn't mean it's "good for us."

In short: If we let the robots write all the books, paint all the pictures, and diagnose all the diseases, we might save time today, but we will have a very boring, low-quality world tomorrow.

1. Problem Statement

The paper addresses a critical paradox in the deployment of Generative AI (genAI): while genAI offers immediate individual productivity gains, its widespread adoption may degrade the quality of the models themselves through model collapse.

Model Collapse: When AI models are retrained on data generated by previous versions of themselves, the tails of the original human data distribution disappear, leading to a loss of diversity and accuracy.
The Social Dilemma: Individuals are incentivized to use genAI because it lowers the cost of work ( $c_a < c_h$ ). However, as adoption increases, the performance of the AI ( $a$ ) deteriorates. This creates a feedback loop where individually rational behavior leads to collectively suboptimal outcomes, potentially reducing social welfare below the level of a world without AI.
Gap in Literature: Existing studies often treat genAI as a static technology with stable performance. This paper models the endogenous degradation of AI performance driven by population behavior.

2. Methodology

The authors employ Evolutionary Game Theory to model the dynamics of cultural production and AI adoption.

The "Collaboration Game"

Agents: A large population ( $N \to \infty$ ) of individuals engaging in pairwise collaborations to produce a joint cultural artifact.
Strategies: Individuals choose one of three strategies:
1. Human Work (H): Incur cost $c_h$ , contribute benefit $b/2$ .
2. No Work (N): Incur cost 0, contribute benefit 0 (Free-riding).
3. AI Work (AI): Incur cost $c_a$ (where $c_a < c_h$ ), contribute benefit $a(x_{AI})/2$ .
Payoff Matrix: The total benefit is shared equally. The payoff depends on the partner's strategy and the specific costs/benefits.
Model Collapse Mechanism: The benefit of AI, $a$ , is not constant. It is a function of the frequency of AI usage in the population ( $x_{AI}$ ). The authors model this linearly:
$a(x_{AI}) = b - m \cdot x_{AI}$
Where $m > 0$ represents the severity of model collapse. As $x_{AI}$ increases, $a$ decreases.
Dynamics: Strategies evolve via replicator dynamics (imitation of more successful peers), governed by the differential equation:
$\dot{x}_s = x_s (\pi_s(x) - \phi(x))$
Where $\pi_s$ is the payoff of strategy $s$ and $\phi$ is the average population payoff (Social Welfare).

Taxonomy of Tasks

The authors classify tasks based on two dimensions:

Incentive Strength: Whether the task is performed without AI.
- High-Incentive: $b > 2c_h$ (Humans work even without AI).
- Low-Incentive: $b < 2c_h$ (Humans do not work without AI).
Collapse Severity: The strength of the feedback loop ( $m$ $m$ ).
- Weak Collapse: $m < m_c$ (AI performance remains high even with high adoption).
- Strong Collapse: $m > m_c$ (AI performance degrades significantly with adoption).

Habit Formation and Spillover

To address real-world complexity, the authors extend the model to a two-task setting:

Task 1 ("Busy-work"): Low-incentive, weak collapse (AI is beneficial).
Task 2 ("Poetry"): High-incentive, strong collapse (AI is detrimental).
Habit Constraint: Individuals are constrained to use the same strategy across both tasks (e.g., if they use AI for busy-work, they must use AI for poetry). This models the psychological tendency to form habits around AI tools.

3. Key Contributions

Formalization of Model Collapse as a Social Dilemma: The paper mathematically demonstrates that model collapse creates a tragedy of the commons where individual rationality (using cheaper AI) leads to collective ruin (degraded AI performance).
Two-Dimensional Taxonomy: It provides a framework to predict when AI will help or harm society based on task incentives and collapse severity.
Theorem 1 (Welfare Reduction): A rigorous proof showing that in high-incentive domains with strong model collapse, the introduction of AI always reduces social welfare compared to a baseline of human-only work, even if a stable mix of human and AI work emerges.
Negative Spillover Effect: The discovery that AI adoption in "low-stakes" beneficial tasks can spill over into "high-stakes" detrimental tasks via habit formation, amplifying overall welfare losses.

4. Key Results

Equilibrium Outcomes

Without AI: The population converges to either all-H (if $b > 2c_h$ ) or all-N (if $b < 2c_h$ ).
With AI (Weak Collapse): The population converges to All-AI. Since AI remains effective, everyone switches to it.
With AI (Strong Collapse):
- Low-Incentive Tasks: Converges to a mix of N (No work) and AI. Some free-ride, some use AI.
- High-Incentive Tasks: Converges to a stable mix of H (Human) and AI.

Welfare Analysis ( $\Delta S$ )

Low-Incentive Tasks: AI generally increases social welfare. It enables production that would otherwise not happen (replacing N with AI), even with some model collapse.
High-Incentive Tasks (The Critical Finding):
- If collapse is weak, AI may increase welfare if cost savings ( $c_h - c_a$ ) exceed performance loss ( $m$ ).
- If collapse is strong (Theorem 1): The introduction of AI strictly decreases social welfare ( $\Delta S < 0$ ).
- Mechanism: Even though a mix of H and AI emerges, the AI component is degraded enough that the average quality of the output drops below the pure human baseline. The cost savings of AI cannot compensate for the loss in quality caused by model collapse.

Spillover and Habit Formation

In a two-task model, if individuals are forced to use the same strategy for both "Busy-work" (Type Ia) and "Poetry" (Type IIb):
- As time spent on "Busy-work" increases, more people adopt AI for that task.
- Due to habit constraints, they also adopt AI for "Poetry."
- This forces the "Poetry" domain into the detrimental AI regime, causing a negative spillover.
- Result: The overall social welfare difference becomes increasingly negative as the proportion of "busy-work" increases, even though AI is beneficial for busy-work in isolation.

5. Significance and Implications

Policy Intervention: The paper argues that the primary point of intervention is the data feedback loop. Policies must prevent AI-generated content from being re-integrated into training corpora without detection.
- Recommendations: Deploy AI detection pipelines and enforce data provenance/labeling to preserve human data integrity.
Re-evaluating "Human-AI Collaboration": The paper challenges the optimistic view that a stable mix of human and AI work is a sign of healthy complementarity. In high-value domains, such a mix may actually be a signature of welfare-reducing feedback loops.
Broader Context: The findings align with the Braess Paradox in network theory, where adding a resource (AI) worsens collective outcomes. It suggests that individually rational adoption of genAI can assuredly reduce collective welfare in domains where human contribution is most valuable (e.g., creative writing, scientific research, journalism).
Future Outlook: Without intervention, the "poetry-like" domains (high-value cultural production) face a trajectory of stagnation and quality loss, as AI-generated content pollutes the training data for future models, creating a self-reinforcing cycle of degradation.

In summary, the paper provides a mathematical proof that unregulated adoption of genAI in high-value, creative domains is socially detrimental due to model collapse, and that habit formation can spread these negative effects to other domains, necessitating strict data governance policies.