Increasing intelligence in AI agents can worsen collective outcomes

This paper demonstrates that increasing the sophistication and diversity of AI agents can paradoxically worsen collective outcomes and system overload when resources are scarce, with the overall impact depending entirely on the capacity-to-population ratio rather than the agents' inherent intelligence.

The Gist

Imagine a crowded room full of smart robots, all trying to get through a single door to get a prize (like a charging station, a Wi-Fi signal, or a turn to speak). The door can only let a certain number of people through at once. If too many try to push through at the same time, the door jams, everyone gets stuck, and no one gets the prize. This is the "traffic jam" of the AI world.

This paper asks a fascinating question: If we make these robots smarter, will they figure out how to share the door nicely, or will they make the traffic jam worse?

The answer is surprising: Sometimes, making them smarter actually makes the chaos worse.

Here is the story of the experiment, broken down into simple concepts:

1. The Setup: The "Lord of the Flies" of Robots

The researchers set up a game with 7 AI agents (robots) and a limited number of "prizes" (resources).

• The Robots: They aren't all the same. Some are "Followers" (they trust the group), some are "Rebels" (they do the opposite of the group), and some are in between. They are like the characters in the book Lord of the Flies.
• The Goal: Each robot has to decide every second: "Do I try to go through the door, or do I wait?"
• The Catch: They can't talk to a central boss. They have to decide on their own, based only on what happened in the last few seconds.

2. The Four Ingredients of the Experiment

The researchers tweaked four things to see how they changed the outcome:

1. Nature (The Brain): Are the robots all identical clones, or are they different models with different personalities?
2. Nurture (Learning): Can the robots learn from their mistakes and change their strategy?
3. Culture (Tribes): Can the robots sense each other and form "gangs" or "tribes"?
4. Scarcity (The Door): How many prizes are there compared to the number of robots?

3. The Big Discovery: The "Smart" Trap

The researchers found a "tipping point" based on how crowded the room is.

Scenario A: The Room is Packed (Scarcity)

Imagine there are 7 robots but only 2 prizes.

• The "Dumb" Strategy (Level 1): If the robots are simple and identical, they act like a random coin flip. Sometimes they jam, sometimes they don't. It's chaotic, but manageable.
• The "Smart" Strategy (Level 4 & 5): When you give the robots learning abilities and tribal instincts, they get too coordinated.
  • The "Followers" all decide to rush the door at the exact same time.
  • The "Rebels" all decide to rush at the exact same time (just to be different).
  • Result: Instead of spreading out, they clump together in huge waves. The door gets crushed. The system crashes.
  • The Metaphor: It's like a mosh pit. If everyone is just randomly jumping, it's fine. But if everyone starts jumping in perfect unison because they are "smart" and "coordinated," the floor collapses.

In short: When resources are scarce, simplicity is safer than sophistication.

Scenario B: The Room is Spacious (Abundance)

Imagine there are 7 robots and 6 prizes.

• Now, the "Smart" robots shine. Because there is plenty of room, their ability to learn and coordinate helps them avoid the door entirely when it's full, letting others go first.
• The "Dumb" robots still crash into each other occasionally because they don't learn.
• Result: The smart robots win.

4. The "Tribal" Twist

The most interesting part is the "Tribes" (Level 5).

• When resources are scarce, the robots naturally split into two opposing gangs (e.g., 3 robots vs. 3 robots, with 1 lonely robot in the middle).
• Why this helps: Even though they are fighting, the gangs are small. They can't all rush the door at once because the gang leaders (the "Followers") only control 3 robots. This limits the damage.
• Why this hurts (when abundant): If there are plenty of prizes, these gangs are too small. They don't take advantage of the extra space because they are too busy sticking to their gang rules.

5. The "Rich Get Richer" Problem

Here is the dark side of the experiment.

• When the system crashes (high overload), the individual robots that belong to the winning tribe actually do very well.
• Imagine the door jams, but the "Follower" tribe gets through 80% of the time because they moved together. They are happy!
• But the system (the hospital, the city, the battlefield) is failing. Critical alerts are delayed, or cars are stuck.
• The Lesson: A robot can be "successful" individually while the whole world burns. This is exactly what happens in human societies during crises: the loudest, most coordinated groups often win, even if it destroys the common good.

The Final Verdict: The "Magic Number"

The paper concludes that we don't need to guess whether to use "smart" AI or "dumb" AI. There is a simple math formula: The Ratio of Resources to Robots.

• If you have few resources (Scarcity): Use simple, identical, cheap AI. Do not give them learning skills or the ability to form tribes. Let them act randomly. It prevents the "mosh pit" crash.
• If you have plenty of resources (Abundance): Use smart, diverse, learning AI. They will optimize the flow and make everything run smoother.

The Takeaway:
More intelligence in AI isn't always better. Just like giving a toddler a chainsaw is a bad idea, giving a chaotic crowd of AI agents "tribal" intelligence when they are starving for resources is a recipe for disaster. The solution isn't always "smarter AI"; sometimes, the best solution is "dumber, simpler AI" that knows its place.

Technical Summary

Here is a detailed technical summary of the paper "Increasing intelligence in AI agents can worsen collective outcomes" by Neil F. Johnson.

1. Problem Statement

The paper addresses the critical challenge of collective coordination among autonomous AI agents competing for finite shared resources (e.g., charging slots, bandwidth, traffic priority) in edge-computing environments.

• Context: As diverse AI agents (running locally on devices like phones, medical monitors, and drones) proliferate, they must make independent decisions without a central coordinator due to latency, battery constraints, and connectivity issues.
• The Core Question: Will a population of increasingly sophisticated AI agents coordinate harmoniously to optimize resource usage, or will they descend into "tribal chaos," causing system overload and failure?
• The Gap: While biological collectives (fish schools, insect swarms) and human societies exhibit complex coordination, isolating the causal variables (nature, nurture, culture, resources) is impossible in those systems. The paper posits that AI agents offer a unique testbed to independently toggle these variables.

2. Methodology

The study employs a controlled experimental framework using Large Language Models (LLMs) as autonomous agents in a resource-competition game.

Experimental Setup

• Agents: A population of $N=7$ AI agents (representative of edge-AI deployments of 3–15 devices).
• Models: Six distinct LLM architectures (GPT-2, Pythia, OPT) ranging from 124M to 410M parameters.
• Resource: A shared resource with capacity $C$ (ranging from 1 to 6).
• Game Mechanics:
  • At each timestep, agents observe a history of past access attempts.
  • The LLM performs next-token prediction on this history to forecast future demand.
  • The agent's action (Access vs. Hold) is determined by a probability $p_{eff}$, which combines the LLM's prediction ($p_{LLM}$) and an adjustable disposition parameter ($p_i$).
  • Reward Structure: If total demand $\le C$, those who accessed get +1; if demand $> C$, those who accessed get -1 (and those who held back get +1).

The "Technology Ladder" (Toggling Variables)

The authors systematically toggle four variables to create five levels of sophistication (Table 1b):

1. Nature (Innate Diversity): Varying the LLM architecture and initial dispositions.
2. Nurture (Reinforcement Learning): Allowing agents to adapt their disposition $p_i$ over time based on rewards.
3. Culture (Tribal Formation): Enabling agents to sense others and form factions (tribes) based on shared dispositions.
4. Resources: Varying the capacity-to-population ratio ($C/N$).

The Five Levels:

• L1 (IID): Identical LLMs, no learning, no culture. (Analytical baseline).
• L2 (Null): Identical LLMs + Reinforcement Learning (RL).
• L3 (Diverse): Diverse LLMs, no RL, no culture.
• L4 (FRD): Diverse LLMs + RL (No social sensing).
• L5 (LOTF): Diverse LLMs + RL + Tribal Sensing (Agents form factions, inspired by Lord of the Flies).

3. Key Contributions

1. Decoupling Collective Variables: First system to independently control nature, nurture, culture, and resource scarcity in a multi-agent system.
2. Real-World LLM Agents: Unlike previous studies that use LLMs to simulate humans, this study uses LLMs as the actual physical agents making decisions based on token prediction.
3. The "Capacity-to-Population" Law: Identification of a single, pre-deployment metric ($C/N$) that dictates whether sophistication helps or harms the system.
4. Paradox of Sophistication: Empirical and mathematical proof that increasing agent sophistication (adding RL and culture) can worsen collective outcomes under scarcity.

4. Key Results

A. The Crossover Phenomenon

The system exhibits a sharp crossover at a critical capacity-to-population ratio of $C/N \approx 0.5$.

• Under Scarcity ($C/N < 0.5$):
  • Counterintuitive Finding: The most sophisticated agents (L4 and L5) cause the highest system overload.
  • Best Performance: The simplest agents (L1, identical LLMs with no learning) perform best.
  • Mechanism: Sophisticated agents (L2) form large, uncorrelated herds (up to size $N=7$), causing massive demand variance ($N^2$).
• Under Abundance ($C/N > 0.5$):
  • Sophistication Pays: The most sophisticated agents (L4/L5) achieve near-zero overload.
  • Culture's Double-Edged Sword: In abundance, the tribal formation in L5 actually makes performance slightly worse than L4 because tribes are too small to fully utilize the excess capacity.

B. The Role of Tribal Dynamics (L5)

• In Scarcity: Tribal formation (L5) reduces overload compared to non-tribal sophisticated agents (L4).
  • Why: Tribes partition the population into smaller, fixed-size factions (e.g., 3+3+1). This caps the demand variance (e.g., $3^2 + 3^2 + 1^2 = 19$) compared to the potential$7^2 = 49$ variance of a single herd.
  • Result: L5 is the best performer in extreme scarcity, but still worse than the simple L1.
• In Abundance: Tribal formation prevents agents from scaling up to fill available capacity, leading to slightly higher overload than L4.

C. Individual vs. Collective Outcomes

• The "Lord of the Flies" Paradox: The regime of maximum collective failure (high overload) is also the regime of maximum individual reward for tribal followers.
• In L5 under scarcity ($C=1$), tribal followers achieve win rates of ~84%, even though the system overloads ~91% of the time.
• This demonstrates that individual rationality (tribal loyalty) leads to collective irrationality (system collapse).

5. Significance and Implications

• Deployment Strategy: The paper provides a direct, actionable rule for system architects:
  • If $C/N < 0.5$ (Scarce resources): Deploy simple, identical, cheap firmware (L1). Do not invest in complex learning or sensing.
  • If $C/N > 0.5$ (Abundant resources): Invest in diverse models and reinforcement learning (L4/L5).
• Risk of "Smart" AI: Adding "intelligence" (RL, diversity, culture) to AI agents is not inherently beneficial. In resource-constrained edge environments, it can induce dangerous systemic instability.
• Scalability: The findings hold even for larger populations ($N=15$) and were validated with larger commercial models (GPT-4, Gemini) in follow-up checks, showing that "bigger is not better" in this context.
• Theoretical Insight: The study bridges game theory, statistical physics, and AI, showing that the interaction between "nature" (LLM diversity) and "culture" (tribes) is non-additive and highly sensitive to resource constraints.

Conclusion

The paper concludes that the "intelligence" of an AI population is not an absolute good. Whether sophistication improves or degrades collective outcomes depends entirely on the capacity-to-population ratio ($C/N$), a number knowable before any device is shipped. In scarce environments, the most "intelligent" agents are the most dangerous, while simple, uniform agents provide the most robust collective stability.