💻 computer science

Emergence of agriculture in an artificial society of reinforcement learning agents

This paper demonstrates that agriculture can spontaneously emerge in a simulated society of reinforcement learning agents through the interplay of individual planning, social learning, and ecological feedback, revealing that social learning acts as a critical mechanism to suppress cheaters and stabilize the irreversible transition to farming.

Original authors: Gautier Hamon, Martí Sánchez-Fibla, Clément Moulin-Frier, Ricard Solé

Published 2026-05-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gautier Hamon, Martí Sánchez-Fibla, Clément Moulin-Frier, Ricard Solé

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a digital sandbox, a tiny virtual world populated by simple computer characters (agents) and a patch of land with three types of plants:

The "Gold" Plant: Delicious and nutritious, but rare and easily crowded out by weeds.
The "Weed": Grows everywhere, takes up space, but tastes like cardboard (zero reward).
The "Wild Berry": Grows randomly and is easy to pick, but not very filling.

The computer characters have one goal: eat as much as they can. They learn by trial and error, kind of like a dog learning to sit for a treat. If they eat the "Gold" plant, they get a big treat; if they eat the "Wild Berry," they get a small snack.

This paper asks a big question: How do these simple characters accidentally invent farming?

Here is the story of what the researchers found, explained through simple analogies:

1. The "Long-Term Thinker" Requirement

At first, the characters just run around grabbing the easy "Wild Berries." It's the path of least resistance. But to invent farming, a character has to be a long-term planner.

Think of it like this: If you are starving right now, you won't plant a seed because you won't eat the fruit for months. You need to be able to say, "I will skip a snack today so I can have a feast next month." In the computer model, this is controlled by a "discount factor." If the characters are too focused on the immediate moment, they never farm. They only start farming when they are smart enough to value future rewards over present snacks.

2. The "Gardening" Breakthrough

Once the characters are good at planning, something magical happens. They realize that the "Gold" plant is the best food, but the "Weed" is killing it.

The Strategy: They start doing two things:
1. Weeding: They actively pull up the "Weed" to make room for the "Gold."
2. Watering: They carry water to the "Gold" plant to help it grow faster.
The Result: The "Gold" plant explodes in number. The characters stop running around looking for wild berries and stay put in one spot, tending their garden. They have accidentally invented agriculture.

3. The "Cheater" Problem

The researchers then asked: "What happens if we add more characters?"
In a small group, everyone helps. But in a large crowd, a new type of character appears: The Cheater.

The Scenario: Imagine a community garden. Most people are watering and weeding. But then, a few characters show up who don't do any work. They just wait for the garden to grow and then eat the "Gold" plants.
The Collapse: If there are too many of these freeloaders, the hard-working farmers get tired of doing all the work for nothing. They stop farming, the garden dies, and everyone goes back to eating the low-quality "Wild Berries." The system collapses.

4. The "Firewall" of Social Learning

This is where the paper gets really interesting. How do you stop the cheaters from ruining the farm in a big group?
The researchers introduced Social Learning.

The Analogy: Imagine a parent teaching their child. If a parent is a great farmer, they pass their "farming brain" (their strategy) directly to their child. The child doesn't have to figure it out by trial and error; they just inherit the successful habits.
The Result: Even if cheaters try to join, the successful farming families keep growing and spreading their "farming DNA" to the next generation. The good strategies stick and spread, while the bad ones die out. Social learning acts like a firewall, protecting the farming community from being eaten alive by freeloaders.

5. The "Point of No Return"

Finally, the researchers tested if the characters could go back to being hunter-gatherers if the "Wild Berries" suddenly became abundant again.

The Finding: They couldn't. Once the farming system is established, it gets locked in.
The Metaphor: It's like building a house on a foundation. Even if you find a pile of wood nearby (the wild berries), you don't tear down your house to sleep in the woods. The system has changed so much that going back is impossible. The characters are now "farmers" forever, even if the environment changes.

Summary

The paper shows that agriculture isn't just a human invention; it's a natural outcome of simple rules when you have:

Patience: The ability to wait for a future reward.
Cooperation: Working together to manage resources.
Culture: Passing down successful tricks to the next generation to stop freeloaders.
Commitment: Once you start farming, you can't easily go back to the old ways.

The researchers used these computer agents to prove that you don't need complex brains or explicit instructions to invent farming; you just need the right mix of patience, social learning, and a little bit of environmental pressure.

Technical Summary: Emergence of Agriculture in an Artificial Society of Reinforcement Learning Agents

Problem Statement
The origin of agriculture represents a major evolutionary transition where species transform their environments to secure reliable food sources, a process observed in humans and various insect taxa. While comparative studies identify convergent traits (sowing, spatial engineering, crop protection), the specific mechanisms linking individual decision-making, social interactions, and ecological feedbacks that drive this transition remain unclear. Key questions include: Under what conditions does agriculture spontaneously emerge? How do systems handle "cheaters" (exploitative agents who benefit without contributing)? Is the agricultural state reversible, or does it exhibit a "lock-in" effect? This paper addresses these questions by modeling a population of artificial agents interacting with a dynamic ecological environment to identify the general principles underlying the transition from foraging to farming.

Methodology
The authors employ a Multi-Agent Reinforcement Learning (MARL) framework embedded in a simulated 2D spatial landscape ( $30 \times 30$ grid) with complex ecological dynamics.

Environment: The ecosystem contains three plant types:
1. $P_1$ (Domesticated): High nutritional reward but ecologically outcompeted by $P_2$ and requires active management (watering, weed removal).
2. $P_2$ (Weed): Common, provides no reward, but ecologically dominates $P_1$ .
3. $P_3$ (Wild/Foraging): Spontaneously generated, low reward, requires no management.
- Dynamics: The environment features seasonal cycles (Summer/Winter). Water acts as a fertilizer, improving growth in a $3 \times 3$ neighborhood. Seeds germinate based on competitive probabilities favoring $P_2$ . Agents can move, pick/drop items (seeds, water), harvest, and protect plants.
Agents: Agents are Reinforcement Learning (RL) entities using a Transformer-based neural network to map sequences of (observation, action, reward) to action probabilities.
- Objective: Maximize cumulative discounted reward $G = \sum \gamma^t R_t$ . The discount factor $\gamma$ controls the agent's temporal horizon (foresight).
- Learning: Agents use Proximal Policy Optimization (PPO) in a decentralized manner. They observe a local $11 \times 11$ neighborhood and act to maximize individual returns.
Experimental Variations:
- Hyperparameter Analysis: Varying the discount factor ( $\gamma$ ) and the spontaneous growth rate of the foraging plant ( $P_3$ ).
- Population Dynamics: Testing stability in small vs. large populations ( $N$ ).
- Social Learning (SL): Introducing a "cloning" mechanism where agents with high cumulative rewards reproduce, passing their policy to offspring, simulating cultural transmission.
- Hysteresis Testing: Gradually varying the availability of $P_3$ to test reversibility.
- Theoretical Validation: A mean-field mathematical model derived from RL results to analyze bifurcation and hysteresis.

Key Results

Conditions for Emergence: Agriculture spontaneously emerges only under specific conditions:
- High Cognitive Horizon: A high discount factor ( $\gamma$ ) is required, enabling agents to value delayed rewards (e.g., the future yield of a planted seed) over immediate foraging gains.
- Environmental Stress: Agriculture is favored when the spontaneous growth of the wild foraging plant ( $P_3$ ) is low. In resource-rich environments ( $P_3$ abundant), foraging constitutes a strong local optimum that agents fail to escape.
- Behavioral Shift: Successful agents transition from a "foraging phase" (high movement, exploiting $P_3$ ) to a "domestication phase" characterized by sedentary behavior, selective removal of the competing weed ( $P_2$ ), and active watering of $P_1$ .
Lock-in and Hysteresis:
- Once established, the agricultural state is effectively irreversible. The system exhibits hysteresis: if the environment is altered to make foraging resources ( $P_3$ ) abundant again, the population does not revert to foraging.
- A mathematical mean-field model confirms this, showing a discontinuous transition (bifurcation) where the system remains trapped in the agricultural attractor even when environmental conditions theoretically favor foraging.
Social Learning as a Firewall:
- Vulnerability to Cheaters: In larger populations ( $N \gtrsim 5$ ) without social learning, the agricultural strategy collapses. "Cheaters" (agents that harvest $P_1$ without contributing to watering or weed removal) invade, undermining the cooperative investment required for cultivation.
- Stabilization via SL: Introducing social learning (policy inheritance via cloning) acts as a "firewall." Successful agricultural strategies are propagated to new generations, allowing the population to sustain cooperative behaviors despite the presence of potential cheaters. This leads to nonlinear amplification of domesticated resources and sustained population growth.

Key Contributions

Mechanistic Explanation: The paper identifies four universal ingredients for the emergence of agriculture: (1) individual planning (valuation of delayed rewards), (2) social vulnerability to cheaters, (3) stabilization via social learning, and (4) an emergent lock-in effect.
Artificial Society as a Platform: It demonstrates that complex cultural innovations (like agriculture) can emerge from the coupling of reinforcement learning and ecological dynamics without explicit instruction or pre-existing sociocultural structures.
Role of Social Learning: It provides a computational demonstration that social learning is not merely a transmission mechanism but a critical stabilizer that prevents the collapse of cooperative regimes in larger groups.

Significance and Claims
The authors claim that their results reveal universal mechanisms linking individual decision-making, social interactions, and ecological feedbacks. The study highlights the potential of artificial societies as experimental platforms to investigate major evolutionary transitions and the emergence of cultural innovations.

The paper modestly notes limitations, such as the absence of resource storage, lack of genetic variability (precluding co-evolutionary feedbacks), and the minimal representation of social learning (cloning rather than complex teaching). However, it posits that the framework offers a tractable approach to studying broader cognitive transitions, including the emergence of ultrasociality in insects. The work suggests that the transition to agriculture is not merely a response to environmental scarcity but a complex interplay of cognitive foresight and social transmission mechanisms that create stable, self-reinforcing ecological niches.