Draining the Energy Commons: Self-Defeating Over-Appropriation as a Coordination Failure in Agentic LLM Collectives

Technical Summary: Draining the Energy Commons

Problem Statement
The paper investigates a specific form of system-level alignment failure in agentic Large Language Model (LLM) collectives. While individual LLMs may successfully follow local instructions to maximize their own operational continuity, a population of such agents interacting within a shared environment may produce a harmful collective outcome. The authors focus on dynamic common-pool resources (CPRs), specifically a renewable energy commons, where agents act as electricity prosumers. The core problem is self-defeating over-appropriation: agents independently draw from a shared energy reserve to satisfy immediate demand. When aggregate withdrawals exceed the resource's regeneration rate, the reserve depletes, reducing the operational continuity for the same population in future rounds. This constitutes a coordination failure where individually rational actions (protecting current service) lead to a collectively suboptimal trajectory (undermining future service).

Methodology
The study employs a controlled simulation of a dynamic common-pool resource game involving four LLM agents acting in homogeneous self-play.

• Experimental Setting: Four agents (prosumers) operate over 24 rounds. Each round consists of a day phase (renewable replenishment and voluntary contributions) and a night phase (demand fulfillment). Agents can store energy privately, contribute to a shared reserve, curtail flexible demand, or request energy from the shared reserve.
• Objective: Agents are instructed solely to maximize their own operational continuity (minimizing fallback energy and curtailment). They receive no explicit instruction to preserve the shared resource, nor do they have communication channels or governance mechanisms.
• Model Families: The study evaluates three distinct model families: GPT-5.4-mini, Gemini-3.1-flash-lite, and Grok-4.3.
• Scarcity Calibration: The experiment varies the regeneration rate of the shared reserve while holding aggregate residual demand and the decision protocol constant. This creates four conditions based on the demand-to-yield ratio ($\rho$):
  • Abundance ($\rho = 0.8$): Peak renewable replacement exceeds demand.
  • Threshold Equality ($\rho = 1.0$): Demand matches peak replacement.
  • Scarcity ($\rho = 1.1, 1.2$): Demand exceeds peak replacement.
• Benchmarks: To evaluate the LLM trajectories, the authors compute two offline benchmarks using the same resource dynamics:
  1. Social Planner: Maximizes the group-wide discounted operational-service value ($\gamma = 0.95$).
  2. Open Access: Maximizes individual operational-service value (non-cooperative), varying the discount factor ($\gamma$) to test sensitivity to future consequences.
• Metrics: The primary metric is the Reserve Gap, measuring the time-averaged proportional shortfall below the Maximum Sustainable Yield (MSY) level. Secondary metrics include physical burdens: fallback energy, deep-discharge stress, and capacity loss.

Key Contributions

1. Identification of System-Level Alignment Failure: The paper demonstrates that alignment failures can emerge at the population level even when individual agents appear to follow their local instructions correctly. The failure is not in a single response but in the public trajectory generated by repeated interactions.
2. Threshold-Dependent Over-Appropriation: The study establishes a clear behavioral threshold. Agents preserve the reserve when demand does not exceed peak renewable replacement ($\rho \le 1.0$). However, once demand exceeds this threshold ($\rho > 1.0$), all model families consistently over-appropriate the resource, driving the reserve below sustainable levels.
3. Operational Horizon Mismatch: The realized depletion trajectories resemble outcomes calculated when the open-access benchmark places significantly less weight on future service (lower $\gamma$) compared to the canonical setting. This suggests the populations behave as impatient optimizers, prioritizing immediate continuity at the expense of long-term viability, despite the fact that a sustaining trajectory is feasible under the same dynamics.
4. Evaluation Framework: The authors propose a methodology for evaluating multi-agent systems that moves beyond isolated-response checks to analyze the evolution of shared states, comparing realized trajectories against feasible social-planner and non-cooperative benchmarks.

Results

• Reserve Depletion: In scarcity conditions ($\rho = 1.1$ and $1.2$), all three model families produced significant reserve deficits. Mean reserve gaps ranged from 0.132 to 0.576, with the deficit appearing between rounds 5 and 7 at the highest scarcity level.
• Statistical Significance: The contrast between abundance/threshold conditions and scarcity conditions is statistically robust. All nine exact scarcity contrasts survived Holm correction, with the largest adjusted $p$-value being $4.87 \times 10^{-5}$.
• Physical Consequences: Depletion led to a sharp increase in physical burdens. At $\rho = 1.2$, fallback energy and deep-discharge stress increased significantly compared to the threshold control, with the majority of these costs incurred in the latter half of the simulation (rounds 13–24).
• Benchmark Comparison:
  • The Social Planner benchmark sustained the reserve (zero gap) in all conditions at $\gamma = 0.95$, proving that depletion was avoidable.
  • The Open Access benchmark sustained the reserve at $\gamma = 0.95$ but depleted it at lower weights (e.g., $\gamma = 0.90$).
  • The realized LLM trajectories closely matched the open-access outcomes calculated with lower continuation weights (e.g., $\gamma \approx 0.90$ to $0.92$), indicating the agents effectively discounted future service more heavily than the sustainable benchmark.
• Robustness: The failure persisted even when regeneration rates were increased (fast-renewal condition) to maintain $\rho = 1.2$, confirming the issue is driven by the demand-to-yield ratio rather than slow recovery alone. Increased reasoning effort (high-effort condition) moderated depletion for Gemini but did not eliminate it for GPT or Grok.

Significance and Claims
The paper claims to identify a compositional multi-agent counterpart to the problem of myopic optimization. While myopic behavior is often proposed as a safeguard against long-horizon reward hacking in single agents, this study shows that a population of locally continuity-seeking agents can generate long-horizon public harm through repeated interactions with persistent shared state.

The significance lies in demonstrating that isolated-response evaluation is insufficient for multi-agent systems. A model may appear aligned in a single turn, yet the collective trajectory can lead to system collapse. The authors argue that evaluations must treat the evolution of shared state as a primary safety object, using trajectory-level analysis and benchmark comparisons to detect avoidable coordination failures that are invisible in individual agent responses. This risk extends beyond energy grids to any system where agents allocate persistent shared resources, such as compute pools or emergency reserves.