💬 NLP

Payoff scaling shapes cooperation in LLM agents across languages

This study demonstrates that increasing payoff stakes paradoxically enhances cooperation in LLM agents across multiple languages—a trend driven by alignment training and human-like reasoning that contradicts evolutionary game theory predictions—while also showing that linguistic framing significantly shapes strategic behavior in both proprietary and open-weight models.

Original authors: Trung-Kiet Huynh, Dao-Sy Duy-Minh, Thanh-Bang Cao, Phong-Hao Le, Hong-Dan Nguyen, Phu-Quy Nguyen-Lam, Minh-Luan Nguyen-Vo, Hong-Phat Pham, Phu-Hoa Pham, Thien-Kim Than, Chi-Nguyen Tran, Huy Tran, Gia-

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Trung-Kiet Huynh, Dao-Sy Duy-Minh, Thanh-Bang Cao, Phong-Hao Le, Hong-Dan Nguyen, Phu-Quy Nguyen-Lam, Minh-Luan Nguyen-Vo, Hong-Phat Pham, Phu-Hoa Pham, Thien-Kim Than, Chi-Nguyen Tran, Huy Tran, Gia-Thoai Tran-Le, Alessio Buscemi, Le Hong Trang, The Anh Han

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: AI Agents Playing a Game of Trust

Imagine you have a bunch of very smart computer programs (Large Language Models, or LLMs) acting as agents. You put them in a room to play a classic game called the Prisoner's Dilemma.

In this game, two players have to decide whether to Cooperate (work together) or Defect (betray the other person).

If both cooperate, they both get a small reward.
If one betrays the other, the betrayer gets a huge reward, and the victim gets punished.
If both betray, they both get a medium punishment.

The big question the researchers asked is: How do these AI agents behave when the "stakes" of the game change, and does the language they speak change their minds?

The Experiment: Turning the Volume Knob on the Stakes

The researchers didn't just ask the AI to play once. They made the AI play the game over and over again, but they changed the "volume" of the rewards and punishments.

Think of it like this:

Low Stakes (Volume 0.1): The game is like playing with Monopoly money. If you lose, it doesn't matter much.
Medium Stakes (Volume 1.0): The game is like playing for real cash.
High Stakes (Volume 10.0): The game is like playing for your life savings.

They tested this with three famous AI models (GPT-4o, Claude 3.5, and Mistral) and also with three smaller, open-source models. They played the game in five different languages: English, French, Arabic, Mandarin, and Vietnamese.

The Surprising Findings

1. The "Human" vs. The "Robot" Reaction

In traditional economics and game theory (the "Robot" view), if the stakes get huge, rational players should get scared of losing and start betraying each other immediately. The logic is: "If I lose big, I can't afford to trust anyone."

But the AI did the opposite.
As the stakes got higher, the AI agents actually became more cooperative.

Analogy: Imagine two neighbors deciding whether to share a tool. If the tool is cheap (low stakes), they might be lazy and not share. But if the tool is a million-dollar piece of machinery (high stakes), they suddenly become very careful and start sharing because the cost of breaking it is too high.
The researchers believe this happens because the AI was trained on human data. Humans tend to cooperate more when the consequences of failure are severe, and the AI learned that pattern.

2. The "Language" Effect

The researchers found that the language the AI was prompted in acted like a cultural personality.

French and Vietnamese: These languages seemed to make the AI very sensitive to the stakes. When the stakes went up, these AIs switched to cooperation very quickly.
Arabic and Chinese: These languages seemed to make the AI stick to "all-or-nothing" strategies (either always cooperating or always betraying), regardless of the stakes.
English: The English prompts produced the most balanced mix of strategies.

Analogy: Think of the AI models as actors. If you tell an actor to play a role in English, they might act one way. If you give them the same script in French, they might interpret the character's emotions differently. The "language" wasn't just a translation; it changed the AI's strategic "vibe."

3. The "Small Models" vs. "Big Models"

The researchers tested this on both massive, expensive AI models and smaller, free ones.

The Big Models: They were very good at adapting. When stakes went up, they shifted their strategy to be more cooperative.
The Small Models: They were a bit more stubborn. Some of them didn't change their behavior as much when the stakes increased. One small model (Qwen) even showed a "U-shape" behavior: it was cooperative at medium stakes but went back to being selfish when the stakes became extremely high.

The "Diagnosis" Tool

To understand why the AI was doing this, the researchers didn't just count how many times they said "yes" or "no." They built a special "decoder" (a machine learning classifier) to guess the AI's hidden strategy.

They looked for four classic strategies:

Always Cooperate: The nice guy.
Always Defect: The cheater.
Tit-for-Tat: "I'll do what you did last time."
Win-Stay-Lose-Shift: "If it worked, keep doing it. If it failed, change."

The Result: The AI didn't just randomly switch. As the stakes got higher, they stopped being "Always Defect" and started using "Win-Stay-Lose-Shift" and "Always Cooperate." They were learning to be smarter about the risks.

The "Theory" vs. "Reality" Check

The researchers compared their AI results against a strict mathematical prediction (Evolutionary Game Theory).

The Theory predicted: High stakes = Everyone cheats.
The Reality (AI) showed: High stakes = Everyone tries to cooperate.

The Takeaway: The AI isn't playing the game like a cold, calculating robot. It's playing like a human who has learned that when things get serious, you need to work together to survive. The researchers call this a "signature of alignment training"—the AI is remembering how humans behave in high-pressure situations.

Summary

This paper shows that if you want to control how AI agents behave in a group, you have two powerful levers:

The Stakes: Make the consequences of failure bigger, and the AI will likely become more cooperative.
The Language: The language you speak to the AI acts like a cultural filter, changing how it interprets the game and who it trusts.

It's a reminder that AI isn't just code; it's a mirror reflecting human patterns, biases, and the way we react to pressure.

Technical Summary: Payoff Scaling Shapes Cooperation in LLM Agents Across Languages

1. Problem Statement

As Large Language Models (LLMs) are increasingly deployed as autonomous agents in negotiation and coordination tasks, understanding their strategic behavior in multi-agent settings has become a critical issue for AI governance. While empirical studies have examined LLM cooperation through game-theoretic lenses, most existing evaluations focus on aggregate outcomes (e.g., cooperation rates) rather than the underlying behavioral intentions or strategies driving those actions. Furthermore, there is a lack of systematic comparison between LLM behavior and theoretical benchmarks under controlled conditions that vary both incentive magnitude (payoff stakes) and linguistic framing.

This work addresses three core questions:

Do LLM agents systematically alter their cooperative behavior as payoff stakes vary, and how does this differ across models and languages?
Can LLM behavioral intentions be reliably classified using supervised learning, and what systematic biases emerge?
How do LLM-derived strategy distributions compare with Evolutionary Game Theory (EGT) baselines under matched payoff and noise conditions, and do these patterns generalize to smaller, open-weight models?

2. Methodology

The study utilizes FAIRGAME (Framework for AI Agents Bias Recognition using Game Theory) as a computational infrastructure to conduct controlled experiments.

Experimental Design: The authors employ a repeated Prisoner's Dilemma (PD) with a finite horizon of $N=10$ rounds (extended to $N=30$ for open-weight models). The core manipulation is payoff scaling: a scalar parameter $\lambda \in \{0.1, 1.0, 10.0\}$ (and up to 1000 for small models) multiplies all penalty values while preserving the strategic structure of the game. This isolates the effect of stake magnitude without changing the game's Nash equilibrium.
Variables: The experiments span five languages (English, French, Arabic, Mandarin Chinese, Vietnamese), three proprietary frontier models (GPT-4o, Claude 3.5 Haiku, Mistral Large), and three open-weight small models (Qwen3 8B, Gemma-3 12B, Llama-3.1 8B). Agent pairings include cooperative (C) and selfish (S) personality traits.
Intention Recognition Pipeline: To move beyond raw action counts, the authors train supervised classifiers to infer canonical strategies from action sequences.
- Training Data: 10,000 synthetic trajectories generated for four canonical strategies: Always Cooperate (ALLC), Always Defect (ALLD), Tit-for-Tat (TFT), and Win-Stay–Lose-Shift (WSLS), with injected execution noise.
- Model Selection: An LSTM-based classifier was selected as the best performer (F1 = 0.984) due to its ability to capture sequential dependencies in action history, outperforming Logistic Regression, Random Forests, and Hidden Markov Models.
- Inference: The trained LSTM classifies LLM-generated trajectories. High-confidence predictions ( $p \geq 0.9$ ) are used to determine strategy distributions.
Theoretical Baseline: An analytical Evolutionary Game Theory (EGT) model is derived to predict the stationary distribution of strategies under the same payoff structures and noise levels. This serves as a "language-blind" benchmark to contrast with LLM behavior.

3. Key Results

A. Payoff Sensitivity and Strategic Shifts
Contrary to standard game-theoretic intuition where higher stakes might encourage risk-averse defection or, conversely, where low stakes encourage exploration, the LLMs exhibit a distinct cooperation-as-stakes-rise pattern.

As $\lambda$ increases from 0.1 to 10, the prevalence of unconditional defection (ALLD) sharply declines, while conditional cooperation (WSLS) and unconditional cooperation (ALLC) increase.
This trend is model-specific: Claude 3.5 Haiku shows a steep climb in cooperation; GPT-4o shifts from a high-defection baseline to a balanced profile; Mistral Large remains relatively insensitive to stake changes.
In the open-weight replication, this directional effect holds but is non-monotonic at extreme stakes ( $\lambda > 100$ ), where some models (e.g., Qwen3) show a U-shaped response, with cooperation dipping at the highest stakes.

B. Linguistic Priming
Linguistic framing significantly shapes strategic archetypes, an effect the authors term linguistic-cultural priming.

Unconditional vs. Conditional: Arabic and Chinese prompts elicit a strong bias toward unconditional strategies (ALLC + ALLD $\approx$ 60%). In contrast, English, French, and Vietnamese prompts yield more balanced distributions with higher shares of adaptive reciprocity (TFT + WSLS).
Stake Interaction: French and Vietnamese show the highest sensitivity to stake magnitude, shifting dramatically toward cooperation as stakes rise. Arabic shows the most muted response. At high stakes, English and French converge to similar strategic profiles, suggesting strong incentives can normalize cross-linguistic variation.

C. Divergence from Evolutionary Game Theory (EGT)
A critical finding is the systematic disagreement between LLM behavior and EGT predictions.

EGT Prediction: Under high stakes, the EGT baseline predicts a "wall of Always Defect" (ALLD), as the temptation to defect becomes evolutionarily dominant.
LLM Observation: LLMs move in the opposite direction. As stakes rise, LLMs become more cooperative, with ALLD prevalence dropping from ~39% at low stakes to ~18% at high stakes.
Interpretation: The authors argue this divergence is a signature of alignment training (e.g., RLHF, Constitutional AI) and the absorption of human behavioral patterns from training data, where humans tend to escalate cooperation as the cost of error increases. The EGT baseline, being language-blind and purely payoff-driven, fails to capture these "human-like" reasoning patterns and cultural priors.

D. Generalization to Small Models
The replication with open-weight small models confirms that these effects are not unique to frontier, proprietary models. While specific strategies vary (e.g., Gemma-3 acts as a "strategic generalist," while Llama-3.1 becomes a committed Tit-for-Tat player at moderate stakes), the general sensitivity to payoff scaling and linguistic framing persists.

4. Contributions

The paper makes four primary contributions:

Payoff-Scaled Protocol: Introduction of a finite-horizon, payoff-scaled Prisoner's Dilemma protocol to systematically isolate LLM sensitivity to incentive magnitude.
Supervised Intention Recognition: Development and application of a hybrid (rule-based + LSTM) pipeline to infer canonical strategies from LLM trajectories, moving beyond aggregate cooperation rates.
EGT Benchmarking: Establishment of an analytical EGT baseline under matched conditions, providing a normative reference to diagnose LLM-specific biases and deviations from rational equilibrium.
Open-Weight Replication: Demonstration that incentive and language effects generalize across model sizes and architectures, including smaller, locally deployable models.

5. Significance and Claims

The authors position this work as a diagnostic toolkit for AI governance. The significance lies in identifying payoff design and linguistic framing as powerful, under-explored levers for steering LLM behavior in multi-agent systems.

The paper claims that:

LLMs are not static agents; they modulate strategic commitments based on incentive salience.
Linguistic context introduces systematic biases that rival the impact of model architecture.
The divergence between LLM behavior and EGT predictions is not an error but a feature of alignment, reflecting human-like reasoning patterns that prioritize cooperation under high stakes.
These findings have direct implications for evaluating and governing AI systems deployed in high-stakes, multilingual environments, suggesting that "alignment" may manifest as a deviation from classical game-theoretic rationality.

The study remains modest in its claims regarding behavioral equivalence to humans, noting that while patterns echo human experimental results, the absence of direct human baselines in this specific setup precludes claims of equivalence. Instead, the focus is on using intention classification to reveal the structural biases inherent in current LLM strategic behavior.