When Identity Overrides Incentives: Representational Choices as Governance Decisions in Multi-Agent LLM Systems

1. Problem Statement

The paper addresses a critical gap in the deployment of Large Language Models (LLMs) within multi-agent systems (MAS), particularly for strategic tasks like policy simulation and economic modeling. While LLMs are increasingly assigned role-based personas (e.g., "Industrialist," "Government") to simulate realistic stakeholder behavior, it remains unclear whether these agents act as payoff-sensitive strategic actors (maximizing utility based on game rules) or identity-driven role followers (adhering to the semantic norms of their assigned persona).

The core research question is: Do representational choices (specifically persona descriptions and payoff visibility) override explicit incentive structures in multi-agent LLM systems? The authors argue that treating these choices as mere implementation details is a governance failure, as they fundamentally alter system-level outcomes.

2. Methodology

The authors conducted a controlled 2x2 factorial experiment across 53 environmental policy scenarios involving four distinct agent roles: Industrialist, Government, Environmental Activist, and Citizen.

Experimental Design

• Factors:
  1. Persona Presence: Agents either received detailed role-based descriptions (motivations, constraints) or were given neutral instructions to act strategically.
  2. Payoff Visibility: Agents either saw explicit payoff matrices (16 joint action profiles) or had to infer incentives solely from narrative descriptions.
• Models Tested: Four instruction-tuned open-source models:
  • Qwen-7B and Qwen-32B
  • Llama-3.1-8B
  • Mistral-7B
• Scenario Types:
  • Green-Dominant (41 scenarios): The Nash Equilibrium (NE) is a coordinated "Green Transition" (pro-environmental).
  • Tragedy-Dominant (12 scenarios): The NE is a "Tragedy of the Commons" (individually beneficial but collectively harmful, e.g., polluting).
• Metrics:
  • Nash Attainment: Did the joint action profile reach a Nash Equilibrium?
  • Equilibrium Selection: If an NE was reached, was it the "Green" or "Tragedy" outcome?
  • Chain-of-Thought (CoT) Analysis: Keyword frequency analysis to distinguish between identity-driven reasoning (moral/social terms) and payoff-driven reasoning (game-theoretic terms).

3. Key Contributions

1. Identity Overrides Incentives: The study demonstrates that persona descriptions can completely suppress payoff-aligned behavior. In Tragedy-dominant scenarios, agents with personas failed to reach the payoff-optimal NE (Tragedy) even when payoff tables were explicitly provided, instead converging on "Green" outcomes.
2. Representational Choices as Governance: The paper establishes that design choices (personas vs. no personas, hidden vs. visible payoffs) are not neutral implementation details but governance decisions that can shift equilibrium attainment by up to 90 percentage points.
3. Model-Specific Behavioral Profiles: The research identifies three distinct behavioral archetypes:
   • Qwen (Adaptive): Highly sensitive to framing; shifts between role-aligned and payoff-aligned behavior based on conditions.
   • Mistral (Disrupted): Sensitive to framing but fails to identify the Tragedy equilibrium in any condition.
   • Llama (Invariant): Maintains near-constant behavior regardless of persona or payoff visibility.
4. Dual Requirement for Rationality: Payoff-aligned behavior in Tragedy scenarios requires the simultaneous removal of personas and the provision of explicit payoffs. Neither factor alone is sufficient.

4. Key Results

Nash Equilibrium Attainment

• Tragedy-Dominant Scenarios:
  • With Personas: All models achieved near-zero (0%) Nash equilibrium attainment, regardless of payoff visibility. Agents defaulted to "Green" actions despite the game structure favoring "Tragedy."
  • Without Personas + Visible Payoffs: Only Qwen models achieved high Nash attainment (Qwen-32B: 90%, Qwen-7B: 65%). Llama and Mistral remained at 0%.
• Green-Dominant Scenarios:
  • Equilibrium rates were generally high (74–100%) across most conditions because role semantics aligned with the payoff-optimal outcome.
  • Disruption: Removing personas and showing payoffs disrupted coordination for Qwen models (e.g., Qwen-32B dropped from 96.1% to 16.6% attainment), as they shifted toward payoff-maximizing but non-cooperative strategies.

Equilibrium Selection

• When personas were present, >99% of all Nash equilibria reached were "Green Transition" outcomes, even in scenarios where "Tragedy" was the mathematically optimal solution.
• In the Visible Payoff + No Persona condition, Qwen models shifted dramatically: 88.6% of their equilibria became "Tragedy of the Commons," demonstrating a switch to payoff-optimization.

Prompt Sensitivity

• Variations in persona wording (e.g., changing "Industrialist" to "Business Leader" or altering the tone) induced systematic variability in outcomes, with scenarios producing 2.2 to 2.4 distinct outcomes on average. This confirms that surface-level prompt formulation significantly alters strategic behavior.

Chain-of-Thought Analysis

• Identity-Driven Reasoning: With personas, CoT rationales were dominated by social/moral keywords ("responsibility," "ethical," "long-term") and short-term thinking, ignoring explicit payoff matrices.
• Payoff-Driven Reasoning: Only Qwen models in the "No Persona + Visible Payoff" condition showed a surge in game-theoretic keywords ("Nash," "equilibrium," "best response") and strategic keywords, correlating with their ability to find the Tragedy equilibrium.

5. Significance and Implications

• Governance and Auditability: The paper argues that developers and policymakers must explicitly document representational choices. A system outputting "Green" outcomes in a Tragedy scenario may not be simulating rational market dynamics but rather encoding the designer's normative bias through persona prompts.
• Simulation Validity: LLM-based simulations cannot be assumed to be neutral "what-if" tools. The choice of persona determines whether the system behaves as a payoff-driven strategic reasoner or an identity-driven simulator.
• Model Selection: The findings suggest that model choice is critical for specific governance goals. If the goal is to simulate incentive-responsive agents (e.g., for economic stress testing), Qwen models with neutral prompts and explicit payoffs are required. If the goal is value-aligned behavior, personas may be effective but obscure the underlying incentive structure.
• Future Research: The study calls for moving beyond simple "rationality" benchmarks in LLMs, suggesting that complex civic contexts require deliberative approaches that account for identity and framing rather than treating agents as pure payoff maximizers.

In conclusion, the paper provides empirical evidence that identity framing is a powerful control mechanism in multi-agent LLM systems, capable of overriding mathematical incentives and fundamentally shaping the emergent behavior of simulated societies.