Technical Summary: Modeling Decision-Making with Will for Cooperation in Social Dilemmas

Problem Statement

Standard rational actor models in economics and cognitive science typically attribute failures in social dilemmas (e.g., Stag Hunt, Prisoner's Dilemma) to insufficient incentives or a lack of computational power. These models assume agents continuously optimize expected utility by performing local cost-benefit evaluations. However, this commitment to scalar maximization can amplify perceived risks and strategic caution, driving agents toward safe but collectively suboptimal equilibria. Existing solutions often rely on external mechanisms like punishment, reputation systems, or centralized enforcement, which incur implementation costs. This paper posits that the root of the problem may not be insufficient optimization, but rather the reliance on continuous rational computation itself. It asks whether a cognitive mechanism that strategically suspends ongoing re-evaluation can enable groups to escape suboptimal non-cooperative equilibria without altering the underlying payoff structure.

Methodology

The authors propose a framework of "Will," defined as a mechanism where an agent persistently pursues a target goal while ignoring local cost-benefit fluctuations. The study employs three distinct analytical approaches:

1. Mathematical Formalization

The social environment is modeled as a Markov Game with Will, defined by the tuple $M = \langle N, S, A, P, R, G, D, T \rangle$.

• Willed Agents: Unlike rational agents that maximize cumulative extrinsic utility ($\sum R_i$), Willed Agents are defined as potential minimizers. They minimize the distance $D_i(s)$ to a target state set $G_i$ using a pseudometric. Their policy $\pi_{will}$ greedily minimizes the expected distance to the goal, effectively ignoring the scalar reward signal $R$.
• Potential Attractors: States where a Willed Agent becomes "pinned" by environmental boundaries or interaction dynamics, acting as fixed boundary conditions in the phase space.
• Population Dynamics: The authors analyze an infinite population ($N \to \infty$) consisting of two modes:
  • Willed Mode: Agents rigidly committed to Cooperation ($G_1$) or Defection ($G_2$) with fixed proportions $n_1$ and $n_2$.
  • Rational Mode: Agents ($m = 1 - n_1 - n_2$) who adapt strategies to maximize cumulative reward based on payoff differentials $\Delta f(x)$.
  • The system evolution is modeled via a Langevin equation subject to social noise, where Willed Agents constrain the feasible state space $\Omega = [n_1, 1-n_2]$.

2. Theoretical Analysis of Canonical Games

The framework is applied to three game paradigms to observe how Willed Agents reshape collective outcomes:

• Stag Hunt: Willed Cooperators ($n_1$) can eliminate the coordination trap if they exceed the tipping point, or stochastically accelerate convergence by raising the starting floor of the population.
• Snowdrift Game: Willed Agents act as a window, potentially forcing the system to over-cooperation or under-cooperation depending on whether the fixed proportions exceed the optimal interior equilibrium.
• Prisoner's Dilemma: Willed Agents provide a stability floor ($x^* = n_1$), preventing total utilitarian collapse, though they remain exploited by rational defectors.

3. Spatiotemporal Simulations

To address finite populations and temporal constraints, the authors simulate a Markov Stag Hunt Game on a grid world ($H \times W$).

• Agents:
  • Rational Mode: Uses Bayesian Theory of Mind to infer peer goals from trajectories and Monte Carlo simulations to plan for maximum reward.
  • Willed Mode: Uses Manhattan distance minimization to a target prey.
• Will Strength ($\alpha$): A parameter controlling the duration of commitment. Agents follow the Willed Mode for the first $|\alpha|T$ steps before switching to Rational Mode.
• Experiments: The study varies the proportion of Willed Agents, the strength of will ($\alpha$), and the coordination threshold ($\theta$). It also tests Endogenous Will Selection, where agents autonomously decide when to switch from planning to commitment (Intermittent vs. Phased strategies).

Key Results

1. Willed Agents as Cooperation Catalysts

In the Stag Hunt game, Willed Agents function as "cooperation catalysts." By restricting the feasible state space, they act as boundary constraints that accelerate convergence.

• Context Dependence: Their efficacy is non-monotonic and depends on the difficulty of coordination ($\theta$).
  • At low difficulty ($\theta=2$), persistence is a liability; rational agents coordinate better.
  • At moderate difficulty ($3 \le \theta \le 7$), a moderate level of will ($\alpha \approx 0.5$) optimally anchors agents to the cooperative equilibrium.
  • At extreme difficulty ($\theta \ge 8$), the strategy fails as the probability of success becomes too low.
• Critical Mass: Under high coordination demands or tight time constraints, a critical mass of Willed Agents is required to "fast-track" the group toward the cooperative equilibrium before the episode ends.

2. Heterogeneity and Altruism

• Optimal Heterogeneity: In heterogeneous populations, the optimal distribution of will strength varies with difficulty. High-$\alpha$ agents anchor the goal, while lower-$\alpha$ agents retain flexibility.
• Individual Cost: High-will agents often earn lower individual payoffs than their lower-will counterparts. They lower the energetic barrier for coordination but pay a fitness cost for their rigidity, suggesting the "Will to Stag" functions as a form of altruistic commitment.

3. Endogenous Selection and Resolute Choice

• Phased vs. Intermittent: Agents that commit to a goal early (Phased strategy) and suspend re-evaluation outperform those that frequently re-evaluate (Intermittent) under high-stakes uncertainty.
• Superiority of Suspension: Under high incentives, an "Instant" strategy (planning once at $t=0$ and executing unwaveringly) significantly outperforms continuous rational optimizers. Continuous re-evaluation makes agents hypersensitive to stochastic noise, leading to the premature abandonment of joint goals.

Significance and Claims

The paper claims to provide the first mathematical formulation of "will" for multi-agent decision-making. Its primary contributions are:

1. Reframing the Problem: It challenges the premise that cooperation failures are due to insufficient optimization, proposing instead that the reliance on continuous rational computation is destabilizing.
2. Mechanism of Action: It demonstrates that "will" is not a new decision rule replacing utility maximization, but a complementary mechanism that operates on top of it by strategically constraining calculation.
3. Stabilizing Cooperation: The findings suggest that successful cooperation in social dilemmas relies on the cognitive capacity to suspend local cost-benefit re-evaluation. By "binding their will" to a target, agents transform into reliable focal points, signaling predictability to peers and enabling groups to surmount high-risk thresholds that purely rational agents cannot cross.

The authors conclude that while "willed" behavior incurs local fitness costs, it serves a critical evolutionary and social function in stabilizing cooperation under conflicting incentives and strategic uncertainty. Future work is suggested to explore the evolutionary dynamics of these agents and their integration with multi-agent reinforcement learning for meta-policy regulation.