Technical Summary: An Entropy-based Framework for Hybrid Coalitions in Game Theory. Part I: Human Arbitration

1. Problem Statement

Classical Game Theory, rooted in von Neumann and Morgenstern's framework, assumes agents with fixed decision authority and standard preference structures satisfying axioms like completeness, transitivity, and the Archimedean property. However, hybrid Human–AI systems present a fundamental challenge: execution authority can alternate endogenously between Human and AI components within a digital environment. In such settings, coalition behavior cannot be represented by a single smooth preference ordering or a standard utility function. Existing hybrid approaches often blend utilities or treat authority as static, failing to capture the dynamics of alternating control where the executing agent is selected at each step based on policy divergence. This paper addresses the insufficiency of classical equilibrium concepts for hybrid coalitions operating under "Virtual Nature" (the algorithmic analogue of classical Nature) and seeks to formalize a framework where delegation is dynamic and regime-dependent.

2. Methodology: Neo-Game Theory

The authors introduce Neo-Game Theory, an extension of classical Game Theory designed for hybrid Human–AI coalitions. The framework is built on the following pillars:

2.1. Virtual Nature and Hybrid Games

The environment is modeled as Virtual Nature ($V$), a transition function $V: S \times A \to \Delta(S)$ that captures algorithmic stochasticity, hard thresholds, and discontinuities. A hybrid game is defined as a tuple $N(V) = (S, A, U, \lambda, P_V)$, where:

• $S$ is the state space.
• $A$ is the common executed action space (union of Human and AI proposal sets).
• $U = \{U_H, U_{AI}\}$ are individual utility functions.
• $\lambda_T \in \{0, 1\}$ is a binary delegation factor determining which agent executes the action at time $T$.
• $P_V$ is the transition kernel generated by Virtual Nature.

2.2. Axiomatic Departures

The framework relaxes classical axioms at the coalition level:

• Failure of Transitivity: Alternating control can induce non-transitive revealed preferences (e.g., $A \succ B, B \succ C, C \succ A$) even if individual agents are transitive.
• Failure of the Archimedean Axiom: The coalition does not exhibit smooth trade-offs; human adjustment is gradual while AI optimization can be discrete.
• Lexicographic Utility: Coalition outcomes are compared lexicographically. The execution regime (Human vs. AI) is selected before utility evaluation, meaning no compensatory trade-off exists between regimes.

2.3. Entropy-Based Delegation Mechanism

Delegation is determined by the Jensen–Shannon (JS) divergence ($D_{JS}$) between the empirical Human policy ($p_H$) and AI policy ($p_{AI}$). The framework utilizes two fixed thresholds, $\alpha_{agree}$ and $\alpha_{disagree}$, to partition the policy space into three regimes:

1. Agreement ($D_{JS} \le \alpha_{agree}$): AI executes ($\lambda = 0$).
2. Disagreement ($D_{JS} \ge \alpha_{disagree}$): Human executes ($\lambda = 1$).
3. Contextual ($\alpha_{agree} < D_{JS} < \alpha_{disagree}$): Execution is determined by a probabilistic rule where the probability of Human execution equals $D_{JS}$.

2.4. Learning Dynamics and Equilibrium

The paper focuses on Scenario 1: Human Arbitration, where the Human retains ultimate execution authority.

• Policy Updates: The AI updates its policy in all regimes toward the executed action. The Human updates their policy only in the agreement regime ($D_{JS} \le \alpha_{agree}$).
• Utility Traces: Utilities are updated online using an Exponentially Weighted Moving Average (EWMA) trace based on the executed action.
• Equilibrium Concept: The paper defines a Frequency-Convergence Equilibrium. Unlike classical Nash equilibria (fixed points of best responses), this equilibrium is a dynamic stabilization of the stochastic policy-adaptation process. It is characterized by the asymptotic stabilization of empirical regime frequencies ($f_{agree}, f_{ctx}, f_{disagree}$).

3. Key Contributions

1. Formalization of Neo-Game Theory: The paper establishes the axiomatic basis for hybrid coalitions, demonstrating how alternating authority leads to lexicographic utility structures and potential failures of classical transitivity.
2. Delegation via Entropy: It proposes a novel delegation rule based on the Jensen–Shannon divergence, utilizing thresholds to create distinct execution regimes (Agreement, Contextual, Disagreement).
3. Human Arbitration Regime: The authors develop and simulate the first regime of this theory, where the AI learns by observation and frequency matching while the Human acts as the final arbiter.
4. Theoretical Characterization: The paper proves that under Robbins–Monro step sizes or constant gains, the system converges to a stable distribution of policy frequencies. It establishes that the divergence process is asymptotically stationary and ergodic under specific conditions.

4. Results

The authors conducted simulations over various horizons ($H \in \{200, \dots, 50,000\}$) and parameter grids (thresholds, learning rates, step-size schedules, and state generation modes). Key findings include:

• Convergence: Across all horizons, the Human and AI policies progressively align, causing the Jensen–Shannon divergence ($D_{JS}$) to approach zero.
• Role of Parameters:
  • Learning Rate ($\eta$): Constant step sizes accelerate early alignment but induce small oscillations. Diminishing step sizes (Robbins–Monro) produce smoother, more stable convergence trajectories.
  • Thresholds: The thresholds determine the frequency of regime switches but do not alter the long-run alignment outcome.
  • State Generation: Whether states are generated exogenously (random) or endogenously (action-dependent) affects short-term volatility but not the asymptotic equilibrium.
• Transient vs. Asymptotic: Parameters primarily influence the transient learning path (speed and noise) rather than the existence of convergence itself. At sufficiently large horizons ($H \ge 30,000$), the system reaches a saturated regime where policies are aligned and $D_{JS} \approx 0$, regardless of the specific learning schedule or environmental specification.
• Stability: The system demonstrates robust policy alignment without relying on classical best-response optimization. Stability emerges from repeated execution, empirical adaptation, and the statistical structure of the delegation rule.

5. Significance and Claims

The paper claims to provide the first regime (Human Arbitration) of a broader Neo-Game Theory framework. Its significance lies in:

• Theoretical Foundation: It offers an axiomatic basis for hybrid systems where authority alternates, moving beyond models that simply aggregate utilities.
• Behavioral Equilibrium: It redefines equilibrium not as a static strategic fixed point, but as a dynamic stabilization of empirical frequencies and policy divergence.
• Practical Implementation: The simulation results validate that entropy-based delegation can robustly drive policy alignment in hybrid systems, suggesting a viable mechanism for human-in-the-loop training and trust-sensitive AI systems.

The authors explicitly state that this work lays the foundation for future extensions, specifically the AI-control and Negotiation regimes, which will allow for systematic comparisons with established solution concepts like Nash and Stackelberg equilibria in digitally mediated environments. The paper does not claim to solve all hybrid interaction problems but rather to formalize a specific class of interactions where execution authority is endogenous and policy-driven.