Technical Summary: RuleSmith – Multi-Agent LLMs for Automated Game Balancing

Problem Statement

Balancing asymmetric strategy games is a persistent challenge in game design and multi-agent learning. Traditional approaches rely on human experts iterating through cycles of manual tuning, heuristic adjustments, and subjective playtesting. This process is slow, expensive, and difficult to scale, particularly as modern games feature combinatorial action spaces, long-horizon objectives, and richly parameterized rule systems. Furthermore, the problem extends beyond entertainment to domains like economic simulations, policy design, and cybersecurity, where assessing how small parameter changes propagate through multi-step interactions is critical. While Large Language Models (LLMs) have demonstrated the ability to act as "zero-shot" simulators for multi-agent systems, leveraging them to optimize the rules of those environments remains largely unexplored.

Methodology

The authors introduce RuleSmith, a framework that automates game balancing by coupling a game engine, multi-agent LLM self-play, and Bayesian optimization over a multi-dimensional rule space.

1. The Testbed: CivMini

To validate the framework, the authors constructed CivMini, a simplified, parameterized, turn-based asymmetric strategy game inspired by 4X mechanics.

• Factions: Two asymmetric factions, Empire and Nomads.
  • Empire: Specialized economy with distinct Farmer (resource gathering only) and Soldier (combat only) units.
  • Nomads: Versatile Cavalry units with higher mobility that gain resources by killing enemy units, necessitating an aggressive playstyle.
• Parameters: The game exposes 12 tunable parameters ($\theta$) governing economy (initial resources, gathering efficiency), combat (damage, HP), production (unit costs), and scoring (weights for resources, battles, surviving units).
• Objective: Optimize $\theta$ to minimize a balance loss function $L(\theta) = |w_E - 0.5| + |w_N - 0.5| + 0.5 \cdot w_D$, where $w_E$ and $w_N$ are win rates and $w_D$ is the draw rate.

2. LLM Self-Play as Evaluator

RuleSmith utilizes two LLM agents (one per faction) to play the game based on natural-language rulebooks and structured game states.

• Input: Agents receive a turn index, faction summaries, enemy positions, strategy guides, and a list of legal actions.
• Output: Agents generate a structured JSON object containing simultaneous actions for all units.
• Reliability Mechanisms:
  • RAG (Retrieval-Augmented Generation): A lightweight system retrieves relevant rules from the rulebook based on game context to reduce hallucinations.
  • Structured Output: Enforcing JSON output with explicit examples reduces parsing errors and illegal move detection burdens.
• Evaluation: For a given parameter set $\theta$, $N$ self-play games are run to estimate empirical win rates and balance metrics.

3. Bayesian Optimization with Adaptive Sampling

Directly searching the discrete rule space is intractable due to combinatorial explosion. RuleSmith employs Bayesian Optimization (BO) over a continuous relaxation of the rule space.

• Surrogate Model: A Gaussian Process models the balance loss $L(\theta)$.
• Discrete Projection: Continuous candidates proposed by the optimizer are deterministically projected to valid discrete game configurations (e.g., rounding HP to integers).
• Acquisition-Based Adaptive Sampling: To address the high computational cost and noise of LLM evaluations, the framework dynamically allocates the evaluation budget ($N_t$).
  • Candidates with high Expected Improvement (EI) (promising points) receive more games ($N_{max}$) for accurate assessment.
  • Exploratory candidates with low EI receive fewer games ($N_{min}$).
  • This strategy concentrates resources on critical configurations while maintaining efficient exploration.

Key Contributions

1. Executable Zero-Shot Self-Play: Demonstrated that multi-agent LLMs can perform zero-shot self-play in an executable, asymmetric strategy game using only natural-language rulebooks and structured states, producing legal and verifiable actions without training.
2. Automated Balancing Pipeline: Presented a general framework integrating multi-agent LLM self-play with Bayesian optimization and acquisition-based adaptive sampling. This pipeline automatically adjusts rule parameters to achieve balanced outcomes, improving sample efficiency by allocating more budget to promising candidates.
3. Comprehensive Empirical Validation: Validated RuleSmith on CivMini across different model sizes (2B and 8B parameters) and faction configurations. The system consistently achieved near-balanced outcomes (win rates within $50\% \pm 5\%$) and demonstrated that balanced parameters transfer across evaluation settings when model capacities match.

Experimental Results

• Convergence: RuleSmith successfully converged to highly balanced configurations, reducing win-rate disparities to 0% even from intentionally imbalanced initializations.
• Model Capacity Effects: Experiments showed that increasing the model size of one faction shifts the win distribution in its favor. Notably, performance gaps were most significant when a larger model was evaluated against a smaller counterpart using parameters optimized for a smaller model, highlighting the "smarter" agent's ability to exploit strategic advantages.
• Ablation Studies:
  • Optimization Methods: Compared against Random Search and (1+1)-Evolution Strategy, RuleSmith's Bayesian Optimization with adaptive sampling was the only method to consistently converge to near-equal win rates (51%|49%). Fixed-sampling BO and other baselines failed to achieve balance.
  • Game Designs: The framework maintained balanced outcomes across varying map sizes (5×5 to 11×11) and turn limits, demonstrating robustness to spatial and temporal configuration changes.
• Interpretability: The discovered parameters provided interpretable insights into how health scaling, resource efficiency, and production tempo jointly determine fairness. The system found diverse parameterizations achieving balance, rather than converging to a single canonical setting.

Significance and Claims

The paper claims that RuleSmith represents a shift from using LLMs merely as playtesting tools to using them as effective mechanisms for optimizing complex, rule-governed multi-agent environments. By treating the game itself as a parameterized asymmetric environment and directly optimizing the rule space, the framework offers a scalable and interpretable approach to balancing.

The authors posit that this paradigm has broader applicability beyond game design to domains such as policy design, economic modeling, cybersecurity, and medical decision-making, where rule-based asymmetric interactions are the norm. They emphasize that the framework is designed as an offline analysis and design-time tool, intended to support safer, more transparent, and systematic design of rule-based systems, rather than serving as a real-time decision execution system. The work acknowledges limitations, noting that LLM self-play in simplified environments may not fully capture human behavior or provide formal guarantees under distribution shifts.