🤖 AI

FootsiesGym: A Fighting Game Benchmark for Two-Player Zero-Sum Imperfect-Information Games

This paper introduces FootsiesGym, an open-source, vectorized benchmark environment based on the minimalist fighting game Footsies, designed to facilitate efficient research and training of reinforcement learning algorithms for two-player zero-sum imperfect-information games.

Original authors: Chase McDonald, Nathan Tsang, Wesley N. Kerr

Published 2026-07-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chase McDonald, Nathan Tsang, Wesley N. Kerr

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you want to teach a robot how to be a master fighter. You have two bad options for training it:

The Chess Board: It's too simple. The rules are clear, the moves are short, and there's no real "feeling" of a fight. It's like teaching someone to swim by having them walk on a treadmill.
The Real Olympics: It's too hard. Games like Street Fighter or Dota 2 are so massive and complex that you'd need a supercomputer just to run a single practice session. It's like trying to learn to swim by jumping into the middle of a hurricane.

FootsiesGym is the "Goldilocks" solution. It's a new, open-source training ground built on a tiny, stripped-down fighting game called Footsies. It's simple enough to run on a normal computer, but complex enough to teach a robot the deep, tricky psychology of fighting.

Here is how the paper breaks it down, using some everyday metaphors:

1. The Game: A Game of "Rock, Paper, Scissors" with a Twist

In a real fighting game, there are long combos and flashy moves. In Footsies, they stripped all that away to focus on the Neutral Game.

Think of the Neutral Game as the moment two boxers circle each other, waiting for an opening. It's a constant game of Rock, Paper, Scissors:

If you attack, I can block.
If I block, you can move in closer.
If I move in, you can dodge.

There is no single "best move." If you always punch, you get blocked. If you always run, you get hit. You have to mix your moves randomly, like a poker player bluffing, so your opponent can't predict you. This is called a "non-transitive" strategy (where A beats B, B beats C, but C beats A).

2. The Training Gym: A High-Speed Simulator

The researchers built a "vectorized simulator." Imagine a regular video game is like a single person running on a treadmill. FootsiesGym is like a massive gym with 128 treadmills running at once, all controlled by one brain.

Speed: It runs so fast that a standard computer can simulate 52,500 game steps every second. That's like playing a million matches in the time it takes to drink a cup of coffee.
The "Blind" Factor: Just like in real life, you can't see your opponent's thoughts. The AI only sees what's on the screen, not what the other player is planning. This makes it a true test of prediction, not just reaction.

3. The "Secret Move" Problem

One of the most interesting findings in the paper is about a "Special Attack." In this game, you can charge up a powerful move by holding a button for a specific amount of time (60 frames).

The Analogy: Imagine trying to teach a dog to bark on command. If you just throw treats randomly, the dog might accidentally bark once by chance, but it won't understand the pattern.
The Result: The researchers found that standard AI training methods were terrible at discovering this special move. Because the move requires a very specific sequence of actions (hold for 60 frames, then release), random exploration is like finding a needle in a haystack. The AI mostly ignored it, sticking to simple punches and blocks. This shows that even in a "simple" game, some smart strategies are too hard for standard AI to find on its own.

4. The Results: Smart but Boring?

The researchers tested several AI algorithms to see who learned best.

The Winners: The AIs got really good at beating "dumb" opponents (like a robot that just stands still). They learned to win about 90% of the time.
The Catch: As the AIs got smarter, they became too cautious. They learned that the safest way to win was to just wait and react, rather than taking risks or using the cool special moves. They became "boring" fighters who never initiated an attack.
The Lesson: Being the "best" at winning doesn't always mean being the most fun or engaging opponent. The paper suggests we need new ways to teach AI to be not just strong, but also proactive and interesting.

Why This Matters

The paper argues that FootsiesGym is a perfect "test lab" for researchers. It's fast enough to run thousands of experiments, but complex enough to show us where current AI methods fail (like discovering hidden strategies or balancing aggression with defense). It bridges the gap between simple math puzzles and the chaotic reality of real video games.

In short: FootsiesGym is a tiny, fast, open-source fighting arena designed to help us figure out how to teach AI to think like a human fighter, not just a robot that follows a script.

Technical Summary: FootsiesGym

Problem Statement

Progress in multi-agent and game-theoretic reinforcement learning (RL) is currently hindered by a lack of benchmarks that balance structural complexity with computational tractability. Existing environments tend to fall into two extremes:

Small-scale theoretical benchmarks (e.g., Kuhn Poker, Goofspiel): These capture cyclic, mixed-strategy structures and allow for exact exploitability calculations but lack the long horizons, spatial dynamics, and exploration requirements of richer games.
Large-scale real-time environments (e.g., StarCraft II, Dota 2): These possess deep complexity and long time horizons but require substantial computational resources and time for training, making rapid iteration and analysis difficult.

Fighting games represent a potential "middle ground" due to their real-time, spatial nature and reliance on non-transitive strategic interactions (often described as a "rock-paper-scissors" dynamic). However, standard fighting games introduce excessive complexity, such as long combo sequences, which shift the strategic focus from cyclic neutral play to transitive execution contests. There is a need for an environment that isolates the neutral game—the phase where neither player has an advantage and strategic maneuvering occurs—while remaining simple enough for efficient analysis and training on standard hardware.

Methodology

The authors introduce FootsiesGym, an open-source environment built on HiFight's minimalist 2D fighting game, Footsies. The environment is designed to isolate the cyclic, non-transitive strategic interactions of fighting game neutral play.

Environment Design

Core Mechanics: The game removes health bars and combos. A round is lost only when a player is hit by a special attack. Players can block up to three times before suffering a "guard break."
Action Space: The default action space has size $|A|=6$ (NONE, BACK, FORWARD, ATTACK, BACK+ATTACK, FORWARD+ATTACK). A "special-charge" mode expands this to $|A|=9$ by adding discrete actions to toggle the charging state required for special moves, addressing the difficulty of discovering these moves through random exploration.
Observations: Agents receive an 85-dimensional (or 88-dimensional with special-charge) vectorized observation describing the state of both players. Crucially, agents observe privileged self-features (e.g., dash availability, charge progress) not exposed to the opponent, mirroring information asymmetry.
Rewards: The game is zero-sum ( $r_i = -r_{-i}$ ). A terminal win/loss yields $\pm 10$ reward; timeouts yield 0. Dense rewards are optional.
Action Delay: The environment supports configurable action delays (frames between selection and execution). This parameter controls the balance between reactive play (low delay) and predictive play (high delay), preventing degenerate purely reactive strategies.

Implementation

FootsiesGym utilizes a vectorized simulator built in C# within the Unity engine, decoupled from the rendering loop for headless training.

Vectorization: A single headless process steps $K$ independent game instances in parallel, exposed via a gRPC port.
Throughput: On a standard 24-core workstation, the system achieves up to ~52,500 environment steps per second (with 4 concurrent processes and 128 in-process environments), enabling high-throughput training accessible on standard hardware.
API: Python clients interact via the PettingZoo API.

Experimental Baselines

The authors trained several algorithms to benchmark the environment, noting that these are naively tuned and not definitive implementations:

PPO (Proximal Policy Optimization): Two variants were tested—one with a fixed entropy coefficient and one with an annealing schedule (PPO Sched.).
EMAgnet: A policy gradient method that regularizes towards an exponential moving average of its own weights.
PFSP (Prioritized Fictitious Self-Play): A population-based approach that maintains a reservoir of past checkpoints and preferentially samples opponents the current policy loses to.

All agents used a 2-layer MLP over the observation space and were trained for $2.5 \times 10^7$ steps across five random seeds.

Key Results

Performance Against Heuristics

Against uniform-random and "no-op" (non-attacking) opponents, all policies achieved win rates of 85–95%. However, as policies improved, they became increasingly reactive, rarely initiating engagement with stationary opponents. This suggests a potential misalignment between maximizing win rates and creating engaging AI behavior.

Approximate Exploitability

Since exact exploitability is intractable, the authors trained approximate best responses (using PPO) against the fixed checkpoints of the baseline policies.

PFSP demonstrated the highest robustness, achieving a positive return against all other algorithms and the lowest loss rate against best responses.
EMAgnet showed a disconnect: it achieved positive head-to-head returns against PPO variants but was the most consistently exploited by dedicated best responses.
PPO Variants: Performed approximately equivalently to each other in terms of exploitability.

Discovery of Special Attacks

A critical finding was the difficulty of discovering special attacks through random exploration. In the standard configuration, executing a directional special attack requires a specific sequence of 15 consecutive steps, a sequence with a $\approx 0.001\%$ probability under a uniform random policy.

Across all algorithms, the average usage of special attacks remained near zero throughout training.
The fixed-entropy PPO policy briefly learned the move before abandoning it, suggesting that high entropy regularization (often beneficial in small benchmarks) may prevent the discovery of complex, high-value strategies in this environment.

Significance and Contributions

The paper positions FootsiesGym as a productive middle ground for research in multi-agent RL and game theory. Its significance lies in several areas:

Isolation of Neutral Play: By stripping away combos and health bars, the environment isolates the cyclic, mixed-strategy dynamics inherent to fighting game neutral play, making it suitable for studying non-transitive interactions.
Accessibility and Reproducibility: Unlike platforms wrapping proprietary games (e.g., FightingICE, DIAMBRA), FootsiesGym is fully open-source with a vectorized simulator, allowing for high-throughput training on standard workstations without licensing friction.
New Research Directions:
- Game-Theoretic Training: The environment serves as a testbed for population-based training (PSRO, PFSP) and regularization methods, specifically challenging them to retain valuable but hard-to-discover strategies (like special attacks) that standard regularization might suppress.
- Play-Style vs. Skill: The results highlight a gap between raw skill (exploitability minimization) and "engaging" play. Policies that maximize win rates may become overly passive, failing to utilize core mechanics or engage opponents proactively.
- Approximate Exploitability: The environment offers a balanced complexity for testing approximate methods (like best-response training) where exact calculation is impossible but the scale is manageable.
- Human Evaluation: The ability to export to WebGL facilitates human-vs-AI experiments and imitation learning data collection to evaluate whether AI behavior is perceived as fun or engaging.

The authors conclude that while FootsiesGym is more complex than small-scale benchmarks and less complex than commercial fighters, it provides a unique, resource-efficient platform to address open questions regarding strategy discovery, the trade-off between skill and engagement, and the scalability of exploitability estimation.