🤖 machine learning

Play Like Champions: Counterfactual Feedback Generation in Latent Space

This paper introduces "Latent Maps of Performance," a framework utilizing a Guided Variational Autoencoder trained on professional StarCraft II replays to generate actionable, multi-step counterfactual feedback trajectories that guide amateur players toward winning configurations by traversing a learned latent space of expert behavior.

Original authors: Andrzej Białecki, Adam Mastalerz, Han Zhou

Published 2026-07-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Andrzej Białecki, Adam Mastalerz, Han Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a professional StarCraft II player lose a match. You know they played well, but they still lost. Now, imagine an AI that doesn't just tell you who won, but acts like a time-traveling coach. It asks: "If you had made just a few different moves, could you have won?"

This paper introduces a system called Latent Maps of Performance that does exactly that. It uses advanced math to create a "map" of how games are played and then draws a new path for a losing player to follow that leads to a win.

Here is how it works, broken down into simple concepts:

1. The "Black Box" of Game Data

StarCraft II is a complex game with thousands of numbers changing every second (how much gold you have, how many units you built, how fast you are clicking).

The Analogy: Imagine trying to describe a whole movie by listing every single pixel that changes on the screen. It's impossible for a human to read.
What the paper did: The researchers took thousands of professional replays and fed them into a special AI (a Guided Variational Autoencoder). Think of this AI as a master chef who tastes a complex stew and writes down a simple recipe card with just the key ingredients (like "more gold," "faster army," "better timing"). This recipe card is the "latent space"—a compressed, simplified version of the game.

2. The "Championship Model"

In sports science, coaches often study how champions train to help regular athletes improve. They look at the champion's path and say, "If you do what they do, you might win too."

The Analogy: Imagine a GPS. Usually, a GPS tells you the fastest route from Point A to Point B. But in this paper, the GPS is special. It knows where "Winning" is located in the map. If you are currently at "Losing," the GPS doesn't just show you the road; it draws a new, hypothetical road that connects your current location to the "Winning" zone, showing you exactly which turns to take.
The Paper's Goal: Instead of asking "Who will win?", the system asks, "What specific changes would make this player win?"

3. Drawing the Path (The Four Strategies)

Once the AI has the map, it needs to draw a line from the "Losing" point to the "Winning" point. The paper tested four different ways to draw this line, like four different types of hikers trying to get to the summit:

Linear Interpolation (The Straight Line): This is like walking in a perfectly straight line from the bottom of the mountain to the top. It's simple, but sometimes the terrain in between is rocky or doesn't exist in reality (like walking through a lake).
Iterative Optimal Transport (The Smart Hiker): This method is smarter. Instead of aiming for one specific spot, it looks at the whole "Winning" crowd and constantly adjusts its path to stay on solid ground, moving toward the densest part of the winning area. It's like following a river that naturally flows uphill toward the summit.
Gradient Ascent (The Climber with a Compass): This method uses a compass that always points toward "higher win probability." It climbs step-by-step, trying to get the steepest path up. The paper found this method is very good at finding a win, but sometimes it takes a weird, jagged path that doesn't look like a real human game.
Neural Flow (The River Current): This is the most advanced method. The AI learns how the "water" of the game flows naturally from losing to winning states and then rides that current. It creates a very smooth, realistic path.

4. The Result: Actionable Feedback

Once the path is drawn, the system translates the "recipe card" back into real game advice.

The Output: It gives the player a ranked list of changes. For example: "If you had spent 10% more money on workers in the first 5 minutes, and built your army 2 minutes earlier, your chance of winning would have gone from 40% to 80%."
The Catch: The paper tested this on data from amateur players (people who didn't play in the tournaments used to train the AI). They found that while some methods (like the "Smart Hiker" and "River Current") worked very well and stayed on realistic paths, others (like the "Climber") sometimes took shortcuts that didn't make sense in the real game.

Summary

The paper claims to have built a bridge between AI that plays games and AI that teaches humans how to play better. By using a "latent map" of professional gameplay, they can generate "what-if" scenarios. They don't just say "You lost"; they say, "Here is the specific, step-by-step path you could have taken to win, based on how champions actually play."

The authors conclude that while the technology works, there is a trade-off: some methods find a win faster but take unrealistic paths, while others stay realistic but might take longer to find the win. They hope this work inspires future tools that help human players improve by learning from the "ghost" of a perfect game.

Technical Summary: Play Like Champions: Counterfactual Feedback Generation in Latent Space

1. Problem Statement

While recent advances in reinforcement learning (RL) have produced superhuman agents in competitive games like StarCraft II, the prevailing research focus has been on defeating human players rather than providing actionable feedback to improve them. Unlike chess and Go, where AI has become integral to player training by surfacing novel strategies, Real-Time Strategy (RTS) games lack principled frameworks for translating expert knowledge into specific, actionable guidance for human improvement. Existing tools often focus on broadcast analysis or high-level statistics rather than generating "algorithmic recourse"—specific, minimal changes a player can make to transition from a losing to a winning state. This paper addresses the gap between agent expertise and human skill acquisition by introducing a framework to generate counterfactual improvement trajectories within a learned latent space.

2. Methodology

Data and Preprocessing

The study utilizes the "SC2EGSet," a dataset of 23,476 professional StarCraft II tournament replays.

Feature Extraction: Each replay is converted into a 196-dimensional feature vector per player, capturing macro-economic gameplay structure. Features are segmented into three temporal windows (early, middle, late game), final economy states, economy differences, and a "supply capped percent" scalar.
Labeling: The match outcome (win/loss) from Player 0's perspective serves as the supervision signal for disentanglement.
Data Splits: The dataset is split into training (18,780), validation (2,347), and test (2,178) sets. To test generalizability, an Out-of-Distribution (OOD) set of 2,178 amateur replays (2016–2020) was sampled for evaluation.

Model Architecture: Guided Variational Autoencoder (VAE)

The core of the system is a modified Guided VAE designed to learn a compressed, continuous latent representation where specific dimensions are conditioned on the match outcome.

Structure: A symmetrical encoder-decoder MLP with ReLU activations.
Disentanglement: A subset of latent dimensions is supervised to predict the match outcome ( $y$ ), while an adversarial classifier is trained on the remaining "free" dimensions to prevent outcome information from leaking into them. This forces the model to separate "winning strategy" features from other gameplay variations.
Training: The model is trained using AdamW optimizers, jointly optimizing the VAE reconstruction loss, the supervised classifier loss, and an adversarial loss to enforce disentanglement.

Latent Space Traversal Strategies

Once a player's game state is encoded into a latent vector ( $z_{start}$ ), the system generates a path to a "winning region" in the latent space. Four distinct traversal strategies were implemented and compared:

Linear Interpolation (LERP): A straight-line trajectory between the starting vector and a target winning vector (either the centroid of all winning vectors or the nearest $k$ neighbors).
Iterative Optimal Transport (OT): A dynamic approach that treats the winning distribution as a target mass. At each step, it calculates an optimal transport plan to find a local barycentric target and takes an Euler step toward it, mimicking a vector field flow.
Density-Regularized Gradient Ascent: An optimization approach that maximizes the predicted win probability while penalizing deviations from the data manifold (using Gaussian Kernel Density Estimation) to ensure generated paths remain realistic.
Neural Flow Matching: A continuous normalizing flow approach where a neural network learns a global velocity field mapping the "losing" distribution to the "winning" distribution. Trajectories are generated by integrating this velocity field.

Feedback Generation

The generated latent path is decoded back into the original feature space. Feedback is extracted as a ranked list of feature changes, calculated via:

Raw change from start to end.
Minimum viable change (stopping at the first point where win probability $> 0.5$ ).
Win-probability-weighted changes.

3. Key Results

Model Performance

The Guided VAE demonstrated high fidelity in reconstruction and classification:

Reconstruction: Low Mean Squared Error (MSE) on both in-distribution and OOD data.
Classification: The supervised latent dimension achieved 98.76% accuracy and a 0.9991 ROC-AUC on the test set, indicating robust separation of winning and losing profiles.

Traversal Strategy Comparison

The study evaluated the four strategies on both the professional dataset and the amateur OOD dataset. Key findings include:

Success Rate: Gradient Ascent achieved the highest success rate (nearly 100% on both datasets) in reaching a win probability $> 0.5$ . However, analysis suggests this is due to "adversarial off-manifold drift," where the path exploits classifier blind spots rather than following realistic gameplay evolution.
Efficiency and Smoothness: Optimal Transport (OT) demonstrated superior efficiency, reaching the winning state much earlier in the path (Crossover $\alpha \approx 0.12$ ) compared to Linear methods ( $\alpha \approx 0.65$ ). OT also maintained near-perfect monotonicity ( $> 0.99$ ), ensuring win probability consistently increased along the path.
Robustness: Linear methods (especially k-NN) showed significant fragility under distributional shift, with success rates dropping by over 30% on OOD data. Neural Flow and OT maintained high monotonicity and stability on OOD data.
Trade-offs: The results highlight a trade-off between raw success rate and path quality. While Gradient Ascent "wins" the most often, its paths are erratic and less interpretable. OT and Neural Flow provide smoother, more grounded trajectories that better respect the underlying data manifold.

4. Contributions and Significance

The paper claims to bridge the critical gap between representational learning and actionable human feedback in esports.

Framework for Counterfactuals: It introduces "Latent Maps of Performance," a framework that models player improvement as an algorithmic recourse problem within a learned representation space.
Shift in Paradigm: The work shifts the analytical focus from "who will win" (predictive modeling) to "what the player should do differently" (prescriptive modeling).
Validation of Traversal Methods: By systematically comparing linear, transport-based, and flow-based strategies, the paper identifies that simple interpolation is insufficient for complex RTS games, while Optimal Transport offers a robust balance of efficiency and interpretability.
Foundation for Future Tools: The authors conclude that their solution provides a computationally feasible foundation for developing practical feedback systems, moving beyond passive analysis to active, data-driven coaching.

5. Limitations

The authors acknowledge several constraints:

Temporal Scope: The model is trained on multi-year data without explicit accounting for game patches or meta-shifts.
Granularity: The feedback is based on macro-economic features and does not provide micro-level tactical advice (e.g., specific unit control).
Race Agnostic: The model does not encode player race information, limiting race-specific advice.
Human Validation: The system has not been validated with human participants in a "human-in-the-loop" experiment; evaluation was purely algorithmic.

In summary, the paper demonstrates that it is possible to extract actionable, counterfactual improvement trajectories from expert gameplay data, offering a new pathway for AI-assisted skill development in complex RTS environments.