🤖 AI

Monte Carlo Pass Search: Using Trajectory Generation for 3D Counterfactual Pass Evaluation in Football

This paper introduces Monte Carlo Pass Search (MCPS), a novel framework that leverages a 3D ball-conditioned world model and a value model to simulate pass execution variants and evaluate their counterfactual outcomes, thereby enabling distribution-aware attribution and ranking of football passes using high-fidelity Bundesliga tracking data.

Original authors: Andrew Kang, Priya Narasimhan

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Andrew Kang, Priya Narasimhan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a football (soccer) game. A player kicks the ball to a teammate. In the past, analysts would look at the result: Did the teammate catch it? Did they score? If yes, it was a "good" pass. If no, it was a "bad" pass.

But this paper argues that judging a pass only by its result is like judging a chef only by whether the customer ate the meal. It ignores whether the chef actually cooked a masterpiece that just happened to get dropped, or if they served a burnt meal that the customer ate anyway.

The authors, Andrew Kang and Priya Narasimhan from Carnegie Mellon University, built a new system called Monte Carlo Pass Search (MCPS). Think of it as a "What-If Simulator" for football passes.

Here is how it works, broken down into simple parts:

1. The Problem: The "Butterfly Effect" of Football

In football, two passes can look identical but have very different outcomes.

Pass A: A perfect, slightly risky pass that hits a teammate's foot perfectly. Great outcome.
Pass B: The exact same pass, but the teammate slips slightly. Bad outcome.

Traditional stats treat these as different. This system asks: How fragile was that pass? If the player had kicked it 2% harder or 1 degree to the left, would it still have worked?

2. The Solution: Running a "Simulation Race"

Instead of looking at just one pass, MCPS runs thousands of "what-if" scenarios for every single kick. It's like a video game that simulates the next few seconds of the game over and over again, but with tiny changes each time.

The system has three main parts, like a team of experts:

The "Kick Detective" (Policy Model):
First, it figures out exactly how the player kicked the ball. Since cameras aren't perfect, it uses math to guess the speed, spin, and angle of the kick.
- Analogy: Imagine a detective trying to reconstruct exactly how a car hit a wall just by looking at the skid marks.
The "Crystal Ball" (World Model):
Once it knows how the ball was kicked, it predicts what happens next. It simulates how other players will run, where the ball will bounce, and who will touch it next.
- Analogy: This is like a weather forecaster, but instead of rain, it predicts player movements. It uses a special "token" system (like words in a sentence) to predict the future step-by-step, which helps it work well even with limited data.
The "Scorekeeper" (Value Model):
After simulating the future, it asks: "Did this new scenario help the team get closer to scoring?" It gives the scenario a score.
- Analogy: It's like a coach watching the simulation and shouting, "Good move!" or "That was a mistake!"

3. The Two Types of Simulations

The system runs two different kinds of "races" to evaluate a player:

Race 1: The "Execution" Test (Local Search)
The system takes the exact same pass idea but changes the kick slightly (a little faster, a little to the left).
- Goal: To see if the pass was fragile. If a tiny mistake ruins the play, the pass was risky. If the play still works even with a bad kick, the pass was robust.
- Result: This tells us if a player is a "lucky" kicker or a "consistent" kicker.
Race 2: The "Opportunity" Test (Global Search)
The system changes the destination of the pass entirely. It asks, "Could the player have kicked the ball to a different teammate for a better result?"
- Goal: To see if the player made the best choice.
- Result: This tells us if a player is a "smart" decision-maker who sees the best options.

4. What They Found

Using real data from the German Bundesliga (a top football league), they tested this on a specific match.

They found that some passes looked great but were actually very "narrow"—meaning they only worked if everything went perfectly.
They found that some players consistently made passes that were robust (hard to mess up) or chose options that were much better than the average alternative.
They proved that even with a small amount of public data (only 7 matches), their "token-based" simulation method was more accurate at predicting player movements than other existing methods.

Summary

This paper doesn't just say "Pass X was good." It says:

How good was the decision? (Did they pick the best teammate?)
How good was the execution? (Did they kick it perfectly, or did they get lucky?)
How risky was it? (Would a tiny mistake have ruined it?)

By simulating thousands of "what-if" futures, they turn a single moment of football into a rich distribution of possibilities, helping coaches and scouts understand the true quality of a player's passing, not just the result.

Technical Summary: Monte Carlo Pass Search (MCPS)

Problem Statement

Football (soccer) analytics has traditionally relied on proprietary datasets and point-estimate metrics (e.g., expected threat, expected pass) that evaluate a single realized outcome. This approach fails to disentangle three critical components of a passing decision:

Option Quality: Whether the passer chose a good target.
Execution Fragility: Whether the chosen option had a narrow "success window" (i.e., was highly sensitive to small errors in kick parameters).
Execution Quality: Whether the actual kick was better or worse than the typical execution for that specific attempt.

Existing models often treat passes as deterministic events or rely on rule-based physics simulations that cannot capture complex, game-contextual off-ball movements. The lack of public, high-fidelity 3D tracking data has further hindered the development of reproducible, distribution-aware evaluation frameworks.

Methodology: Monte Carlo Pass Search (MCPS)

The authors recast pass evaluation as a Monte Carlo Tree Search (MCTS)-like problem, replacing hand-designed simulators with learned generative models. The framework operates in raw spatiotemporal space and consists of three core components:

1. The Value Model (Possession Value)

A Transformer-based model assigns a scalar Possession Value (PV) to any game state.

Definition: $PV = P(\text{Ego scores within 10s}) - P(\text{Opponent scores within 10s})$ .
Training: Trained on tracking windows with binary cross-entropy heads, using shots within 10 seconds as labels (weighted by expected goals to mitigate sparsity).
Role: Provides the reward signal for evaluating the outcome of simulated rollouts.

2. The World Model (Trajectory Generation)

A generative model predicts short-horizon futures (player and ball trajectories) conditioned on a candidate ball flight. To ensure sample efficiency on limited public data, the authors adapt SMART, an autoregressive discrete-token model from autonomous driving.

Architecture: A decoder-only space-time Transformer using discrete tokens.
- Tokens: Player motion (10D planar velocities) and ball motion (15D 3D velocities) are quantized into vocabularies via k-means.
- Conditioning: The model conditions on 8 history tokens and predicts 24 rollout tokens.
Interaction Modules:
- Player-to-Touch: Predicts the next player to touch the ball and the time-to-touch using a survival-style objective.
- Ball-at-Touch: Predicts post-touch ball velocity using a diagonal-Gaussian regressor.
Rollout: The model generates stochastic multi-agent futures until the first meaningful ball interaction (touch, deflection, or out-of-bounds).

3. The Policy Model (Search over Pass Variants)

For each observed pass, MCPS infers the kick parameters (initial 3D velocity and spin) using a CEM-style solver coupled with a physics simulator. It then constructs two counterfactual action sets:

Local Search: Samples 256 variants by applying small perturbations to the inferred kick parameters (modeling execution noise).
Global Search: Samples 256 variants with larger perturbations to explore alternative passing options (different targets/directions) from the same pre-pass state.

Evaluation Framework

Each candidate pass (observed or counterfactual) is rolled out through the World Model and scored by the Value Model to obtain a distribution of gained Possession Value ( $\Delta PV$ ). Two distribution-aware metrics are computed for attribution:

Mean-Difference Surplus ( $S_{mean}$ ): The difference between the observed pass's $\Delta PV$ and the mean $\Delta PV$ of the counterfactual set.
Percentile Surplus ( $S_{pct}$ ): The percentile rank of the observed pass's $\Delta PV$ within the counterfactual distribution.

Key Contributions

Trajectory-Conditioned Rollout Evaluation: A framework that generates diverse short-horizon futures conditioned on candidate ball trajectories, moving beyond point estimates to full distributions.
Dual Monte Carlo Search: A mechanism that separates execution sensitivity (local search) from opportunity set approximation (global search).
Distribution-Aware Metrics: Introduction of $S_{mean}$ and $S_{pct}$ scores that allow for nuanced analysis of pass robustness and decision quality.
Adaptation of Autoregressive Models: Demonstration that discrete-token, autoregressive trajectory generators (adapted from SMART) yield strong best-of-20 forecasting accuracy on limited public football data, enabling fully hypothetical rollouts.

Results

The framework was evaluated on a public Bundesliga dataset (7 matches, 25 Hz tracking, 3D ball coordinates).

Trajectory Forecasting: The adapted SMART model achieved a Best-of-20 minADE of 2.4 and minFDE of 4.7, outperforming baselines including a static model, constant velocity, a naive Transformer, and the public Sports-Traj checkpoint (minADE 4.2).
Player-to-Touch: The hybrid model (Trajectory + Touch prediction) achieved a Receiver Top-1 Accuracy of 0.605. The authors note this is a conservative lower bound due to the objective mismatch (training on first-touch hazard vs. inference on hypothetical futures) and rollout uncertainty.
Case Study (Bochum vs. Leverkusen):
- Execution Sensitivity: Analysis of a specific pass by Florian Wirtz showed a non-trivial spread in $\Delta PV$ even under small local perturbations, highlighting that some passes are valuable only under near-perfect execution.
- Player Rankings: Aggregating metrics across a match allowed for ranking players based on:
  - Local Search: Consistency relative to execution difficulty (high percentile surplus indicates robust execution).
  - Global Search: Systematic selection of high-value options (high mean-difference surplus indicates choosing better targets than the average alternative).

Significance and Claims

The paper claims to establish a new baseline for football analytics that reconnects the field with the broader Computer Vision community by utilizing public, high-fidelity data.

From Point Estimates to Distributions: MCPS explicitly models the risk and fragility of passes, distinguishing between a "good decision with bad execution" and a "bad decision with good execution."
Disentanglement of Factors: By separating local (execution) and global (option) searches, the framework provides a more granular view of player performance than existing metrics.
Feasibility on Public Data: The work demonstrates that high-fidelity, distribution-aware evaluation is possible even with small public datasets (7 matches) by leveraging efficient, token-based generative modeling techniques adapted from autonomous driving.

The authors acknowledge limitations, including the small scale of training data, potential noise in tracking synchronization, and the simplification of search distributions. They suggest future work should focus on multi-interaction horizons, better modeling of rare interactions (e.g., aerial duels), and more realistic proposal policies for global search.