🤖 machine learning

Training-Free Off-Screen Player Imputation for Broadcast-Based Spatial Football Analytics

This paper introduces a training-free, online method called role-anchored centroid voting to impute off-screen player positions from broadcast footage, significantly reducing spatial football metric errors and improving downstream analytics like possession-quality scores without requiring additional training data. For the two real World Cup windows, the imputed scores are not claimed to be 'true' values, but rather represent a sensitivity result demonstrating how verdicts can change when off-screen players are imputed, given the lack of full-pitch ground truth on broadcast clips.

Original authors: Seongjin Choi

Published 2026-07-14✓ Author reviewed ⓘ

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Seongjin Choi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a football match on TV. The camera follows the ball, zooming and panning to keep the action in focus. But here's the catch: the screen is too small to show everyone. At any given moment, you can only see about 10 to 16 of the 22 players on the field. The other 6 to 12 players are "off-screen," invisible to your eyes.

For years, computer programs trying to analyze the game (like calculating who controls the pitch) have simply ignored these invisible players. They acted as if the missing teammates didn't exist. This paper argues that this is a huge mistake. It's like trying to guess the outcome of a tug-of-war by only looking at the people on one side of the rope. If you ignore the invisible team, your math is wrong, and you might think a team is in control when they are actually losing the battle.

The Big Problem: The "Invisible" Distortion
The authors set up a simulation to measure just how bad this "invisible player" problem is. They took perfect, full-field data (where they could see all 22 players) and then artificially hid the ones the TV camera would miss.

When they ignored the off-screen players, the error in their "pitch control" calculations skyrocketed. The mistake grew by 25.1 to 26.9 percentage points. That is a massive gap. It means the computer thought a team controlled a huge chunk of the field, when in reality, the invisible players were holding that space. The error in calculating how much of the field a team "owns" jumped by 11.1 to 13.4 points.

The Solution: A "Ghost" Team That Doesn't Need Training
The paper introduces a clever, "training-free" way to guess where the missing players are. Think of it like a game of "follow the leader" with a twist.

Usually, if you can't see a player, you might guess they stayed where they were last seen. But the authors found a better way: Role-Anchored Centroid Voting.

Here is how it works in plain English:

The Team Huddle: Every player on a team has a specific "job" or role relative to the center of their team. A defender stays near the back; a striker stays near the front.
The Vote: The players you can see on the screen vote on where the center of the whole team is. They do this by saying, "I am here, and I am usually this far from the center, so the center must be there."
The Ghosts: Once the computer figures out where the team's center is, it uses the known "job distances" to place "ghosts" (imaginary players) in the right spots for the off-screen teammates.

This method doesn't need a supercomputer, it doesn't need to learn from thousands of past games, and it doesn't need to see the future. It just uses the players currently on screen to guess where the others are.

How Well Did It Work?
In their simulations, this "voting" method was a game-changer:

It cut the error in hidden-zone control roughly in half (dropping from ~26 points down to 12.2–13.8 points).
It reduced the error in team ownership calculations to just 28–48% of what the "ignore" method produced.
For the players who were hidden for a short time (less than 9.6 seconds), the guess was surprisingly accurate, with a median error of only 3.3 to 8.9 meters.

However, the paper is honest about its limits. When players are hidden for a long time (more than 9.6 seconds), the guesses get fuzzier, with errors growing to 15.6–16.9 meters. The authors note that while this method is great for real-time TV analysis, a "learning" computer that has seen many past games might do even better if it had time to study and could peek at future frames. But for a live broadcast where you can't wait or train, this "ghost" method is the best simple tool available.

Why It Matters: Changing the Verdict
The authors tested this on real World Cup broadcast footage. They looked at two specific moments in a match between the Netherlands and Morocco. Because there is no full-pitch ground truth available for these broadcast clips, the imputed scores are not claimed to be 'true' values. Instead, they serve as a sensitivity result, demonstrating how verdicts can change when off-screen players are imputed.

In one moment, the "visible-only" view said the team was creating space with a score of +15.6. With the "ghost" players added, the score jumped to +32.8.
In another moment, the visible view said the play was a "dead possession" (a score of -10.9). But once the invisible players were accounted for, the score flipped to +4.7, changing the verdict from "dead" to "weak progression."

This proves that ignoring off-screen players doesn't just add a tiny bit of noise; it can completely flip the story of the game. By adding these "ghosts," the analysis becomes much fairer, giving credit to the players who are doing the work even when the camera isn't looking at them.

What the Paper Rules Out
The authors are very clear about what this method is not.

It is not a magic fix that replaces all advanced AI. They admit that if you have training data and can see the future (like in a video game replay), a "learned" AI model is still better.
It is not perfect for long, hidden periods. The method struggles when a player is off-screen for more than 9.6 seconds, which happens in about 50–57% of hidden-player moments in their data.
It is not a replacement for perfect identity tracking. In real life, if the camera loses a player's track or can't read their jersey number, the "ghost" might get the wrong person. The paper suggests that better identity tracking is the next step.

The Bottom Line
This paper shows that the "invisible" players in football broadcasts are a massive blind spot for current analysis. By using a simple, smart voting system to guess where they are, we can fix a huge chunk of the error without needing expensive training or supercomputers. It turns a biased, incomplete view of the game into a much more accurate one, proving that sometimes, you have to imagine the missing pieces to see the whole picture.

Technical Summary: Training-Free Off-Screen Player Imputation for Broadcast-Based Spatial Football Analytics

Problem Statement
Spatial football analytics, such as pitch control, fundamentally rely on the positions of all 22 players. However, the primary source of positional data for broadcast analysis—the main camera view—is inherently partial. Due to panning and zooming to follow the ball, a typical broadcast frame reveals only 10–16 of the 22 players. Current Game State Reconstruction (GSR) pipelines often ignore these off-screen players, treating them as absent. This "visible-only" approach introduces a systematic bias: teams with defenders off-screen appear to cede control of the hidden half of the pitch, not due to tactical failure, but due to camera framing. While this distortion is acknowledged, it has rarely been quantified on open data, and existing learned solutions (e.g., DeepMind's Graph Imputer) rely on proprietary data, future observations (bidirectional inference), and offline training, rendering them inapplicable to strictly online, real-time broadcast analysis.

Methodology
The paper proposes a training-free, online imputation framework that operates solely on observations from the match being analyzed, without offline training or access to future frames. The core methodology involves a "ladder" of baselines, culminating in a novel approach called Role-Anchored Centroid Voting (B4):

Baseline Strategies:
- B0 (Ignore): The standard visible-only baseline.
- B1 (Last-Seen): Holds the last observed position, decaying toward the visible team mean.
- B2 (Formation Anchor): Stores a player's offset from the visible team centroid while visible and applies this offset to the current visible centroid when hidden.
- B3 (Enhanced Anchor): Combines B2 with Exponentially Moving Average (EMA) offsets and constant-velocity extrapolation.
- B4 (Role-Anchored Centroid Voting): This is the proposed optimal method. It addresses the "viewport subset bias" inherent in B2. When the camera pans, the visible subset of players is spatially biased relative to the true team centroid. B4 mitigates this by having each visible player "vote" for the full-team centroid. Each visible player $i$ proposes the full-team centroid as its current position minus its stored role offset ( $p_i - off_i$ ). The voted centroid $\hat{c}(t)$ is the average of these proposals. Offsets are then updated via EMA relative to this voted centroid. Hidden players are imputed at $\hat{c}(t) + off_j$ . This creates a self-consistent bootstrap where offsets and the centroid are mutually adjusted to cancel out systematic subset bias.
Benchmarking Protocol:
- Data: Open full-pitch tracking data from Metrica Sports (three matches, 25 fps).
- Simulation: A virtual main camera pans horizontally following an exponentially smoothed ball position, creating a viewport of width $W$ (tested at 36m, 44m, 52m, 60m).
- Metrics: The evaluation prioritizes downstream decision-relevant metrics over pure trajectory fidelity:
  - Hidden-Player Position Error: Distance between imputed and true position.
  - Pitch-Control Map MAE: Error in the spatial control map, specifically in the hidden zone.
  - Team Control-Share Error: Error in the headline metric of "percentage of pitch controlled by Team A."

Key Contributions

Reproducible Distortion Benchmark: The authors establish an open benchmark quantifying the error introduced by ignoring off-screen players. They demonstrate that the "ignore" baseline inflates hidden-zone pitch-control error by 25.1–26.9 percentage points and control-share error by 11.1–13.4 points.
Training-Free Online Baselines: The paper introduces a hierarchy of methods that require no training data. The best performer, Role-Anchored Centroid Voting (B4), reduces hidden-zone control error by roughly half (to 12.2–13.8 points) and cuts control-share error to 28–48% of the ignore-policy baseline across all viewport widths.
End-to-End Integration: The method is integrated into a broadcast-video GSR pipeline and tested on real World Cup footage. Given the lack of full-pitch ground truth for these broadcast clips, the study presents a sensitivity analysis rather than claiming the imputed scores as 'true' values. It demonstrates that imputation can alter downstream possession-quality scores (Space-Creation Index) by 15.6 to 17.2 points, sufficient to flip the verdict class (e.g., from "dead possession" to "weak progression") in specific game windows, thereby highlighting the potential impact of camera framing on tactical interpretation.

Results

Performance: At a standard viewport width of 44m, B4 achieves a median position error of 9.7–11.6 m for hidden players, outperforming the formation anchor (B2) and significantly outperforming the ignore baseline.
Robustness: The method performs consistently across three different matches and varying viewport widths. The improvement is most pronounced in narrower (harder) viewports where fewer players are visible.
Occlusion Regimes: In the short-occlusion regime (≤9.6 s), comparable to prior learned work, B4 achieves median errors of 3.3–8.9 m. Crucially, the benchmark reveals that 50–57% of hidden-player observations in broadcast footage fall into the long-occlusion regime (>9.6 s), a domain where prior bidirectional learned models cannot operate. B4 provides a viable solution for this majority of cases without future context.
Ablation: The voting mechanism (B4) is shown to be superior to simple offset application (B3E) because it aligns the reference frame for storing offsets with the reference frame for applying them, preventing the accumulation of bias.

Significance and Claims
The paper maintains a modest position, explicitly stating that learned imputers remain preferable when training data and future context are available. However, the authors establish three critical points:

Quantified Distortion: They demonstrate that the distortion caused by ignoring off-screen players is large enough to invalidate spatial metrics if unaddressed.
Training-Free Floor: They provide a performance baseline (B4) that learned methods should exceed, proving that significant gains are possible without offline training or GPU resources.
Long-Occlusion Viability: They highlight and score the long-occlusion regime (>9.6 s), which constitutes the majority of hidden-player observations in broadcast data and lies outside the scope of the closest prior learned work.

The work concludes that for strictly online, broadcast-based analytics, a training-free method like Role-Anchored Centroid Voting is a necessary and effective component to recover spatial integrity, preventing camera framing from dictating tactical analysis.

Technical Summary: Training-Free Off-Screen Player Imputation for Broadcast-Based Spatial Football Analytics

More like this