💻 computer science

SoccerTrack v2: A Full-Pitch Panoramic Video Dataset for Game State Reconstruction and Ball Action Spotting

SoccerTrack v2 introduces a comprehensive dataset of 932 minutes of 4K panoramic video from ten university-level matches, integrating full-pitch player trajectories, metric coordinates, and actor-linked ball action events to enable unified research on game state reconstruction and ball action spotting.

Original authors: Atom Scott, Ikuma Uchida, Kento Kuroda, Yufi Kim, Keisuke Fujii

Published 2026-10-06✓ Author reviewed ⓘ

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Atom Scott, Ikuma Uchida, Kento Kuroda, Yufi Kim, Keisuke Fujii

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

To understand the modern game of soccer, one must look at two different kinds of information. The first is a map of where everyone is standing at any given moment, tracking the movement of every player and the ball across the grass. The second is a log of what they actually do: a pass, a shot, a tackle, or a goal. For years, researchers have struggled to study these two things together. Most available data comes from television broadcasts, where the camera follows the action like a spotlight. This means that when the ball moves to the other side of the field, the players left behind in the dark are simply ignored. Their positions are guessed at or left out entirely, making it impossible to see the full picture of the game as it unfolds. Without seeing the whole field, it is difficult to understand how a team's shape changes or how a specific action ripples through the entire group.

A team of researchers from Nagoya University and the University of Tsukuba has now released a new resource that finally brings the full view and the full story together. They call it SoccerTrack v2. Instead of relying on a camera that pans and zooms, they used fixed cameras mounted high above the stadium that capture the entire pitch in a single, wide shot. This setup ensures that no player ever disappears from the frame, no matter where the ball goes. The dataset includes ten full university-level matches, totaling over 930 minutes of continuous video. For every single frame of this footage, the researchers have recorded the exact location of every player, their jersey number, their role, and which team they belong to. They have also tagged every ball action, from a simple pass to a complex tackle, and linked each action directly to the specific player who performed it. This creates a complete, synchronized record where the movement of the players and the events of the game are tied together in a way that has never been possible before.

The researchers used this new data to test how well computers can reconstruct the game state and spot specific actions. They built a system to track players across the entire duration of a match, which can last up to 45 minutes for a single half. When they tested this system on short clips of just 30 seconds, it performed quite well, correctly identifying players and their locations. However, when they let the system run for the full length of the match, its accuracy dropped significantly. The longer the sequence of video became, the harder it was for the computer to keep track of who was who. The system began to lose its grip on identities, mixing up players or losing them entirely as time went on. This finding is crucial because it shows that a system that works perfectly on a short snippet of video cannot be assumed to work just as well over a full game. The challenges of long-term tracking are real and distinct from the challenges of short-term analysis.

In a second experiment, the team tried to teach a computer to recognize ball actions, such as a pass or a shot, using only the movement data of the players and the ball, without looking at the video pixels themselves. They found that knowing where the players were was enough to guess most actions with reasonable accuracy. However, when they added the specific path of the ball to the information, the computer's ability to spot actions improved dramatically, especially for events that happen very quickly. The ball's movement provided a critical clue that the players' positions alone could not fully capture. Interestingly, the computer struggled the most with actions that involved two players fighting for the ball, such as a tackle or a block. These moments are difficult to predict because the outcome depends on a physical collision that is hard to see just by looking at where the players are standing.

The release of this dataset also highlights the limitations of current technology. The researchers noted that while they could see the entire field, the camera angle meant that the ball often appeared as just a few pixels, making it nearly impossible for a computer to detect it directly from the video. Instead, they had to rely on the ball's recorded path, which was reconstructed from the event logs rather than watched frame-by-frame. This means that the current results are based on a mix of observed player movement and reconstructed ball data. The team also pointed out that the data comes from university-level matches, which may play differently than professional games, and that the camera systems used are fixed, unlike the moving cameras used for television.

Despite these limitations, the work represents a major step forward. By providing a resource where the full field, the full duration, and the full story are available in one package, the researchers have given the scientific community a new standard for testing. They have shown that evaluating a system on a short clip can give a misleadingly optimistic view of its performance. To truly understand the game, and to build systems that can watch a full match without losing track, researchers must test their methods on the full length of the game. SoccerTrack v2 offers the first complete playground for this kind of work, allowing scientists to see exactly where their current tools succeed and where they fail, paving the way for a deeper understanding of the beautiful game.

Technical Summary: SoccerTrack v2

Problem Statement
Soccer analytics relies on two distinct data modalities: spatio-temporal data (player and ball positions) and event data (actions performed). Existing public datasets typically offer these separately or combine them only on broadcast footage. A critical limitation of broadcast-based data is that the camera follows the play, leaving players outside the frame unobserved or requiring their state to be imputed rather than measured. Furthermore, most existing benchmarks for game state reconstruction (GSR) and action spotting are defined on short clips (e.g., 30 seconds), failing to capture the challenges of maintaining persistent identities and accurate state over full match durations (45+ minutes). There is a lack of resources that provide continuous, full-pitch video alongside long-term player trajectories and actor-linked events in a single, synchronized resource.

Methodology and Dataset Construction
The authors introduce SoccerTrack v2, a dataset comprising ten university-level matches recorded by fixed, stitched panoramic camera systems. The field of view spans the entire pitch and its surroundings, ensuring players are not lost to camera motion.

Data Volume: The corpus includes 932 minutes of 4K panoramic video (25 fps), totaling 1.4 million annotated frames and 30.7 million player-frame records.
Annotations:
- Game State: Every frame contains metric pitch coordinates (on a canonical 105 × 68 m field), jersey numbers, persistent player identities, roles, and team sides. The median track is annotated for 45 continuous minutes.
- Ball Actions: 21,428 events across twelve classes (e.g., Pass, Shot, Goal) are annotated. Crucially, 99.5% of these events are linked to the acting player using the same persistent identifiers found in the trajectories, allowing events to resolve to specific tracked players at metric locations.
Splits: A fixed match-level split is provided, holding out matches M7 and M9 for testing (the SoccerTrack Challenge 2025 split).
Baselines: The paper establishes two reference systems:
1. Game State Reconstruction (GSR): A pipeline adapted from the TrackLab framework, incorporating person detection, pose estimation, ReID, jersey number recognition, and tracking. The authors modified the pipeline to handle match-length sequences (addressing memory and quadratic scaling issues) and introduced a per-detection team-assignment variant.
2. Ball Action Spotting (BAS): An offline temporal convolutional network trained on the supplied ground-truth player trajectories. Experiments were conducted with and without an auxiliary ball track (which is a reconstruction anchored at event timestamps rather than per-frame measurement).

Key Results

Game State Reconstruction:
- Sequence Length Impact: When evaluated over full halves (45–49 minutes), the GS-HOTA score (12.25) is significantly lower than when evaluated on the opening 30 seconds (31.35). This gap persists even when attribute matching is disabled, indicating that clip-scale evaluation overstates match-length accuracy.
- Bottlenecks: For the full-half runs, association (AssA) is the primary bottleneck, with scores ranging from 3.3 to 18.6, compared to detection accuracy (DetA) ranging from 15.9 to 51.2. The pipeline struggles to maintain identity over long durations, often reusing identities across gaps rather than fragmenting tracks.
- Variability: Performance varies significantly across matches, influenced by capture conditions, calibration quality, and jersey legibility.
Ball Action Spotting:
- Trajectory vs. Ball: Using player trajectories alone yields a mean average precision (mAP) of 0.222 at $\tau=1$ s. Adding the supplied ball track increases this to 0.599.
- Temporal Precision: The benefit of the ball track is most pronounced at tight tolerances. At $\tau=1$ s, adding the ball track improves mAP by a factor of 2.7 compared to trajectories alone.
- Class Performance: The model performs well on ball-centric events (e.g., Pass, Drive) but struggles with duel outcomes (e.g., Ball Player Block, Player Successful Tackle), suggesting that the resolution of positional data is insufficient to determine contact outcomes.

Significance and Claims
The paper claims that SoccerTrack v2 is the entry in the group of full-pitch video releases that carries match-length full-pitch video, all-player state within the released field of view, and actor-linked events together. Its primary significance lies in enabling the evaluation of methods on the full duration of a match rather than short clips.

Evaluation Protocol: The authors demonstrate that opening-clip scores (30 seconds) overstate the accuracy of systems when applied to full halves. They argue that full-match evaluation is a distinct protocol, not merely a longer clip, and that sequence length fundamentally degrades performance in current pipelines.
Inspection Capability: Because identities are annotated for whole halves and linked to events, the dataset allows researchers to inspect reconstructed states and event attributions against the actual footage at the exact moment they occur, facilitating detailed failure analysis.
Complementarity: The authors position SoccerTrack v2 as complementary to broadcast-based datasets (like SoccerNet). While broadcast data is essential for models intended for television, fixed panoramic recordings provide a complete field of view where players are not lost, offering a unique resource for studying game state and action spotting without the occlusion and imputation issues inherent in broadcast tracking.

The paper concludes that while the provided baselines establish reference points, they also highlight significant limitations in current systems regarding long-term identity maintenance and the interpretation of positional data for specific event types (like duels). The release of the data, split, and evaluation tools is intended to facilitate direct comparison of future contributions on the same full-pitch footage.

More like this