💻 computer science

TacSIm: A Dataset and Benchmark for Football Tactical Style Imitation

The paper introduces TacSIm, a large-scale dataset and benchmark designed to evaluate and model football tactical style imitation by replicating the coordinated actions of all 11 players in a team using broadcast footage, with a focus on spatial and temporal similarity rather than traditional reward-based objectives.

Original authors: Peng Wen, Yuting Wang, Qiurui Wang

Published 2026-03-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Peng Wen, Yuting Wang, Qiurui Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a football coach. You have a brilliant team with a specific "style" of playing—maybe they swarm the ball like a hive of bees, or maybe they stretch out like a rubber band to catch opponents off guard. You want to teach your new players this style, or perhaps you want to test a "what-if" scenario: "What if we played like Manchester City against this specific opponent?"

Until now, computers were terrible at this. They could tell you who scored a goal, but they couldn't understand how the team moved together to get there. They were like a referee who only counts goals but doesn't understand the game plan.

This paper introduces TacSIm, a new tool that acts like a "Tactical Time Machine" for football. Here is how it works, broken down simply:

1. The Problem: The "Black Box" of Broadcasts

When you watch a football match on TV, you see the players moving. But to a computer, that's just a blurry video with a shaky camera. It's hard to tell exactly where every single player is, especially when the camera zooms in, switches angles, or when a crowd blocks the view.

Previous computer programs tried to learn football by looking at individual players (like a single dancer) or by playing fake games in a video game (like FIFA). But real football is about 11 players moving as one giant organism. The old tools couldn't capture that group dance.

2. The Solution: TacSIm (The "Tactical Mirror")

The authors built a massive library of real Premier League matches and did something magical: they turned the TV broadcast into a perfect, 3D map.

The Magic Glasses: They used special computer vision "glasses" to look at the TV footage and figure out exactly where every single player and the ball were on the field, even if the camera was zoomed in or the view was blocked.
The Virtual Twin: They took this real-world data and dropped it into a virtual football simulation. Now, they have a digital twin of a real match.
The Goal: The computer's job is to watch the first few seconds of a real play and then predict how the team will move for the next few seconds. It's like asking a student to finish a sentence after hearing the first few words.

3. How Do We Grade the Computer? (The "Ruler")

How do you know if the computer is actually learning the team's style and not just guessing? The authors created a new grading system with two rulers:

Ruler 1: The "Where" (Spatial Occupancy): Imagine the football field is a grid of tiles. Did the computer's virtual players stand on the same tiles as the real players? If the real team huddled on the left side, did the virtual team do the same?
Ruler 2: The "How" (Movement Vectors): Did they move in the same direction and with the same energy? If the real team rushed forward like a wave, did the virtual team rush forward, or did they just shuffle awkwardly?

They combine these two scores to give a final grade on how well the computer "imitated" the team's soul.

4. The Experiments: Testing the "Students"

The researchers tested several different AI "students" (algorithms) to see who could learn the best.

The "Copycat" (Behavior Cloning): This student just tries to memorize what happened. It's okay for short plays but gets confused quickly.
The "Strategist" (CoDAIL & DRAIL): These students try to understand the rules of the game and the relationship between players. They are much better at predicting long-term moves.

The Big Discovery:
They found a "Goldilocks Zone" for the grid size.

If the grid is too coarse (big blocks), the computer misses the details (like a blurry photo).
If the grid is too fine (tiny pixels), the computer gets overwhelmed by noise and forgets the big picture (like trying to read a book by looking at one letter at a time).
Medium-sized grids worked best, balancing detail with the big picture.

5. Why Does This Matter?

Think of TacSIm as a flight simulator for football coaches.

For Coaches: They can test new strategies in the virtual world before risking them in a real match. "What happens if we push our defense up 10 yards?" The simulator can show the result.
For Scouts: It helps identify players who fit a specific tactical style.
For Fans: Eventually, this could lead to better broadcasts that show you exactly what the players are thinking and planning in real-time.

In a Nutshell

TacSIm is the first time we have a massive, high-quality dataset that teaches computers not just who is on the field, but how the whole team dances together. It turns football from a game of "guessing the next move" into a science of "understanding the team's style," bridging the gap between watching a match on TV and simulating it in a video game.

1. Problem Statement

Current football imitation research predominantly focuses on optimizing reward-based objectives (e.g., win rates, goals scored) rather than accurately replicating real-world tactical behaviors and team styles. Existing approaches face three major challenges:

Data Limitations: Most research relies on synthetic data or limited tracking data. Real-world broadcast footage is difficult to process due to camera switching, occlusions, and the lack of full 11-vs-11 trajectory data for both teams simultaneously.
Imitation Imbalance: Existing methods often prioritize individual behavior cloning or reward-maximizing agents, failing to capture the complex, coordinated spatio-temporal dependencies of a full team.
Evaluation Gaps: Current frameworks focus on individual errors or short-term rewards, lacking metrics to assess the spatial and temporal consistency of team-level tactical styles between real matches and model-generated simulations.

2. Methodology

A. Dataset Construction (TacSIm)

TacSIm is a large-scale dataset derived from 140 matches of the 2024-2025 English Premier League (EPL).

Data Acquisition: Raw 1080p broadcast footage is segmented into possession-based clips (Attack, Defense, Transition) to ensure tactical continuity.
Annotation: Performed by domain experts, categorizing phases based on ball possession and outcomes (e.g., Successful Attack vs. Failed Defense).
Trajectory Reconstruction:
1. In-camera Reconstruction: Uses YOLOv11 for detection, DeepSORT for tracking, and TVCalib for camera calibration to project 2D image coordinates to a standardized bird's-eye-view pitch coordinate system ( $x \in [-1, 1], y \in [-0.42, 0.42]$ ).
2. Off-camera Reconstruction: Addresses occlusions and missing data using a Conditional Variational Autoencoder (CVAE). This model learns the probability distribution of trajectories to impute missing states, ensuring physically plausible and smooth motion sequences.
Scale: The dataset contains ~194,565 video segments (approx. 38,913 seconds) covering all 22 players and the ball, split 70/15/15 for training/validation/testing.

B. Benchmark Design

TacSIm defines a Tactical Style Imitation task where a model must generate multi-agent trajectories conditioned on the initial state (first frame) to match real-world ground truth.

Adaptive Spatial Discretization: The pitch is divided into grids of varying resolutions (from coarse 60 cells to fine 1768 cells) based on the average displacement magnitude ( $s_t$ ) to balance precision and computational efficiency.
Evaluation Metrics:
1. Spatial Occupancy Similarity ( $S_t$ ): Measures the overlap between ground-truth and predicted player positions using the Jaccard Index.
2. Movement Vector Similarity ( $S_v$ ): Measures the alignment of movement direction and activity patterns using a Cosine-like similarity on flattened motion feature vectors.
3. Final Score: The Harmonic Mean of $S_t$ and $S_v$ , ensuring that a model cannot achieve a high score by excelling in only one dimension (e.g., good positioning but wrong movement direction).

C. Baseline Models

The benchmark evaluates five representative imitation learning approaches:

Behavior Cloning (BC): Supervised learning mapping states to actions.
Coordinated Multi-Agent Imitation Learning (CMIL): Introduces shared latent representations for inter-agent dependencies.
Inverse Reinforcement Learning (IRL): Infers underlying reward functions from demonstrations.
CoDAIL: Decentralized Adversarial Imitation Learning with correlated policies.
DRAIL: Diffusion Rewards Adversarial Imitation Learning, leveraging diffusion processes for generative smoothness.

3. Key Contributions

First Broadcast-Derived Tactical Dataset: TacSIm is the first dataset to reconstruct full-team (22 players + ball) trajectories from monocular broadcast footage with tactical semantics, enabling real-to-virtual simulation.
Standardized Evaluation Protocol: Introduces a rigorous benchmark with adaptive grid discretization and composite metrics ( $S_t, S_v$ ) specifically designed to measure tactical style fidelity rather than just aggregate rewards.
Comprehensive Benchmarking: Provides a unified evaluation of state-of-the-art multi-agent imitation models, revealing their strengths and weaknesses in capturing collective intelligence.

4. Experimental Results & Analysis

Experiments were conducted on an NVIDIA 3090 GPU. Key findings include:

Time Horizon Degradation: All models show significant performance drops as prediction time increases (3s $\to$ 10s). This highlights the difficulty of long-horizon tactical intent deduction compared to short-term motion extrapolation.
Optimal Grid Resolution: Medium-sized grids (e.g., $15 \times 10$ $15 \times 10$ or $20 \times 12$ $20 \times 12$ ) yield the best balance.
- Coarse grids lose tactical detail.
- Fine grids suffer from the "curse of dimensionality," making it hard to learn patterns from sparse data and overfitting to noise.
Model Performance:
- CoDAIL excels in short-to-medium term predictions and medium resolutions, demonstrating superior modeling of multi-agent coordination.
- DRAIL shows robustness in long-term forecasting, particularly in spatial occupancy, suggesting its potential for macro-strategic simulation.
- BC performs poorly in long-term scenarios, confirming that simple supervised learning fails to capture complex strategic dependencies.

5. Significance

TacSIm bridges the gap between computer vision (extracting data from video) and multi-agent reinforcement learning (simulating tactics).

For Coaches: It offers a "sandbox" for testing "what-if" scenarios and analyzing team styles without physical risk.
For Research: It shifts the focus from aggregate metrics (win/loss) to behavioral fidelity, providing a standardized way to evaluate how well AI can mimic the "style" of a team.
Future Impact: The dataset and benchmark lay the groundwork for developing hierarchical frameworks that combine the tactical precision of coordination models with the strategic depth of diffusion-based generative models.