💻 computer science

SoccerNet 2026 Player-Centric Ball Action Spotting: Per-Player Attention with Agreement-Based Ensembling

This paper presents a two-stage system for the SoccerNet 2026 challenge that combines a Track-Aware Action Detector with a Denoising Sequence Transduction transformer featuring spatial-first per-player attention and an agreement-based ensemble, achieving a Macro-F1 score of 58.94.

Original authors: Faisal Altawijri, Ismail Mathkour

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Faisal Altawijri, Ismail Mathkour

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to write a play-by-play commentary for a soccer match, but you have to do it perfectly for every single player on the field, second by second. That is the challenge this paper tackles: figuring out exactly what every player is doing with the ball (dribbling, passing, shooting, etc.) and when.

The authors, from a team called TAHAKOM, built a "digital referee" system that achieved a very high score (58.94 out of 100) in a competition called SoccerNet 2026. Here is how they did it, broken down into simple steps.

The Two-Stage Factory

Think of their system as a two-step factory line.

Stage 1: The "Eye" (TAAD)
First, the system needs to see what's happening. The original system used a basic camera to watch the players. The authors upgraded this to a "Temporal Transformer."

The Analogy: Imagine a regular camera that takes a snapshot of a player every second and guesses what they are doing based only on that single photo. The new system is like a movie director who watches the whole scene. It doesn't just look at one frame; it watches the flow of movement across several frames to understand the story of the action.
The Fix: They also found a glitch in the training process (like a thermostat that got stuck) and fixed it, allowing the system to learn much more effectively.

Stage 2: The "Brain" (DST)
Once the "Eye" sees the actions, the "Brain" has to organize them into a coherent list of events.

The Old Way: The original brain treated all 26 players on the field as a flat, jumbled list of data. It was like trying to understand a conversation by listening to everyone shout at once without knowing who is talking to whom.
The New Way (Per-Player Attention): The authors gave the brain a superpower: Focus.
1. Spatial Attention (The Room): First, the system looks at all players at a single moment and asks, "Who is near whom?" It understands the formation of the team.
2. Temporal Attention (The Timeline): Then, it zooms in on each player individually and watches how that specific person moves over time.
- The Result: By understanding the spatial relationships first (who is where), the system gets a much better context for understanding the timeline (what they are doing).

The "Committee" Strategy (Ensembling)

Instead of relying on just one smart AI, the authors trained four different versions of their "Brain" (Models A, B, C, and D). Each one was slightly different, like having four experts with different specialties.

To get the final answer, they didn't just pick the best one. They used a Committee Voting System:

The Agreement Rule: If only one expert says, "Player #10 is shooting," the system ignores it. It assumes that expert might be hallucinating (seeing things that aren't there).
The Boost: If two or more experts agree, the system boosts the confidence score. It's like saying, "If three people tell me it's raining, I'm definitely going to grab an umbrella."
The "Solo Tackle" Exception: There was one problem: "Tackles" (stealing the ball) are so rare that sometimes only one expert spots them. If they applied the strict agreement rule, they would miss all the tackles. So, they made a special exception: If a tackle is predicted, even by just one expert, they keep it. This ensures they don't miss the rare, tricky moments.

The Results

By combining these improvements, the team saw a steady climb in performance:

Base System: 48.6% accuracy.
Adding the "Movie Director" (Stage 1): Improved slightly.
Adding the "Focus" (Stage 2): Big jump to 55.1%.
The Committee (Ensemble): The final score hit 58.94%.

Why it matters:
The most impressive part is that the "Committee" didn't just get lucky on the practice games (validation); it generalized well to the real test games (challenge). The gap between their practice score and test score shrank from a wide 6-point difference down to just 2 points. This proves that their system isn't just memorizing answers; it's actually learning how to understand soccer.

In short, they built a system that watches soccer like a director, thinks about it like a focused analyst, and makes decisions like a cautious committee of experts.

Technical Summary: SoccerNet 2026 Player-Centric Ball Action Spotting

Problem Statement

The paper addresses the challenge of Player-Centric Ball Action Spotting within the SoccerNet 2026 competition. The objective is to detect and localize specific ball-related actions (e.g., passing, shooting, tackling) performed by individual players in video sequences. The system must output structured event sequences comprising frame timestamps, action classes, and player roles, moving beyond simple frame-level classification to a temporally coherent event detection pipeline.

Methodology

The proposed system adheres to the two-stage pipeline established by the organizers, consisting of a Track-Aware Action Detector (TAAD) and a Denoising Sequence Transduction (DST) model. The authors introduce specific architectural and training modifications to both stages, culminating in an agreement-based ensemble strategy.

Stage 1: Track-Aware Action Detector (TAAD) Improvements

The baseline TAAD, which relies on an X3D architecture, was enhanced to better capture temporal dependencies:

Temporal Transformer Integration: Instead of classifying frames independently, the system projects ROI-aligned features (per player, per frame) into a 256-dimensional space. These are processed through a 4-layer transformer encoder (8 attention heads, 1024 feedforward dimension) with learnable positional embeddings (up to 50 frames). This allows the model to reason about temporal patterns before projecting to a 512-dimensional space for 9-class classification.
Training Refinements: The authors corrected a warmup scheduler bug that previously halted learning rate updates. Additional modifications included:
- A cosine annealing scheduler for all layers except the X3D backbone (which uses a fixed LR).
- Learning rates scaled by $1/\sqrt{8}$ .
- Reduced background class weight (0.05) and increased tracklet count from 4 to 6 starting at epoch 16.
- The resulting logits are treated as frozen inputs for the subsequent DST stage.

Stage 2: Denoising Sequence Transduction (DST) Improvements

The DST component converts game-state features and TAAD logits into structured event sequences via an autoregressive transformer. Key innovations include:

Per-Player Attention Mechanism: The baseline encoder treated the 26-role feature vector as a flat input. The proposed method introduces a two-stage attention mechanism:
1. Spatial Attention (Stage 1): Cross-player attention where all 26 players at a specific timestep attend to one another, capturing spatial configurations.
2. Temporal Attention (Stage 2): Self-attention applied independently to each player's representation across the time dimension to model state evolution.
- Note: Experiments indicated that using only game-state features (positions, velocities, visibility) for this attention mechanism, excluding TAAD logits, yielded superior performance.
Spatial-First Ordering: Processing spatial attention before temporal attention proved critical, yielding a +1.87% Macro-F1 improvement on validation compared to the reverse order.
Architectural Diversity for Ensembling: Four distinct DST models (A–D) were trained with varying hyperparameters to ensure diversity:
- Model A: Base configuration (hidden dim 512, 1 attention layer).
- Model B: Increased hidden dimension (768).
- Model C: Increased attention depth (2 layers) and dimension (dp=128).
- Model D: Parallel attention architecture where spatial and temporal branches process inputs independently and are summed.
Inference Optimization: The authors found that optimizing the logit adjustment temperature ( $\tau$ ) based on per-class thresholds led to overfitting on the validation set. Consequently, they adopted $\tau=0$ with default thresholds for all final predictions.

Ensemble Strategy

The final submission employs a Weighted Event Fusion strategy with Agreement Filtering:

Clustering: Predictions from $N$ models are grouped by (team, shirt number, action class) and clustered within a $\pm 12$ -frame window.
Agreement Boost: For clusters supported by $n$ models, the average score is multiplied by $(n/N)^{0.5}$ .
Filtering: Events predicted by fewer than 2 models are discarded to suppress false positives (hallucinations).
Exception Handling: Due to the extreme rarity of "Tackle" events (26 occurrences in 96 training games) and low per-model F1, the agreement filter is bypassed for tackles. Single-model detections are preserved to maintain recall for this specific class.

Results

The system achieved a Macro-F1 score of 58.94 on the challenge set.

Progressive Gains: The baseline DST with improved TAAD scored 48.6 on the challenge. Adding per-player attention and the spatial-first order increased this to 55.1. The final 4-model ensemble reached 58.94.
Per-Class Performance: The ensemble outperformed individual models on all classes except "Tackle," where Model B remained the strongest individual performer. The ensemble significantly improved performance on common-to-medium frequency classes (e.g., drive, pass, cross) by suppressing false positives while maintaining recall.
Generalization: The gap between validation and challenge performance narrowed from approximately 6 points (single model) to 2 points (ensemble), indicating improved generalization.

Significance and Claims

The paper claims that the primary contribution lies in the integration of per-player attention mechanisms within the DST encoder and the agreement-based ensemble strategy.

The spatial-first attention order is identified as a critical architectural choice that provides richer context for temporal reasoning.
The agreement filtering mechanism is highlighted as the key to reducing false positives without sacrificing recall on frequent classes, while the specific exception for tackle events demonstrates a targeted approach to handling class imbalance.
The authors modestly frame their work as a systematic improvement on the provided baseline pipeline, demonstrating that architectural diversity and rigorous ensemble filtering can significantly boost performance in complex, multi-player action spotting tasks.