💬 NLP

StreamSoccer: Event-Driven Memory for Streaming Soccer Commentary

StreamSoccer is an event-driven system that utilizes a fixed-budget active memory to organize streaming soccer video into semantic event lifecycles, enabling real-time generation of current, recent, and historical commentary with bounded computation while achieving state-of-the-art performance on a newly constructed three-track dataset.

Original authors: Chenxi Shao, Bozhong Wang, Jiaxin Huang, Zhao Liu, Sunwei Zhu, Tianxin Hang, Gaoqi He, Yang Li, Changbo Wang

Published 2026-08-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chenxi Shao, Bozhong Wang, Jiaxin Huang, Zhao Liu, Sunwei Zhu, Tianxin Hang, Gaoqi He, Yang Li, Changbo Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine watching a live soccer match on television. The action moves fast, shifting from a quiet buildup to a sudden goal, then back to a slow tactical struggle. For a human commentator, the challenge is not just seeing the ball, but understanding the story unfolding around it. They must describe the goal that just happened, summarize the last few minutes of pressure, or recall a specific play from twenty minutes ago that explains why the current team is attacking so aggressively. Doing this in real time requires a mind that can hold the present moment, keep a running summary of recent events, and store important past details for later reference, all while processing a continuous stream of information that never stops.

For computers, this task is incredibly difficult. Most existing systems that try to understand video are built to process short clips or entire recorded games after the fact. They struggle when the video is a live stream that keeps growing longer and longer. If a computer tries to remember every single frame of a ninety-minute match, it quickly runs out of memory or slows down to a crawl. Previous attempts to solve this often treated the video as a simple list of pictures or a stream of data points, without understanding that a soccer match is made of distinct "events"—like a corner kick, a foul, or a goal—that have a beginning, a middle, and an end. Without recognizing these events as the building blocks of the game, a computer cannot effectively organize its memory to tell a coherent story.

A team of researchers has introduced a new system called StreamSoccer, designed specifically to solve this problem by changing how the computer remembers the game. Instead of trying to store every visual detail, the system focuses on "events." As the match plays out, the computer watches the video and identifies when a specific play starts. It keeps a small, focused memory of that play while it is happening. Once the play ends—say, when the ball goes out of bounds or a goal is scored—the system closes that memory, saves a short summary of what happened, and clears its working space to focus on the next play. This process allows the computer to maintain a constant, manageable amount of information, no matter how long the match has been going on.

The system organizes these saved summaries into three different types of memory to help it generate commentary. First, it keeps a "current event" memory for the play that just finished, allowing it to describe the immediate action. Second, it holds a "recent window" of the last few completed plays, so it can summarize a sequence of attacks or defenses. Third, it stores "historical records" of older events that can be retrieved later. This structure mimics how a human might think: focusing on what is happening right now, keeping a short-term memory of the last few minutes, and having a long-term memory of the game's key moments. When the system decides to speak, it chooses which type of memory to use based on the situation. It might describe a just-completed goal, summarize a period of sustained pressure, or connect the current play to a similar situation that happened earlier in the game.

To test if this approach works, the researchers created a new dataset of soccer matches where the computer had to generate commentary in real time, using only the information available up to that exact moment. They compared their system against other advanced video-understanding models. The results showed that StreamSoccer was significantly better at describing the game. In tests measuring how well the generated text matched human descriptions, the new system achieved the highest scores for describing current events and recalling historical moments. It also performed very well at summarizing recent sequences of play. Crucially, the system remained fast and efficient throughout the entire match. While other systems slowed down or ran out of memory as the game progressed, StreamSoccer maintained a steady speed, processing the video in less than a quarter of the time it took to watch it, even after ninety minutes of play.

The researchers found that the key to this success was treating the game as a series of events rather than a continuous stream of images. By closing and saving each event once it was finished, the system avoided the problem of trying to remember everything at once. This allowed it to retrieve specific past details when needed without getting bogged down by the sheer volume of the video. The study also showed that simply having access to these saved event summaries improved the quality of the commentary across all three types of tasks. The system did not just memorize the video; it learned to organize the game's history into a usable format that could be accessed instantly.

This work demonstrates that for a computer to act like a skilled sports commentator, it needs more than just powerful vision; it needs a way to structure its memory around the logic of the sport itself. By focusing on the lifecycle of events—activating them, updating them as they unfold, and closing them once they are done—the system can handle the complexity of a live broadcast without losing its place. The findings suggest that this event-driven approach is a viable path forward for real-time video understanding, offering a way to keep long streams of information organized, accessible, and useful for generating natural language descriptions. The system proved that it is possible to watch a full match, remember the important parts, and tell the story of the game as it happens, all while running efficiently on standard hardware.

Technical Summary: StreamSoccer

Problem Statement

Streaming video understanding requires models to causally update their internal state as video arrives, organizing a growing history into semantic units that can evolve, persist, and be recalled under bounded computation and memory. This challenge is particularly acute in live soccer commentary. Unlike offline video captioning or methods relying on predefined clips, a streaming system must continuously understand an evolving match process and generate commentary at appropriate moments without access to future information.

Existing approaches face two primary limitations in this domain:

General Streaming Video-Language Models: Systems like StreamingVLM or TimeChat-Online typically organize history around frames, visual tokens, or key-value caches. While efficient, they often treat the evolving semantic process as a sequence of agnostic tokens rather than maintaining a continuous state unit that must be closed upon completion and reused later.
Soccer-Specific Methods: Existing soccer commentary models (e.g., SoccerNet-Caption, MatchTime) generally rely on predefined video clips, given output timestamps, or offline full-video inputs. They do not inherently support the causal, event-driven lifecycle required for real-time streaming where the system must decide when to speak (or remain silent) based solely on information available up to that moment.

The core problem is identifying an intermediate representation that can absorb new observations as an event develops, compress a variable-duration process into a fixed-budget state, form stable and reusable memory upon completion, and support both recent retention and historical retrieval.

Methodology: StreamSoccer

The authors propose StreamSoccer, an event-driven system that uses event memory as its intermediate representation. The system transforms a growing video history into fixed-budget states with explicit semantic roles through an activate–update–close–consolidate–retrieve lifecycle.

1. System Architecture

Streaming Event Memory Encoder:
- Input Processing: Incoming video clips are encoded into visual tokens by a frozen Qwen3-VL encoder and compressed into a fixed number of clip-level event tokens ( $E_t$ ) via a learnable Clip Event Adapter.
- State Update: An Online Event Update module compares incoming tokens with the previous active memory ( $M_{t-1}$ ). Using a gated integration mechanism, it updates the ongoing state to absorb incremental information. The state maintains a fixed token budget ( $K_m$ ) regardless of the number of clips spanned.
- Event Closure: The system determines when an event ends via two mechanisms: a predicted operational-transition indicator ( $b_t$ ) learned by a transition head, and a deterministic duration safeguard ( $\rho_t$ ) that closes an event if it exceeds a maximum duration ( $D_{max} = 24s$ ).
- Memory Consolidation: When an event closes, the active state becomes an immutable completed-event memory ( $M^c_j$ ). This memory immediately enters a local Recent Event Buffer ( $B_t$ ) for short-term context. Simultaneously, a record branch converts $M^c_j$ into a textual Historical Event Record ( $R_j$ ) via a memory-to-language projector and a language model. These records are stored in a retrievable list ( $L_t$ ) for long-term context.
Multi-Context Commentary Generator:
- A unified generator produces commentary for three distinct tracks using the same language model backbone (with LoRA adaptation) but different context configurations:
  1. Current-event commentary: Uses the just-closed completed-event memory ( $M^c_j$ ).
  2. Recent-window commentary: Concatenates the active event memory ( $M_t$ ) with selected local completed-event memories from the buffer.
  3. Historical-memory commentary: Combines current/recent latent context with retrieved textual records ( $R_t$ ) from the historical list.
- The generator projects these contexts into a soft prefix ( $Z_t$ ) to guide the language model.
Rule-Assisted Scheduler:
- A deterministic scheduler selects the commentary mode (current, recent, historical, or silence) based on fixed rules, cooldowns, and priorities. It does not learn speaking decisions but ensures the system adheres to causal constraints and operational logic (e.g., requiring a salient event for current-event mode).

2. Task Formulation and Data

The authors define streaming soccer commentary as a generation task constrained by temporal visibility. They constructed a three-track dataset from SoccerNet action annotations and MatchTime commentary:

Current-event: Describes a just-completed event.
Recent-window: Summarizes multiple recent events visible before the cutoff.
Historical-memory: Connects current play with earlier match information.
The dataset ensures strict causal validity with explicit observation cutoffs and match-disjoint splits to prevent future leakage.

3. Training

StreamSoccer employs a three-stage training process:

Stage 1: Trains the Streaming Event Memory Encoder for action recognition, operational-transition prediction, and event type classification.
Stage 2: Trains the memory-to-language projector to generate event-record captions from completed memories.
Stage 3: Optimizes the multi-context commentary generator across all three tracks, incorporating auxiliary losses for memory context selection and retrieval ranking.

Key Results

The system was evaluated on 58 matches (174 raw-video runs) with a focus on commentary quality at common causal anchors and long-history efficiency.

Commentary Quality: At shared causal output anchors, StreamSoccer achieved the highest reported CIDEr scores on the current-event (38.62) and historical-memory (17.39) tracks, ranking second on the recent-window track (23.96) behind UniSoccer. It also ranked first on the historical-memory track for BERTScore-F1.
Efficiency and Scaling: StreamSoccer maintained a 95th-percentile Real-Time Factor (RTF) of approximately 0.10–0.22 throughout a 90-minute match. Crucially, this efficiency remained stable without sustained growth as match history increased, unlike baseline models (e.g., VideoLLM-online) whose costs grew with context length or hit context limits.
Event Memory Validation: Ablation studies confirmed that recurrent event memory outperformed both operational-event pooling and fixed-time pooling for downstream caption generation. Furthermore, adding local event memory and historical records progressively improved performance across all three tracks, with the full configuration yielding the best results.
Qualitative Analysis: Case studies demonstrated that StreamSoccer successfully connected ongoing pressure with earlier set-piece events, whereas baseline models often produced repetitive claims, inconsistent timestamps, or failed to organize play.

Significance and Claims

The paper claims that event memory serves as an effective intermediate representation for streaming soccer commentary. By explicitly modeling event lifecycles—activating, updating, closing, and consolidating events—the system organizes a growing video history into reusable, fixed-budget states.

The authors emphasize that StreamSoccer:

Supports Multi-Scope Commentary: It successfully handles current, recent, and historical commentary within a single unified framework, addressing the multi-temporal nature of sports commentary.
Controls Long-History Computation: Unlike methods that compress frames or tokens indiscriminately, StreamSoccer's event-driven approach prevents computational degradation as the match progresses, enabling real-time processing over full matches.
Provides a Causal Framework: The system strictly adheres to causal constraints, ensuring that all commentary is generated based solely on information available before the emission time, a requirement often overlooked in offline or pre-defined clip-based approaches.

The paper concludes that while the system relies on rule-assisted scheduling and deterministic event boundaries, the proposed event-memory architecture offers a robust solution for the specific challenges of live, streaming sports commentary.