💻 computer science

BoxComm: Benchmarking Category-Aware Commentary Generation and Narration Rhythm in Boxing

This paper introduces BoxComm, the first large-scale dataset and benchmark for boxing commentary generation that features a novel taxonomy and evaluations for category-aware content and narration rhythm, revealing current multimodal models' limitations in handling the rapid, subtle, and tactical nature of combat sports.

Original authors: Kaiwen Wang, Kaili Zheng, Rongrong Deng, Yiming Shi, Chenyi Guo, Ji Wu

Published 2026-04-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kaiwen Wang, Kaili Zheng, Rongrong Deng, Yiming Shi, Chenyi Guo, Ji Wu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a boxing match on TV. The action is lightning-fast: a punch lands in the blink of an eye, a fighter dodges, and the crowd roars. Meanwhile, the commentator is a master storyteller. They aren't just saying "He hit him"; they are saying, "Look at that jab! He's setting up a trap, but his footwork is slipping!"

For a long time, computers have been great at understanding slow sports like soccer or basketball, where the action is spread out. But Boxing is different. It's a high-speed, high-stakes game of chess played at 100 miles per hour. Until now, computers have been terrible at "commentating" on boxing because they miss the tiny, split-second details and don't know when to speak.

This paper introduces BoxComm, a new project designed to teach computers how to be professional boxing commentators. Here is the breakdown in simple terms:

1. The Problem: Computers Are "Blind" to the Punch

Think of a boxing match as a rapid-fire drum solo.

Team Sports (Soccer/Basketball): The action is like a slow jazz song. You see a pass, then a shot, then a goal. There are pauses. Computers can easily say, "He passed the ball."
Boxing: The action is like a machine gun. A punch happens in 300 milliseconds (faster than you can blink). The difference between a "jab" and a "hook" is just a tiny change in arm angle.
The Issue: Current AI models are like someone trying to describe a drum solo while wearing noise-canceling headphones. They miss the fast punches, and when they do speak, they talk at the wrong time or say the wrong things. They also don't know the difference between describing a punch (Play-by-Play), analyzing the strategy (Tactical), or telling a story about the fighters (Contextual).

2. The Solution: Building a "Boxing School" (BoxComm)

The researchers built a massive library called BoxComm.

The Library: They collected 445 real World Boxing Championship matches.
The Text: They transcribed over 52,000 sentences spoken by real human commentators.
The Labels: They didn't just write down the words; they tagged every sentence. Is it a quick description? Is it a deep strategy analysis? Is it background info?
The Result: This is the first-ever "textbook" for teaching AI how to talk about combat sports.

3. The Two Tests: Can the AI Pass the Class?

To see if the AI is learning, they created two specific tests:

Test A: The "Costume Change" (Category-Conditioned Generation)
Imagine the AI is an actor. The researchers show it a video clip and say, "Now, act like a Tactical expert. Explain the strategy." Then they say, "Now, act like a Play-by-Play announcer. Just describe the punch."
- The Goal: Can the AI switch its "voice" and style correctly based on what the video shows?
Test B: The "Rhythm Check" (Commentary Rhythm)
This is like a DJ mixing music. A good commentator knows when to talk and when to be silent.
- The Goal: If you let the AI talk freely for a whole match, does it sound like a human? Does it talk too much during quiet moments? Does it stay silent when a huge punch lands? Does it balance the types of comments (strategy vs. action) correctly over time?

4. The Secret Weapon: The "Punch Tracker" (EIC-Gen)

The researchers found that even the smartest AI models were failing these tests. They were hallucinating (making things up) or missing punches entirely.

So, they gave the AI a cheat sheet.

The Analogy: Imagine you are trying to describe a fast car race, but you are blindfolded. You will fail. But if someone hands you a list that says "Red car passed at 10:01, Blue car crashed at 10:02," you can describe the race perfectly.
The Tech: They built a special tool that detects every single punch in the video and turns it into a simple text note (e.g., "Red boxer, left hook, hit at 14.5 seconds").
The Result: When they fed these "notes" to the AI along with the video, the AI's performance skyrocketed. It stopped missing punches and started giving accurate, high-level analysis.

5. Why Does This Matter?

This isn't just about boxing. It proves that for fast, complex sports, AI needs more than just "eyes" (video cameras); it needs "high-speed processing" (event detection) to understand what is happening.

In a nutshell:
The paper says, "We built a giant dataset of boxing commentary, tested the best AI models, and found they are terrible at it. But, if we give the AI a 'punch tracker' to help it see the fast moves, it suddenly becomes a great commentator."

This is a huge step toward having AI that can watch a fight, understand the strategy, and tell the story just like a human expert.

1. Problem Statement

While Multimodal Large Language Models (MLLMs) have advanced general video understanding, their application to sports commentary generation remains limited. Existing benchmarks focus exclusively on team sports (e.g., soccer, basketball), failing to address the unique challenges of combat sports like boxing. The paper identifies two critical gaps:

Temporal and Visual Complexity: Boxing actions occur at sub-second speeds (e.g., a jab lasting ~300ms) with subtle visual differences (e.g., distinguishing a jab from a hook based on arm trajectory) that carry decisive semantic meaning. Current models struggle to perceive these fleeting events.
Commentary Structure: Unlike team sports where "play-by-play" (PbP) dominates, boxing commentary requires a significantly higher proportion of tactical analysis (strategy, distance control, momentum) and contextual framing. Existing benchmarks lack the category-level granularity to evaluate a model's ability to switch between these distinct commentary styles or maintain the correct "narration rhythm" over time.

2. Methodology

A. The BoxComm Dataset

The authors constructed BoxComm, the first large-scale dataset for combat sports commentary.

Scale: 445 World Boxing Championship match videos (2025) with over 52,000 sentence-level annotations.
Annotation Pipeline:
1. Extraction: Used WhisperX for automatic speech recognition (ASR) to generate word-level timestamps.
2. Refinement: Employed GPT-5.2 to correct domain-specific terminology (e.g., "fainting" $\to$ "feinting") and re-segment text based on semantic completeness, preserving micro-level PbP bursts (e.g., "Solid jab!").
3. Categorization: Each sentence is labeled into one of three categories:
  - Play-by-Play (PbP): Immediate description of visible actions.
  - Tactical: Analysis of strategy, technique, and matchup dynamics.
  - Contextual: Background info, fighter stats, and event framing.
Event Extraction: A dedicated pipeline (Figure 5) extracts 260,000 fine-grained punch events using Detectron2, CameraHMR, and Causal TCNs. These events capture timestamp, boxer, distance, technique, target, and effectiveness.

B. Evaluation Protocols

The paper proposes two complementary benchmarks:

Category-Conditioned Generation:
- Task: Given a video window, recent commentary history, and a target category (PbP, Tactical, or Contextual), the model must generate a single sentence matching that category.
- Metrics: BERTScore (semantic similarity) and LLM-as-Judge (Content Consistency Accuracy).
Commentary Rhythm Assessment:
- Task: A streaming setting where the model generates commentary continuously over a match without category constraints.
- Metrics:
  - Temporal Intersection-over-Union (t-IoU): Measures alignment between predicted and ground-truth speech windows.
  - KL Divergence: Measures how well the model replicates the human distribution of commentary types over time (pacing).

C. Proposed Baseline: EIC-Gen

To address the perception gap, the authors propose Event-Informed Commentary Generation (EIC-Gen).

Mechanism: Instead of relying solely on raw video frames, the system extracts structured punch events (e.g., {"time": 14.5s, "tech": "left hook", "target": "head"}) and converts them into natural language templates.
Integration: These text-based action cues are concatenated with the recent commentary history and fed into the MLLM as an explicit "action prior," grounding the generation in precise, detected events.

3. Key Contributions

BoxComm Dataset: The first large-scale dataset for boxing commentary with category-level annotations (PbP, Tactical, Contextual) and temporally aligned fine-grained punch events.
Novel Evaluation Protocols: Introduction of Category-Conditioned Generation and Commentary Rhythm Assessment, moving beyond simple captioning to evaluate stylistic control and temporal pacing.
Benchmarking & Insights: Comprehensive evaluation of state-of-the-art MLLMs (GPT-4o, LLaVA-OV, Qwen3-VL) reveals significant failures in combat sports narration, particularly in tactical reasoning and rhythm maintenance.
EIC-Gen Framework: Demonstrates that injecting structured, machine-detected action cues significantly improves model performance, highlighting the necessity of fine-grained event perception for combat sports.

4. Experimental Results

Category-Conditioned Generation:
- Zero-shot MLLMs performed poorly, with Content Consistency Accuracy (Acc) ranging from 10.9% to 23.7%.
- EIC-Gen Impact: Providing structured punch events (V+E) consistently improved accuracy. For the fine-tuned Qwen3-VL-8B model, Acc jumped from 25.0% (Video-only) to 30.0% (Video+Events).
- Tactical Reasoning: The most significant gains were seen in tactical commentary, where EIC-Gen boosted accuracy to 34.0%, proving that explicit event cues help models infer strategy.
Commentary Rhythm Assessment:
- Streaming models (LiveCC, StreamingVLM) failed to maintain human-like pacing, exhibiting low t-IoU scores (e.g., 0.044–0.119) and high KL Divergence.
- Qualitative analysis showed models often collapsed into repetitive outputs or failed to generate commentary as the match progressed, indicating a lack of temporal understanding.

5. Significance

This work fundamentally shifts the paradigm of sports commentary generation by:

Highlighting Domain Specificity: Proving that combat sports require distinct capabilities (sub-millisecond perception, high tactical density) compared to team sports.
Bridging Perception and Language: Demonstrating that for high-dynamic sports, raw visual encoding is insufficient; structured event extraction is a critical prerequisite for accurate, professional-level narration.
Establishing a New Standard: BoxComm provides the necessary infrastructure (data, taxonomy, and metrics) to drive future research in multimodal reasoning for fast-paced, subtle, and strategic domains.

The paper concludes that advancing MLLMs for combat sports requires not just better video encoders, but architectures capable of integrating fine-grained, structured action priors to overcome the "perception blindness" currently hindering tactical and rhythmic commentary generation.