Technical Summary: SPRINT – Sports Proactive Risk INference Testbed

1. Problem Statement

Current Multimodal Large Language Models (MLLMs) are increasingly evaluated on their ability to detect harmful content or adversarial vulnerabilities. However, a critical gap remains in proactive physical hazard prediction: the capacity to anticipate emerging physical dangers from evolving visual evidence before an accident occurs. Existing benchmarks focus on reactive detection (identifying harm after it happens) or static safety advice, failing to address the temporal dynamics required for real-world safety interventions (e.g., autonomous emergency braking, fall prevention).

The core challenge is twofold:

1. Temporal Anticipation: Hazards often emerge through subtle, gradual cues (e.g., unstable posture, trajectory deviations) rather than a single salient frame.
2. Causal Grounding: Models must not only signal danger but identify the specific physical causes (e.g., equipment failure vs. loss of balance) to enable effective intervention.

Current research lacks a systematic evaluation framework that combines real-world dynamic video data with fine-grained temporal and causal annotations to test these specific capabilities.

2. Methodology

2.1 The SPRINT Benchmark

The authors introduce SPRINT (Sports Proactive Risk INference Testbed), a benchmark designed to evaluate proactive safety reasoning.

• Dataset Composition: 2,888 real-world sports videos sourced from YouTube, comprising:
  • 2,440 Accident Videos: Spanning 14 sports categories (e.g., Gymnastics, Parkour, Skiing) and 3 environmental settings (Standardized Venues, Unstructured Human-made Spaces, Natural & Wilderness Terrain).
  • 448 Safe Control Videos: Manually verified as accident-free, used to diagnose false alarms.
• Annotation Schema:
  • Temporal: Two key timestamps per accident video:
    • $T_1$ (Earliest Cue Moment): The first perceptible abnormal visual cue (e.g., a tilted body axis).
    • $T_2$ (Most Obvious Moment): The irreversible accident moment (e.g., impact).
  • Causal: Hierarchical cause annotations:
    • $H_1$ (Macro Factors): Aligned with the Haddon Matrix (Personal, Interpersonal, Equipment/Facility, Venue/Environmental).
    • $H_2$ (Direct Cause): Natural language descriptions of the immediate trigger.
  • Safe Video Annotation: Annotated with the moment of maximum motion intensity to simulate high-risk scenarios without actual danger.

2.2 Evaluation Protocol

The evaluation assesses MLLMs across three progressive dimensions:

1. D1 (Hazard Detection): Does the model explicitly indicate a current or imminent hazard?
2. D2 (Factor Coverage): Does the model identify the correct macro-level inducing factor ($H_1$)?
3. D3 (Cause Specificity): Does the model accurately articulate the direct physical cause ($H_2$)?

Experimental Designs:

• Full-Video Evaluation: Testing hazard perception with prompts ranging from descriptive to explicit safety inquiries.
• Temporal Window Evaluation: Truncating videos into three windows (Start to $T_1$, Start to mid-point, Start to $T_2$) to test early-warning capabilities under varying levels of visual evidence.
• Diagnostic Experiments: Using safe control videos to measure False Positive Rates under explicit danger queries versus neutral descriptions, isolating prompt-induced bias from genuine visual reasoning.

2.3 Models Evaluated

The study benchmarks state-of-the-art closed-source models (Doubao-seed-1.8, Gemini 3 Flash/Pro, GPT-4o, GPT-5) and open-source models (InternVL-3.5-8B, Qwen3-VL-8B).

3. Key Results

3.1 The Sensitivity-Understanding Gap

A sharp disparity exists between hazard detection and causal understanding:

• D1 (Detection): Top models (e.g., Doubao-seed-1.8) achieve >95% accuracy in signaling hazards when the full video is provided.
• D3 (Causality): The same models drop below 50% in identifying specific causes.
• Conclusion: Models rely on "superficial proactive safety"—detecting visual anomalies associated with accidents but lacking deep physical causal reasoning.

3.2 Prompt Sensitivity and False Alarms

• Prompt Dependency: Performance is highly sensitive to prompt phrasing. Under neutral prompts (e.g., "Describe the situation"), D1 scores for top models drop significantly (e.g., Doubao-seed-1.8 drops from 88% to 59% in early windows).
• Prompt-Induced Bias: Diagnostic tests on safe videos reveal severe false alarms. When prompted with "Is there any danger?", models like GPT-5 and Doubao-seed-1.8 exhibit false-positive rates jumping from ~0.13–0.28 to ~0.59–0.88. This suggests models are triggered by the word "danger" rather than visual evidence.

3.3 Temporal Anticipation

• Early Warning Failure: In the earliest temporal window (Window A, up to $T_1$), performance degrades drastically, especially under neutral prompts. Open-source models fall below 9% D1 accuracy.
• Causal Reasoning: D3 scores remain poor (<0.30) even in later windows, indicating a fundamental inability to ground warnings in specific physical causes.

3.4 Fine-Tuning Utility

Supervised fine-tuning (SFT) using SPRINT annotations on Qwen3-VL-8B yielded significant improvements:

• D1 rose from ~0.48 to 0.99.
• D3 more than tripled (0.16 to 0.52).
• False positives on safe videos decreased, though prompt-induced bias persisted in some conditions. This demonstrates SPRINT's viability as a training resource.

4. Key Contributions

1. SPRINT Benchmark: A dataset of 2,888 real-world sports videos with fine-grained temporal ($T_1, T_2$) and hierarchical causal ($H_1, H_2$) annotations, specifically designed for proactive risk inference.
2. Systematic Evaluation Protocol: A framework covering hazard detection, early-warning timeliness, causal attribution, and false-alarm control, distinguishing between reactive and proactive safety capabilities.
3. Empirical Evidence of Limitations: Comprehensive results showing that current MLLMs exhibit only superficial proactive safety, characterized by high sensitivity to explicit prompts, poor causal grounding, and severe false-alarm rates in hazard-free scenarios.

5. Significance and Claims

The paper argues that current MLLMs lack stable, cause-grounded early warning capabilities necessary for dynamic physical environments. The findings underscore that:

• Superficiality: Models detect "danger" as a visual pattern rather than understanding the underlying physical dynamics.
• Prompt Fragility: Safety behaviors are often artifacts of prompt engineering rather than robust reasoning.
• Need for Shift: Future research must move beyond anomaly detection toward reliable, cause-grounded proactive safety.

The authors position SPRINT as a necessary step to bridge the gap between theoretical safety alignment and practical, real-world hazard anticipation, noting that sports provide a theoretically grounded proxy (via the Haddon Matrix) for broader safety domains like autonomous driving and fall detection.

6. Limitations

The paper explicitly acknowledges several constraints:

• Modality: The benchmark evaluates visual-textual interactions only, omitting audio cues (e.g., collision sounds).
• Scope: Evaluation is limited to general-purpose MLLMs, excluding specialized video anomaly detection models.
• Subjectivity: Temporal boundary annotations ($T_1, T_2$) retain inherent subjectivity despite quality control.
• Annotation Structure: Direct causes ($H_2$) are free-text, lacking a structured kinematic taxonomy, which may limit precision in causal evaluation.