Cross-individual generalizability of machine learning models for ball speed prediction in baseball pitching

Technical Summary: Cross-individual Generalizability of Machine Learning Models for Ball Speed Prediction in Baseball Pitching

Problem Statement
While machine learning (ML) has demonstrated high predictive accuracy for sports performance outcomes (e.g., ball speed, jump height) when trained and tested on data from the same individuals, the cross-individual generalizability of these models remains poorly understood. Athletic movements involve complex, nonlinear inter-joint coordination with significant inter-individual variability. Consequently, models that perform well within a specific athlete's data often fail to generalize to unseen individuals. Previous studies frequently conflated training and test data from the same athletes or lacked clear data partitioning, leading to potential overestimation of model utility in real-world settings. This study addresses the gap by rigorously evaluating how well ML models trained on a group of pitchers can predict ball speed for entirely new, unseen pitchers.

Methodology
The study utilized a dataset of 50 pitchers across five competitive levels (high school, collegiate, industrial, independent, and professional). Motion data were captured using 16 optical cameras (200–500 Hz), and ball velocity was recorded via TrackMan.

• Data Preprocessing: 15 anatomical landmarks were tracked. Data were filtered, time-normalized to 101 frames (spanning 1.0s pre-release to 0.2s post-release), and mirrored for left-handed pitchers.
• Model Architecture: Two baseline architectures were evaluated: a Graph Neural Network–Recurrent Unit (GNN-GRU) and a Transformer. Six hyperparameter configurations were tested for each.
• Validation Strategy: The primary evaluation method was Leave-One-Subject-Out Cross-Validation (LOSOCV). In this setup, all pitches from one pitcher served as the test set, while data from the remaining 49 pitchers formed the training set. This was repeated 50 times.
• Comparative Analysis:
  1. Within-Individual vs. Cross-Individual: A subset of the data was split to compare performance when training and testing on the same individuals versus different individuals.
  2. Expertise Effects: Pitchers were categorized into "Intermediate" (high school/collegiate) and "Expert" (industrial/independent/professional) groups to analyze if skill level influenced prediction error magnitude and direction.
  3. Spatiotemporal Ablation: The study systematically restricted input data to specific body regions (throwing arm, leading arm, trunk, pivot leg, leading leg) and cumulative time windows to identify which spatiotemporal features drive generalizability.

Key Results

• Performance Decline: There was a substantial drop in predictive performance when moving from within-individual to cross-individual evaluation. The coefficient of determination ($R^2$) decreased from 0.91 (within-individual) to 0.38 (cross-individual).
• Baseline Model Selection: Among the candidates, a Transformer model with 4 attention heads, a latent dimension of 64, and a feedforward dimension of 128 achieved the highest cross-individual $R^2$ (0.38) and was selected for subsequent analyses.
• Expertise Bias: While the magnitude of absolute error did not differ significantly between expert and intermediate groups, the signed prediction error did ($p < 0.05$). The model tended to overestimate the ball speed of intermediate pitchers compared to experts. This suggests that for similar kinematic inputs, intermediate pitchers may produce lower output speeds due to lower movement efficiency, a factor the model did not fully account for.
• Spatiotemporal Factors:
  • Body Regions: The trunk and pivot leg demonstrated the highest cross-individual generalizability. Conversely, the leading arm showed the lowest.
  • Temporal Phases: The pivot leg exhibited significant generalizability even during the weight-shift initiation phase ($R^2 > 0.25$). Generalizability for the trunk, leading leg, and leading arm tended to increase as the motion progressed toward ball release, whereas the pivot leg and throwing arm showed U-shaped performance profiles over time.

Significance and Claims
The authors claim that this study underscores the critical necessity of cross-individual evaluation to accurately assess the practical applicability of ML in sports science. The findings suggest that:

1. Generalizability is Limited but Informative: While predictive power drops significantly across individuals, the remaining signal is not random; it reveals biomechanical consistencies.
2. Skill Level Matters: Models may systematically misestimate performance based on skill level, potentially due to differences in movement efficiency. This deviation could be used to quantify individual-specific abilities.
3. Core Biomechanical Drivers: The trunk and pivot leg appear to be the most consistent predictors of ball speed across different individuals, even in early movement phases. This aligns with the biomechanical understanding that these segments initiate the kinetic chain and are less variable in their functional role compared to distal segments like the arms.

The paper concludes that future ML applications in sports must move beyond individual-level validation to understand the biomechanical determinants of performance and to avoid overestimating model capabilities in real-world, diverse populations.