🤖 AI

Interpretable Machine Learning for Football Performance Analysis: Evidence of Limited Transferability from Elite Leagues to University Competition

This study demonstrates that while machine learning models trained on elite European football data yield stable and consistent performance interpretations, these insights fail to transfer reliably to university-level competition, revealing that interpretability robustness is domain-dependent and that explanation instability can serve as a diagnostic signal for structural differences between competition levels.

Original authors: Yu-Fang Tsai, Yu-Jen Chen, Kok-Hua Tan, Sheng-Chieh Huang, You-Ying Ji, Yu-Lun Chen, Chun-Yi Wang, Chien-Ming Hsu

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yu-Fang Tsai, Yu-Jen Chen, Kok-Hua Tan, Sheng-Chieh Huang, You-Ying Ji, Yu-Lun Chen, Chun-Yi Wang, Chien-Ming Hsu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a master chef who has spent years perfecting a recipe for a world-class steak dinner. You know exactly which ingredients (the "performance determinants") make the difference between a 5-star meal and a mediocre one: the right cut of meat, the precise temperature, the specific seasoning. You are so confident in your recipe that you decide to teach it to a group of home cooks in a university cafeteria.

This paper is essentially asking: If you teach your elite steak recipe to university cooks, will the same ingredients still matter in the same way?

The researchers, a team from National Tsing Hua University in Taiwan, used a "machine learning" recipe (a computer program) to figure out what makes a football team win. They first taught the computer using data from the top five professional leagues in Europe (the "Elite" kitchen). Then, they asked the computer to look at data from university football matches (the "University" kitchen) to see if the same rules applied.

Here is what they found, broken down simply:

1. The "Elite" Kitchen is Very Consistent

When the computer analyzed the professional leagues, it found a very clear, stable hierarchy. It was like a choir where everyone sings the same note perfectly.

The Result: Whether the computer looked at the English Premier League or the Spanish La Liga, it agreed on the same "winning ingredients." For example, it consistently said things like "shooting efficiency" and "possession control" were the most important factors.
The Analogy: In the pro world, the rules of the game are so well-established that the computer's "explanation" of why a team won is always the same, no matter which professional league you look at.

2. The "University" Kitchen is Chaotic

When the researchers asked the computer to apply those same rules to the university team, the computer got confused. The "winning ingredients" suddenly changed their order.

The Result: In the university matches, things that were minor in the pro leagues suddenly became huge factors. For instance, simply "touching the ball" (possession) or "making saves" became much more important predictors of winning at the university level than they were for the pros.
The Analogy: It's as if the computer told the university cooks, "Forget the expensive steak! The secret to winning is actually how many times you stir the pot." The recipe didn't transfer; the structure of the game itself is different.

3. The Computer's "Confidence" Crumbled

The researchers didn't just look at what the computer thought was important; they also checked how sure the computer was. They ran the experiment multiple times with slight random changes (like changing the order of the ingredients list) to see if the computer would give the same answer every time.

In the Elite Kitchen: The computer was rock-solid. Even with random changes, it kept saying the same things were important.
In the University Kitchen: The computer became jittery. Sometimes it said "defense" was key; other times, it said "offense." The answer changed depending on how the computer was "woken up" that day.
The Analogy: In the pro league, the computer is like a seasoned judge who gives the same verdict every time. In the university league, the computer is like a nervous student who gives a different answer every time you ask the same question.

4. The "Translation" Failed

The researchers also used two different "languages" (methods) to ask the computer why it made its decisions.

In the Elite Kitchen: Both languages agreed perfectly. They both said, "The most important thing is X."
In the University Kitchen: The two languages started arguing. One said, "It's X!" and the other said, "No, it's Y!"
The Takeaway: This proves that the university game doesn't have a single, clear "structure" yet. The rules aren't as rigid as they are in the pros.

The Big Conclusion

The paper argues that you cannot simply take a model trained on professional sports and assume it works for amateur or university sports.

The researchers suggest that when a computer's explanation becomes unstable or changes its mind (like it did with the university data), it's not necessarily a bug in the computer. Instead, it's a diagnostic signal. It's the computer telling us: "Hey, the structure of this game is messy and undefined. There isn't one single 'right' way to win here yet."

In short:

Pro Football: Like a symphony orchestra. Everyone plays the same sheet music. The rules are clear, and the "winning formula" is stable.
University Football: Like a jam session. Everyone is improvising. The "winning formula" changes from game to game, and trying to force a pro-level recipe onto it doesn't work.

The paper concludes that before we use AI to give advice to university teams, we need to realize that the "logic" of the game is different at that level, and the AI's explanations are much less reliable there than they are for the pros.

Technical Summary: Interpretable Machine Learning for Football Performance Analysis

Problem Statement
Machine learning has become integral to football performance analysis, yet existing research predominantly prioritizes predictive accuracy while implicitly assuming that performance determinants learned from elite competitions are transferable to lower-level contexts. This assumption remains largely unexamined, particularly regarding the "domain shift" from elite professional leagues to university-level competition. University football operates under distinct conditions, including limited training time, heterogeneous skill levels, and variable team compositions. The core problem addressed is whether the structural relationships between performance indicators and match outcomes, along with the interpretability of machine learning models, remain robust when applied across these divergent competitive levels. Specifically, the study investigates if explanations derived from elite data are reliable when deployed on university data, or if instability in these explanations reflects genuine structural ambiguity in the target domain.

Methodology
The study employs an "elite-to-university" transfer framework where models are trained exclusively on elite data and applied to university data solely for inference and interpretability evaluation.

Data Sources:
- Source Domain (Elite): Match event data from the top five European leagues (Premier League, La Liga, Serie A, Bundesliga, Ligue 1) covering three seasons (2022/23–2024/25). Data was retrieved via the soccerdata Python package from WhoScored, aligned to Opta-style definitions.
- Target Domain (University): 17 official matches from the National Tsing Hua University (NTHU) football team (Taiwan's second division). Events were manually annotated by three certified referees following OPTA guidelines, with final counts determined by majority agreement.
Feature Representation: A fixed set of 19 event categories (e.g., shots, passes, tackles, fouls) was used. The prediction target was defined as the goal difference ( $y = \text{Goal} - \text{Enemy Goal}$ ), formulated as a regression task.
Models: Six machine learning algorithms were initially evaluated (Decision Tree, MLP, Random Forest, SVR, XGBoost, TabNet). Based on predictive stability and low variance across random seeds, Random Forest (RF) and Multilayer Perceptron (MLP) were selected for all subsequent interpretability analyses.
Interpretability Methods: Two conceptually distinct methods were used to evaluate feature importance:
1. SHAP (SHapley Additive exPlanations): Used to estimate marginal contributions of features to predictions.
2. Counterfactual Impact Score (CIS): Measured prediction changes under controlled, directional single-feature perturbations based on elite training statistics.
Experimental Design: Five experiments were conducted to assess:
1. Predictive validity on elite data.
2. Cross-domain feature importance comparison (Elite vs. NTHU).
3. Seed-wise reproducibility of rankings.
4. Structural agreement across domains (correlation of rankings between leagues and university).
5. Agreement between explanation methods (SHAP vs. CIS) within domains.

Key Results

Structural Divergence: Performance determinants exhibit a stable, shared hierarchy across elite leagues but undergo substantial reordering when applied to university football. For instance, "Touches" and "Saves" shifted significantly in rank importance in the university domain compared to the elite domain.
Explanation Stability: In the elite domain, feature importance rankings were highly reproducible across random seeds (e.g., RF Spearman correlation $\mu = 0.990$ ). In contrast, the university domain showed reduced stability, particularly for the MLP model ( $\mu = 0.231$ ), indicating that explanation structures are sensitive to training randomness in the target domain.
Cross-Domain Agreement: Strong structural agreement (high Spearman correlations) was observed among elite leagues. However, correlations between elite leagues and NTHU university football were substantially lower, confirming a lack of structural alignment.
Method Invariance: In elite football, SHAP and CIS rankings showed high agreement ( $\rho \approx 0.93$ for RF). In the university domain, this agreement dropped significantly ( $\rho \approx 0.42$ for RF) and lacked statistical consistency, suggesting that feature importance rankings in university football are highly dependent on the choice of explanation method.

Key Contributions

Empirical Evidence of Non-Transferability: The study provides evidence that performance determinants learned from elite competitions do not preserve their structural hierarchy when applied to university-level football, challenging the assumption of cross-level transferability.
Domain-Dependent Interpretability: It demonstrates that interpretability is not an inherent property of a model but a domain-dependent characteristic. While robust in elite contexts, interpretability becomes unstable and method-sensitive under domain shift.
Reframing Instability: The paper proposes a conceptual shift where explanation instability is not viewed merely as methodological noise or model failure, but as a diagnostic signal of structural ambiguity in the target domain. It suggests that in contexts where tactical hierarchies are not yet well-established (like university football), the lack of a stable explanation structure is a meaningful finding in itself.

Significance and Claims
The authors claim that this work shifts the focus of football analytics from purely predictive performance to the empirical evaluation of interpretability robustness. The study argues that before translating model insights into actionable strategies for non-elite contexts, practitioners must evaluate whether explanations are stable, transferable, and method-invariant.

The paper concludes that explanation instability and method disagreement should be treated as indicators of structural uncertainty rather than artifacts. Consequently, insights derived from elite-trained models should not be directly applied to university or amateur settings without rigorous validation of interpretability robustness. The findings underscore the necessity of treating interpretability as a conditional, data-dependent property rather than a guaranteed feature of machine learning models.